Thursday, August 27, 2015

Performance Tuning JVM

Performance tuning is no easy task, and when you are trying to improve application there are many aspects to it. The various levels which can be tuned include: Operating System, Application Server, Database, and Application Configurations/Code. The first place to look when planning to improve performance is to improve the performance of your algorithms, this can be in the application code whether it's in the database or not. Having an algorithm that runs O(n^3) or polynomial then finding a way to reduce it to a order to linear or logarithmic this will beat most any other changes that anywhere else that could be made. However here we will talk specifically about the JVM when running Oracle WebLogic 12c.

Memory Options
If you've had any experience with the JVM then you likely know about a few java options available to you. The usual ones people set are: -Xms, -Xmx, and -XX:MaxPermSize. One thing to note is that with Java 8 PermGen has been replaced with Metaspace as mentioned in this post. While the official minimum amount of memory recommended for WebLogic 12c is 4GB, I would recommend at least 8GB for most applications, but many cases there may be a need for even more memory. This is completely dependent upon the applications running within WebLogic. The second recommendation is that whatever the amount of memory is designated for the maximum amount of memory through Xmx, the same amount should be set for the minimum using Xms . This saves the JVM from wasting time cycles on deallocating memory back to the system.
With respect to MetaSpace a similar option is now available -XX:MaxMetaspaceSizewhich will allow a limit on the new space. Just as with the JVM Heap we want to limit the size of the Metaspace usually a 1G is plenty of space and depending on your application may even be overkill. The important part however is to make sure that there is a limit. Now that we have our various spaces in memory we need to look at how to manage it.

Garbage Collectors
There are a number of different garbage collectors available including: Concurrent Mark Sweep (CMS), ParallelGC, and Garbage First (G1GC). G1GC the newest of the bunch, and it certainly has a lot of advantages many of which can be found here. However, it is still not ready for the lime light. It was introduced in an update to Java 7, and is currently targeted to be the default for Java 9; it could still be pushed back if it still isn't considered stable enough. Concurrent Mark Sweep is an older collector is used to focus on lower pause times which G1GC is planned to replace, and ParallelGC is considered the throughput collector. Once again there is no really solid answer as to which you should use, but from working with clients, ParallelGC has been the most dependable when it came to heavier load tests. While I wouldn't say to use G1GC right now, there are some fixes coming which may change my mind in the future. To use any of these garbage collectors you will need the following commands:
G1GC: -XX:+UseG1GC
ParallelGC: -XX:+UseParallelGC
CMS: -XX:+UseConcMarkSweepGC

Once a garbage collector is selected there are a number of options specific to them which can be explored, and should be explored to find further improvements.

Additional Options
While there are many tweakable options, there are only a few I would look into beside the ones mentioned above. -XX:+ScavengeBeforeFullGC is a setting that deals with performing a scavenge before a full garbage collection, and I would almost alway use unless there is a specific performance hit seen from its use. -XX:MaxTenuringThreshold can be tweaked to different numbers and can be useful depending on long your application tends to hold onto certain information. -XX:+AlwaysPreTouch prepares handles for paging in the beginning, and while it has a performance hit on startup tends to result in quicker memory accesses later. Finally, -XX:+ParallelRefProcEnabled allows for parallel as opposed to serial reference processing as part of the garbage collection process, another flag that I would highly recommend enabling. 

Happy Tuning!

Thursday, August 6, 2015

Difference between WebLogic Server "DbAdapter" and "Data Source" connection pool parameters

For users of Oracle SOA Suite 11g/12c, this blog will attempt to answer whether you should configure your connection pool settings at the adapter level or at the data source level.


Data Source Settings

On Oracle WebLogic Server 11g/12c, navigate to Data Sources > SOADataSource > Configuration > General.

Here, you can obtain the JNDI, such as jdbc/SOADataSource. This JNDI is generally handed to your developers so that they can use it from within their Java code. Keep in mind that this JNDI is also used by the DbAdapter (explained later).


Now click on the Connection Pool tab. Here, you can see the various settings of the connection pool such as the initial and maximum number of connections which you can adjust as necessary.



DbAdapter Settings

Now navigate to Deployments > DbAdapter > Configuration > Outbound Connection Pools > [expand javax.resource.cci.ConnectionFactory] > eis/DB/SOADemo > General. Here, you can obtain the JNDI, such as eis/DB/SOADemo, which is passed on to your SOA developers. Your SOA developers typically use this from within their SOA or OSB code.

(Note that Oracle SOA Suite deploys various JCA adapters as part of the installation of the product.)



Now click on the Properties tab. As you can see here, the connection factory is really based off of the data source (i.e., you are required to reference a data source).



Finally, click on the Connection Pool tab. Here, you also have initial and maximum connection pool settings which you can adjust as necessary.



The Question

The "Max Capacity" setting under the connection factory's connection pool is set to 1000 and under the data source's connection pool is set to 20. Which one actually takes effect?



The Explanation

The JCA connection factory settings and data source settings are totally different settings.

JCA connection pools are to connect to JCA Adapters (such as the DbAdapter, JmsAdapter, etc.). Data Source connection pools are to connect to the database.

When the "Max Capacity" setting under the connection factory's connection pool is set to 1000, this means that you have 1000 available connections to connect to the JCA adapters. This means that your SOA or OSB code has up to 1000 connections to the JCA adapter (not the database).

When the data source's connection pool is set to a maximum value of 20, this means that you have a maximum of 20 connections to connect to the Database.

Generally, the default settings of the JCA connection factory should be sufficient. However, you may need to adjust your Data Source connection pool settings depending on your needs.



Applicable Versions:
  • Oracle WebLogic Server 11g/12c
  • Oracle SOA Suite 11g/12c


Tuesday, August 4, 2015

How to Use The Oracle Maven Repository

With the push of a lot of the 12c Series of products Oracle released it's own Maven repository. They made it functionally public a number of months ago, and I have been able to use it without incident since that time. Along with their release of the repository were instructions on how to use it here. The instructions however over explained in some areas and were a little convoluted in others. As a result I have decided to release a straight forward set of instructions.

Step 0: Maven Configurations
Of course I would say that it's going to be straight forward and then make things a little convoluted. As a matter of preference I like to keep my settings local to me rather than in the global settings file for maven that way between maven upgrades or in a shared environment I can bring my settings with me.

Global Maven Settings are stored in:
  Linux/Mac: $M2_HOME/conf/settings.xml
  Windows: %M2_HOME%\conf\settings.xml

Personal Settings are stored in:
  Linux/Mac: ~/.m2/settings.xml
  Windows: C:\Documents and Settings\<your-username>\.m2\settings.xml

If you want to start your own personal settings file the easiest way is to copy the it from the global. For the remainder of this tutorial I will assume you're working with Personal Settings

Step 1: Setup Maven Security
When connecting to authenticated Maven repositories you need to put your passwords into either your project POM file or into your settings.xml files. Either way your passwords and on your filesystem. To get this started first a master password needs to established and then your passwords in the future can be encrypted. Luckily Apache has a very simple article documenting these steps http://maven.apache.org/guides/mini/guide-encryption.html

Step 2: Configure Server
In the settings.xml add the following to the <servers> section:
    <server>
      <id>maven.oracle.com</id>
      <username>{USERNAME}</username>
      <password>{/ENCRYPTED_PASSWORD}</password>
      <configuration>
        <basicAuthScope>
          <host>ANY</host>
          <port>ANY</port>
          <realm>OAM 11g</realm>
        </basicAuthScope>
        <httpConfiguration>
          <all>
            <params>
              <property>
                <name>http.protocol.allow-circular-redirects</name>
                <value>%b,true</value>
              </property>
            </params>
          </all>
        </httpConfiguration>
      </configuration>
    </server>
In place of the {USERNAME} should be your OTN email, and in place of {/ENCRYPTED_PASSWORD} should be the encrypted version of your password which you should now know how to generate from the Apache guide.

Step 3: Configure Profile
In the settings.xml add the following to the <profiles> section:
    <profile>
      <id>default</id>
      <activation>
        <activeByDefault>true</activeByDefault>
      </activation>
      <repositories>
        <repository>
          <id>maven.oracle.com</id>
          <releases>
            <enabled>true</enabled>
          </releases>
          <snapshots>
            <enabled>false</enabled>
          </snapshots>
          <url>https://maven.oracle.com</url>
          <layout>default</layout>
        </repository>
      </repositories>
      <pluginRepositories>
        <pluginRepository>
          <id>maven.oracle.com</id>
          <url>https://maven.oracle.com</url>
        </pluginRepository>
      </pluginRepositories>
    </profile>

Step 5: Accept Terms and Conditions
With Oracle of course anything you want to download from them there is some sort of agreement, and the Maven Repository is not different. At lease with this one you only need to do it once. Go to https://www.oracle.com/webapps/maven/register/license.html and accept the terms.

Success
Provided that you followed all of these steps you should easily be able use oracle libraries as dependencies in your next project.

Saturday, August 1, 2015

JVM failed to start in GlassFish Server 4.1

Problem:

When trying to start up the domain in Oracle GlassFish Server 4.1, we get the following error:
[glassfish@demo logs]$ $GLASSFISH_HOME/bin/asadmin start-domain 
JVM failed to start: com.sun.enterprise.admin.launcher.GFLauncherException: The server exited prematurely with exit code 1.
Before it died, it produced the following output:
Absolutely nothing is showing up in the $GLASSFISH_HOME/glassfish/domains/domain1/logs/server.log log file.


Solution:

1. Double-check the startup parameters of the JVM in the domain.xml file for invalid entries.


In our example, the $GLASSFISH_HOME/glassfish/domains/domain1/config/domain.xml had the following invalid entry:
<jvm-options>-myUrl=https://dev.raastech.com</jvm-options>
When the correct format should have been as follows:
<jvm-options>-DmyUrl=https://dev.raastech.com</jvm-options>


Applicable Versions:
  • Oracle GlassFish Server 4.1


Monday, July 20, 2015

ORA-17008 when running the RCU

Problem:

When running the RCU (Repository Creation Utility) to create the SOA Suite 12c schemas, you may get the following error:
ORA-17008: Closed ConnectionFile:/u01/share/oracle/middleware/products/fmw1213/soa/common/sql/soainfra/sql/oracle/createschema_soainfra_oracle_LARGE.sql
Statement: ALTER TABLE B2B_DATA_STORAGE ADD CONSTRAINT B2B_DATA_STORAGE_PK PRIMARY KEY (ID)...




Analysis:

Check out the RCU log file /u01/share/oracle/middleware/products/fmw1213/oracle_common/rcu/log/logdir.2015-07-20_11-02/rcu.log and you will find the same error in the logs:
2015-07-20 11:11:38.953 ERROR rcu: oracle.sysman.assistants.rcu.backend.action.AbstractAction::handleNonIgnorableError: Received Non-Ignorable Error: ORA-17008: Closed ConnectionFile:/u01/share/oracle/middleware/products/fmw1213/soa/common/sql/soainfra/sql/oracle/createschema_soainfra_oracle_LARGE.sql
Statement:ALTER TABLE B2B_DATA_STORAGE
    ADD CONSTRAINT B2B_DATA_STORAGE_PK PRIMARY KEY (ID)

2015-07-20 12:47:39.352 ERROR rcu: oracle.sysman.assistants.common.dbutil.jdbc.JDBCEngine::onException: SQLException: Closed Connection
java.sql.SQLRecoverableException: Closed Connection
        at oracle.jdbc.driver.PhysicalConnection.getMetaData(PhysicalConnection.java:3131)
        at...
However, the error is not due to the SQLException: Closed Connection you see above.

Now check out the next log file /u01/share/oracle/middleware/products/fmw1213/oracle_common/rcu/log/logdir.2015-07-20_11-02/soainfra.log and view its contents:
2015-07-20 11:11:38.949 rcu:Extracted SQL Statement: [CREATE TABLE B2B_DATA_STORAGE
(
    VALUE_SELECTOR VARCHAR2(256),
    CLOB_VALUE CLOB,
    BLOB_VALUE BLOB,
    DATA_SIZE NUMBER(10,0),
    ATTRIBUTE1 VARCHAR2(256),
    ATTRIBUTE2 VARCHAR2(256),
    LABEL VARCHAR2(256),
    DOCUMENT_ID VARCHAR2(200),
    JOB_ID VARCHAR2(128),
    ID VARCHAR2(256) NOT NULL,
    CPST_INST_CREATED_TIME TIMESTAMP DEFAULT systimestamp-30,
    IS_OBFUSCATED NUMBER(1)
)
 PARTITION BY RANGE (CPST_INST_CREATED_TIME)
 INTERVAL(NUMTOYMINTERVAL(1, 'MONTH'))
 (PARTITION p0 VALUES LESS THAN (TO_DATE('1-2-2007', 'DD-MM-YYYY')))
]
2015-07-20 11:11:38.949 rcu:Statement Type: 'DDL Statement'
JDBC SQLException - ErrorCode: 439SQLState:67000 Message: ORA-00439: feature not enabled: Partitioning

JDBC SQLException handled by error handler
Here, you can clearly see an ORA-00439: feature not enabled: Partitioning error when the actual DDL statement is executed. This error indicates that partitioning is not enabled in the database that we are trying to run the RCU against.



Solution:

1. In our case, during the RCU installation wizard, do not choose the database profile value of LARGE, and instead choose SMALL (note that this value is case sensitive).

Alternatively, we could upgrade the database to support partitioning.


2. Check all logs (not just the rcu.log) in the RCU logs directory.



Applicable Versions:
  • Oracle Repository Creation Utility (RCU) 12c (12.1.3)
  • Oracle SOA Suite 12c (12.1.3)



Wednesday, July 8, 2015

BEA-090898 Ignoring the trusted CA certificate in OSB 11g due to "Unsupported OID in the AlgorithmIdentifier"

Problem:

Testing a service in the OSB console resulted in the following error in our osb_server1.out log file:
<Jul 6, 2015 2:00:27 PM EDT> <Notice> <Security> <BEA-090898> <Ignoring the trusted CA certificate "CN=Entrust Root Certification Authority - G2,OU=(c) 2009 Entrust\, Inc. - for authorized use only,OU=See www.entrust.net/legal-terms,O=Entrust\, Inc.,C=US". The loading of the trusted certificate list raised a certificate parsing exception PKIX: Unsupported OID in the AlgorithmIdentifier object: 1.2.840.113549.1.1.11.>

<Jul 6, 2015 2:00:27 PM EDT> <Notice> <Security> <BEA-090898> <Ignoring the trusted CA certificate "CN=thawte Primary Root CA - G3,OU=(c) 2008 thawte\, Inc. - For authorized use only,OU=Certification Services Division,O=thawte\, Inc.,C=US". The loading of the trusted certificate list raised a certificate parsing exception PKIX: Unsupported OID in the AlgorithmIdentifier object: 1.2.840.113549.1.1.11.>

<Jul 6, 2015 2:00:27 PM EDT> <Notice> <Security> <BEA-090898> <Ignoring the trusted CA certificate "CN=T-TeleSec GlobalRoot Class 3,OU=T-Systems Trust Center,O=T-Systems Enterprise Services GmbH,C=DE". The loading of the trusted certificate list raised a certificate parsing exception PKIX: Unsupported OID in the AlgorithmIdentifier object: 1.2.840.113549.1.1.11.>

<Jul 6, 2015 2:00:27 PM EDT> <Notice> <Security> <BEA-090898> <Ignoring the trusted CA certificate "CN=T-TeleSec GlobalRoot Class 2,OU=T-Systems Trust Center,O=T-Systems Enterprise Services GmbH,C=DE". The loading of the trusted certificate list raised a certificate parsing exception PKIX: Unsupported OID in the AlgorithmIdentifier object: 1.2.840.113549.1.1.11.>

<Jul 6, 2015 2:00:27 PM EDT> <Notice> <Security> <BEA-090898> <Ignoring the trusted CA certificate "CN=GlobalSign,O=GlobalSign,OU=GlobalSign Root CA - R3". The loading of the trusted certificate list raised a certificate parsing exception PKIX: Unsupported OID in the AlgorithmIdentifier object: 1.2.840.113549.1.1.11.>

<Jul 6, 2015 2:00:27 PM EDT> <Notice> <Security> <BEA-090898> <Ignoring the trusted CA certificate "OU=Security Communication RootCA2,O=SECOM Trust Systems CO.\,LTD.,C=JP". The loading of the trusted certificate list raised a certificate parsing exception PKIX: Unsupported OID in the AlgorithmIdentifier object: 1.2.840.113549.1.1.11.>

<Jul 6, 2015 2:00:27 PM EDT> <Notice> <Security> <BEA-090898> <Ignoring the trusted CA certificate "CN=VeriSign Universal Root Certification Authority,OU=(c) 2008 VeriSign\, Inc. - For authorized use only,OU=VeriSign Trust Network,O=VeriSign\, Inc.,C=US". The loading of the trusted certificate list raised a certificate parsing exception PKIX: Unsupported OID in the AlgorithmIdentifier object: 1.2.840.113549.1.1.11.>

<Jul 6, 2015 2:00:27 PM EDT> <Notice> <Security> <BEA-090898> <Ignoring the trusted CA certificate "CN=KEYNECTIS ROOT CA,OU=ROOT,O=KEYNECTIS,C=FR". The loading of the trusted certificate list raised a certificate parsing exception PKIX: Unsupported OID in the AlgorithmIdentifier object: 1.2.840.113549.1.1.11.>

<Jul 6, 2015 2:00:27 PM EDT> <Notice> <Security> <BEA-090898> <Ignoring the trusted CA certificate "CN=GeoTrust Primary Certification Authority - G3,OU=(c) 2008 GeoTrust Inc. - For authorized use only,O=GeoTrust Inc.,C=US". The loading of the trusted certificate list raised a certificate parsing exception PKIX: Unsupported OID in the AlgorithmIdentifier object: 1.2.840.113549.1.1.11.>

Solution:

1. Enable the "Use JSSE SSL" option on the OSB managed servers (e.g., osb_server1, osb_server2).


References:



Applicable Versions:
  • Oracle Service Bus (OSB) 11g (10.3.6+)



Sunday, July 5, 2015

Provisioning an AWS instance for general Oracle software

This blog post describes how to provision a barebones Linux server from Amazon Web Services (AWS) for the purpose of installing general Oracle software.

This instructions optionally include adding an extra mount point, enabling NFS, and installing VNC for remote GUI connectivity.


Create an EC2 Instance

1. Login to the AWS Management Console at:

https://us-west-2.console.aws.amazon.com/console/home

2. Click on "EC2" on the left-hand menu



3. Navigate to Instances > Launch Instance

4. "Select" the Red Hat Enterprise Linux 7.1 instance type


5. Select an instance type (e.g., t2.medium)


6. Click on "Next: Configure Instance Details"


7. Keep all defaults

8. Click on "Next: Add Storage"


9. Click on "Add New Volume"

10. Enter the size of "10" for 10GB


11. Click on "Next: Tag Instance"

12. Enter the name of your instance (just a name, e.g., "My Oracle Server")


13. Click on "Next: Configure Security Group"


14. Provide a Security Group name and description, and add the firewall rules you want to allow


15. Click on "Review and Launch"



16. Click on "Launch" after reviewing the settings



17. Enter a new key pair name, download key pair, and click on "Launch Instances"



18. Click on "View Instances" and the instance should be up in a few minutes




Create an Elastic IP

19. Click on "Elastic IP" on the left-hand menu

20. Click on "Allocate New Address"


21. Click on "Yes, Allocate"

22. Click on "Close"

23. Right-click on the IP address and select "Associate Address"

24. Click on the Instance field and select your EC2 instance from the list



Create a Putty PPK File for SSH Access

25. Follow these instructions to convert the downloaded .pem file to a Putty compatible .ppk file


26. Login to the server via Putty or equivalent SSH client as the "ec2-user"

27. Then type sudo su - to login as the root user




















Setup the Linux Server

28. Install some basic Oracle required packages

yum install telnet
yum install wget
yum install gcc
yum install gcc-c++
yum install glibc-devel
yum install libaio
yum install libaio-devel
yum install sysstat
yum install libstdc++-devel
yum install compat-libstdc++
yum install compat-libstdc

29. Define a hostname

hostname oradev.raastech.com

30. Type the following to get your local IP address

ifconfig -a | grep broadcast | awk '{print $2}'

31. Edit the local hosts by typing vi /etc/hosts file and manually add your public and private IP addresses and a hostname of your choosing

52.27.XXX.XXX    oradev-ext.raastech.com   oradev-ext
172.31.XXX.XXX   oradev.raastech.com       oradev

32. Create the Oracle unix user

groupadd oinstall
groupadd dba
useradd nobody
useradd -c "Oracle Software Owner" -g oinstall -G dba oracle
passwd oracle

33. Edit the profile for both the 'root' and 'oracle' users to something more readable

vi /root/.bash_profile /home/oracle/.bash_profile

34. Add the following and save the file

export PS1="\u@\h:\$PWD> "
alias ls='ls'
alias ll='ls -l'

35. Edit the sysctl file

vi /etc/sysctl.conf

36. Add the following and save the file, updating the hostname and domainname accordingly

# Controls the maximum shared segment size, in bytes
kernel.shmmax = 68719476736
# Controls the maximum number of shared memory segments, in pages
kernel.shmall = 4294967296
# Controls the default maxmimum size of a mesage queue
kernel.msgmax = 65536
# Controls the maximum size of a message, in bytes
kernel.msgmnb = 65535
# Semaphores: semmsl, semmns, semopm, semmni
kernel.shmmni = 4096
kernel.sem = 256 32000 100 142
fs.file-max = 6815744
fs.aio-max-nr = 1048576
kernel.hostname   = oradev.raastech.com
kernel.domainname = raastech.com
net.ipv4.ip_local_port_range = 9000 65500
net.core.rmem_default=262144
net.core.wmem_default=262144
net.core.rmem_max=4194304
net.core.wmem_max=1048576
kernel.msgmni = 2878

37. Load the sysctl settings

sysctl -p

38. Edit the configuration for the pam_limits module

vi /etc/security/limits.conf

39. Add the following to the bottom and save the file

oracle  soft    nofile  4096
oracle  hard    nofile  65536
oracle  soft    nproc   2047
oracle  hard    nproc   16384


Add an Extra 10GB Mount Point (optional)

40. Create a mount point

mkdir -p /u01
chown oracle:oinstall /u01

41. Add the extra EC2 volume and mount it

echo "View available disks"
lsblk

echo "Confirm that /dev/xvdb is a 'data' volume"
file -s /dev/xvdb

echo "Create file system, all data will be lost in it"
mkfs -t ext4 /dev/xvdb

echo "Backup fstab and edit it"
cp /etc/fstab /etc/fstab.orig.20150528
echo "/dev/xvdb       /u01   ext4    defaults,nofail        0       2" >> /etc/fstab

echo "Mount /u01"
mount /u01
chown oracle:oinstall /u01

echo "Confirm that /u01 is mounted and available"
df -m


Install an NFS Server (optional)

42. Install the necessary NFS packages

yum install nfs-utils

43. Create a share folder

mkdir -p /u01/share
chown oracle:oinstall /u01/share

44. Edit the /etc/exports file

vi /etc/exports

45. Add the following and save the file (the hostnames denote which clients should have access)

/u01/share oraclient1.raastech.com(no_root_squash,rw,sync)
/u01/share oraclient2.raastech.com(no_root_squash,rw,sync)

46. Start the NFS server

service rpcbind start
service nfs start

47. Open port 2049 on the local firewall to allow remote NFS access to other EC2 instances

service firewalld start
firewall-cmd --permanent --zone=public --add-port=2049/tcp
firewall-cmd --reload


Configure the NFS Clients (optional)

48. On the client servers, run the following commands

yum install nfs-utils
telnet soadb.raastech.com 2049
mkdir -p /u01/share
chown oracle:oinstall /u01/share
sudo echo "oradev.raastech.com:/u01/share   /u01/share      nfs     defaults        0 0" >> /etc/fstab
mount /u01/share
df -m


Install and Configure the VNC Server (optional)

49. Install the required VNC packages

yum groupinstall 'Server with GUI'yum install tigervnc-server
yum install xclock

50. Open up the VNC port 5901 for remote access

service firewalld start
firewall-cmd --permanent --zone=public --add-port=5901/tcp
firewall-cmd --reload

51. Start the VNC server

vncserver :1 -geometry 1280x720 -depth 16

52. Edit the profile script

vi /root/.bash_profile

53. Add the following and save the file

xhost +

54. Edit the profile script of additional unix users that will require GUI access

vi /home/oracle/.bash_profile

55. Add the following

export DISPLAY=:1

56. Connect via a VNC client such TightVNC on port 1






















Applicable Versions:
  • Amazon Web Services (2015)
  • Red Hat Enterprise Linux 7.1 64-bit