Performance tuning Oracle SOA Suite 11g

When installed out-of-the-box, Oracle SOA Suite 11g performs adequately for a development environment. In this post, I describe the key areas you need to focus on to improve your performance. I have not done much tuning on the BPEL Process Manager side, so once I do, I will update this blog post.

I have conducted light and heavy load tests on both synchronous and asynchronous Mediator services (not BPEL yet), and my findings and recommendations are documented below.

In summary, here are my findings:

• Upgrading to PS3 addresses memory instability issues
• Size your JVM appropriately
• Moving from Sun JDK to JRockit results in a 32% performance improvement
• Increasing the Mediator worker threads results in a 30% performance improvement for async services
• Changing the audit level from Development to Production results in a 46% performance improvement
• Changing the audit level from Production to Off results in a further 61% performance improvement
• Tuning the audit configuration causes Production and Off audit levels to perform equally the same
• Implementing parallel processing of routing rules may improve performance of Mediator anywhere from 4% to 509%
• I recommend using parallel garbage collection, but don't have statistics

1. Apply the PS3 patchset

I cannot stress this enough. Upgrade to SOA Suite PS3 (11.1.1.4). This resolves an enormous amount of memory issues and will save you a lot of pain.

2. Determine the size needed for your Java Heap Space

If you are running AIA Foundation Pack, then realistically you will need to set your Java Heap Space from 6GB to 10GB, depending on the amount of EBOs you are loading and the amount of composites you have. If you are running just SOA Suite, then you could manage with 2GB to 4GB.

1. Start up your SOA server, and wait until all composites are loaded2. Log in to the WebLogic Admin Console (at http://soaserver:7001/console)
3. Navigate to Home --> Servers --> soa_server1 --> Monitoring --> Performance
4. Click on "Garbage Collect"
5. Inspect the Heap Free Current
6. Though every environment is different, increase your heap space until you at least have 2 GB available

The Java Heap Space is configured in $MW_HOME/user_projects/domains/soa_domain/bin/setSOADomainEnv.sh as follows:

PORT_MEM_ARGS="-Xms6144m -Xmx6144m ...

3. Set your PermSize (for Sun JDK only)

I recommend a min and max of either 1GB or 1.5GB for your PermSize.

The PermSize is configured in $MW_HOME/user_projects/domains/soa_domain/bin/setSOADomainEnv.sh as follows:

PORT_MEM_ARGS="... -XX:PermSize=1024m -XX:MaxPermSize=1024m ...

4. Set your Nursery Size (for JRockit only)

Generally speaking, the Nursery Size should be around 30% of your Java Heap Space.

The Nursery Size is configured in $MW_HOME/user_projects/domains/soa_domain/bin/setSOADomainEnv.sh as follows:

PORT_MEM_ARGS="... -Xns2048m ...

5. Use parallel garbage collection (for Sun JDK only)

Garbage collection (and some other recommended settings) is configured in $MW_HOME/user_projects/domains/soa_domain/bin/setSOADomainEnv.sh as follows:

PORT_MEM_ARGS="... -XX:+UseConcMarkSweepGC -XX:+UseParNewGC -XX:+ExplicitGCInvokesConcurrent -XX:-TraceClassUnloading ...

6. Use JRockit instead of Sun JDK

My performance testing has shown an average of 32% performance improvement when using JRockit versus Sun JDK.

7. Modify and tune the Audit Level

Adjusting the Audit Level and tuning it is necessary.

1. Log in to the Fusion Middleware Console (at http://soaserver:7001/em)
2. Navigate to Farm_soa_domain --> SOA --> (right-click on) soa-infra --> SOA Administration --> Common Properties --> More SOA Infra Advanced Configuration Properties...
3. Click on "Audit Config"
4. Set the following values and click on "Apply" afterwards

AuditConfig/compositeInstanceStateEnabled = false    <-- you must set this for it to work!

AuditConfig/level = Production
AuditConfig/policies/Element_0/isActive = false
AuditConfig/policies/Element_0/name = Immediate
AuditConfig/policies/Element_1/isActive = true
AuditConfig/policies/Element_1/name = Deferred
AuditConfig/policies/Element_1/properties/Element_0/name = maxThreads
AuditConfig/policies/Element_1/properties/Element_0/value = 10
AuditConfig/policies/Element_1/properties/Element_1/name = flushDelay
AuditConfig/policies/Element_1/properties/Element_1/value = 5000
AuditConfig/policies/Element_1/properties/Element_2/name = batchSize
AuditConfig/policies/Element_1/properties/Element_3/value = 100

My performance tests have shown that changing from Development to Production results in an average of 46% performance improvement (without tuning the audit settings).

My performance tests have shown that changing from Production to Off results in an average of  61% performance improvement (without tuning the audit settings).

My performance tests have shown that if you apply the audit tuning described above, both Production and Off perform the same, effectively translating to a 61% performance improvement of the Production audit level.

8. Modify Mediator Service Engine properties (for Mediator only)

Manipulating the threads will have a direct impact on the ability of the server to process asynchronous Mediator services.

1. Log in to the Fusion Middleware Console (at http://soaserver:7001/em)
2. Navigate to Farm_soa_domain --> SOA --> (right-click on) soa-infra --> SOA Administration --> Mediator Properties
3. Edit the settings as follows:

Metrics Level:                                        Disabled
Parallel Worker Threads:                        12              <-- experiment increasing this value
Parallel Maximum Rows Retrieved:         600            <-- set to 50x the above setting
Parallel Locker Sleep Thread:                 1                <-- reduces waits for parallel routing rules

My performance tests have shown that changing from 4 worker threads and  200 max rows to 12 worker threads and 600 rows results in an average of 30% performance improvement for asynchronous Mediator services and 0% performance improvement for synchronous Mediator services.

9. Reduce soa-infra Log Levels

Unless required, reduce the SOA Suite Log Levels to error.

1. Log in to the Fusion Middleware Console (at http://soaserver:7001/em)
2. Navigate to Farm_soa_domain --> SOA --> (right-click on) soa-infra --> Logs --> Log Configuration
3. Set all Log Levels to ERROR:1 (SEVERE)
4. This must be repeated for all soa_server managed servers in your cluster

My performance tests have shown that reducing this from NOTIFICATION:1 (INFO) to ERROR:1 (SEVERE) results in an average of  7% performance improvement.

10. Perform Database Tuning

These are the database parameters I used in a large scale implementation. However, remember that they are based on the database AWR reports, so you may not want to just go ahead and apply them blindly. They are here for your reference and have resulted in some improvement in our environment.

AUDIT_TRAIL            = NONE
DB_FILES               = 1024
DISK_ASYNCH_IO         = TRUE
FILESYSTEMIO_OPTIONS   = SETALL
JOB_QUEUE_PROCESSES    = 10
LOG_BUFFER             = 209715200
NLS_SORT               = BINARY
SESSION_CACHED_CURSORS = 500
PROCESSES              = 1500
SESSION_MAX_OPEN_FILES = 50
UNDO_MANAGEMENT        = AUTO
PLSQL_CODE_TYPE        = NATIVE
MEMORY_TARGET          = 0
SGA_TARGET             = 6g
PGA_AGGREGATE_TARGET   = 2g
TRACE_ENABLED          = FALSE

11. Modify BPEL Process Manager Properties (for BPEL only)

Manipulating the threads will have a direct impact on the ability of the server to process asynchronous BPEL processes.

1. Log in to the Fusion Middleware Console (at http://soaserver:7001/em)
2. Navigate to Farm_soa_domain --> SOA --> (right-click on) soa-infra --> SOA Administration --> BPEL Properties
3. Edit the settings as follows:

Dispatcher System Threads:       10            <-- increase at least from the default value of 2 to 10
Dispatcher Invoke Threads:         20            <-- depends if your targets can handle the load
Dispatcher Engine Threads:        30            <-- should not exceed the sum of the two values above
Payload Validation:                    Disabled

Honestly, these numbers will vary based on the types of process designs you have.


Applicable Versions:

• Oracle SOA Suite 11g (11.1.1.4)