Friday, May 18, 2012

JOC-058, JOC-043, and JOC-823 when starting up SOA Suite 11g

Problem:

Getting the following error when starting up the soa_server1 managed server:

<May 18, 2012 1:53:31 PM EDT> <Warning> <oracle.sdp.messaging.driver.base> <SDP-26035> <JNDI lookup of server command endpoint OraSDPM/ejb/sdpmessagingserver/CommandProcessorRemote failed with the following error message: While trying to lookup 'OraSDPM.ejb.sdpmessagingserver/CommandProcessorRemote' didn't find subcontext 'ejb'. Resolved 'OraSDPM'. Will retry until successful.>
<May 18, 2012 1:57:25 PM EDT> <Error> <oracle.as.cache.groupv2.Distributor> <BEA-000000> <DI Unable to find or become a group coordinator within 300000 seconds>
<May 18, 2012 1:57:27 PM EDT> <Error> <oracle.as.cache> <BEA-000000> <oracle.ias.cache.NetworkException: J2EE JOC-058 distributed cache initialization failure
J2EE JOC-043 base exception:
J2EE JOC-823 unable to contact the packet-distributor at [localhost:20030] segID=0

    at oracle.ias.cache.groupv2.Client.init(Client.java:86)
    at oracle.ias.cache.groupv2.GroupMember.<init>(GroupMember.java:383)
    at oracle.ias.cache.groupv2.GroupMember.create(GroupMember.java:93)
    at oracle.ias.cache.groupv2.GrpCommunication.init(GrpCommunication.java:141)
    at oracle.ias.cache.groupv2.GrpCommunication.init(GrpCommunication.java:121)
    at oracle.ias.cache.Net.init(Net.java:104)
    at oracle.ias.cache.CacheInternal.initNet(CacheInternal.java:2924)
    at oracle.ias.cache.CacheInternal.init(CacheInternal.java:695)
    at oracle.ias.cache.CacheInternal.init(CacheInternal.java:343)
    at oracle.ias.cache.CacheInternal.createNamedCache(CacheInternal.java:2601)
    at oracle.ias.cache.Cache.createNamedCache(Cache.java:661)
    at oracle.mds.internal.cache.JOCCacheProvider.createNamedCacheInternal(JOCCacheProvider.java:292)
    at oracle.mds.internal.cache.JOCCacheProvider.createNamedCache(JOCCacheProvider.java:255)
    at oracle.mds.internal.cache.JOCCacheProvider.<init>(JOCCacheProvider.java:87)
    at oracle.mds.core.MDSInstance.initCache(MDSInstance.java:1620)
    at oracle.mds.core.MDSInstance.<init>(MDSInstance.java:1758)

Solution #1:

This was a 2-node clustered installation of Oracle SOA Suite 11g. There were multiple problems in this case.

1. The AdminServer was incorrectly started up on both nodes. Shut them both down.

2. NodeManager was not started on the second node. Start it up.

3. Shutdown all soa_server managed servers, then bring them all up again (AdminServer, soa_server1, soa_server2)

Solution #2:

This could be due to the Java Object Cache (JOC) of OWSM needing to be reconfigured.

1. Stop all managed servers, except the AdminServer.

2. Run the following commands on SOAHOST1 only:
cd $MW_HOME/oracle_common/common/bin
./wlst.sh
3. Connect as follows:
wls:/offline> connect()
Please enter your username : weblogic
Please enter your password : welcome1
Please enter your server URL [t3://localhost:7001] :
Connecting to t3://adminhost:7001 with userid weblogic ...
Successfully connected to Admin Server 'AdminServer' that belongs to domain 'soa_domain'.
4. Execute the configure-joc.py script:
execfile('/u01/app/oracle/middleware/oracle_common/bin/configure-joc.py')
Enter Hostnames (eg host1,host2) : soahost1,soahost2
Do you want to specify a cluster name (y/n) <y>y
Enter Cluster Name : soacluster
Enter Discover Port : 9991
Enter Distribute Mode (true|false) <true> : true
Do you want to exclude any server(s) from JOC configuration (y/n) <n>n
5. Start all managed servers.


Applicable Versions:
  • Oracle SOA Suite 11g (11.1.1.5)

References:
  • Oracle Note ID 1422223.1

Ahmed Aboulnaga

3 comments:

Anonymous said...

Thanks for the solutions. it works for me.

Anonymous said...

I have Question .. when you say executing WLST on soahost1 only .. what servers are running on that host Admin and Soa Server1 or Just SoaServer1? What if we Admin Server is running on 1 machine and 2 SOA servers are running on 2 other machines .. where we need to tun this PY script and what hosts we should be giving ( I assume Manged server hosts). Please confirm

Anonymous said...

Thanks for the detailed steps.

The 11.1.1.4 version had same issue in cluster environment we could restart the nodes after shutting down both nodes.

Thanks,
Raghu