The persistent store could not be deployed on WebLogic 11g

Problem:

We created a JMS Server, and upon startup of the WebLogic managed server, we received the following error in the logs:

<Jan 12, 2011 3:07:41 PM EST> <Error> <Store> <BEA-280061> <The persistent store "MyJMSStore" could not be deployed: weblogic.store.io.jdbc.JDBCStoreException: open failed>

This is how we created the JMS Server:

1. Create a Persistent JDBC Store with the following settings:

Name:         MyJMSStore
Target:         soa_server1
Datasource: MyJMSDataSource (which was a newly created datasource)

2. Create a JMS Server with the following settings:

Name:                MyJMSServer
Persistent Store: MyJMSStore
Target:               soa_server1

3. Shutdown and restart the AdminServer and soa_server1 



Investigation:

Upon further investigation in the soa_server1.out log, we noticed the following:

<Jan 12, 2011 3:07:20 PM EST> <Warning> <Store> <BEA-280076> <Database table "WLStore" for store "MyJMSStore" is currently owned by "[name={server=soa_server1!host=192.168.1.197!domain=soa_domain!store=MyJMSStore!table=WLStore}:random=8560773209970381667:timestamp=1294862840724]". Trying to wait for ownership.>

<Jan 12, 2011 3:07:41 PM EST> <Error> <Store> <BEA-280077> <JDBC store "MyJMSStore" in this server is not the owner of database table "WLStore". Unexpected current owner is "[name={server=soa_server1!host=192.168.1.197!domain=soa_domain!store=MyJMSStore!table=WLStore}:random=8560773209970381667:timestamp=1294862850740]", expected current owner is "[name={server=soa_server1!host=192.168.1.127!domain=soa_domain!store=MyJMSStore!table=WLStore}:random=-402518923774162410:timestamp=1294862840029]".>

<Jan 12, 2011 3:07:41 PM EST> <Error> <Store> <BEA-280072> <JDBC store "MyJMSStore" failed to open table "WLStore".
weblogic.store.io.jdbc.JDBCStoreException: [Store:280064]280077 (server="soa_server1" store="MyJMSStore" table="WLStore")>

<Jan 12, 2011 3:07:41 PM EST> <Error> <Store> <BEA-280061> <The persistent store "MyJMSStore" could not be deployed: weblogic.store.io.jdbc.JDBCStoreException: open failed>

The log states that the unexpected current owner of this Persistent Store is:

{server=soa_server1!host=192.168.1.197!domain=soa_domain!store=MyJMSStore!table=WLStore}

The log states that the expected owner should be:

{server=soa_server1!host=192.168.1.127!domain=soa_domain!store=MyJMSStore!table=WLStore}

These are different IP addresses!

The IP address of my server was 192.168.1.127, and for some reason, it was picking up the IP address of some other random server!


Solution:

This was because the same database persisted JMS Server (named "MyJMSServer") was created on 2 separate WebLogic servers and pointing to the same datasource.

Oracle documentation states that "Two JDBC stores must not share the same database table, because this will result in data corruption."

1. Delete the JMS Server on one of the servers.

2. Change the JMS Server name from "MyJMSServer" to "MyJMSServer2" (or any other name) to avoid conflict, since the two JMS Servers on both servers are using the same datasource.

3. Now when we start up the AdminServer, the JMS Server starts up correctly:

soa_server1.log:####<Jan 19, 2011 5:57:09 PM EST> <Info> <JMS> <oradev> <soa_server1> <[ACTIVE] ExecuteThread: '1' for queue: 'weblogic.kernel.Dtuning)'> <<WLS Kernel>> <> <> <1295477829640> <BEA-040109> <JMSServer "MyJMSServer2" is started.>

References:

• http://download.oracle.com/docs/cd/E13222_01/wls/docs103/config_wls/store.html#wp1143851
• Many thanks to Abhishek from Oracle Support