Wednesday, January 17, 2018

BEA-000337 and STUCK thread in OSB 11g

Problem

Seeing the AdminServer in warning state and a stuck thread. AdminServer.log shows this:
####<Mar 15, 2017 4:54:00 PM GMT> <Error> <WebLogicServer> <osbhost1.raastech.com> <AdminServer> <[ACTIVE] ExecuteThread: '17' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <dd3937adc7e76694:5ea517f4:15ac77c3297:-8000-0000000000004fb0> <1489596840017> <BEA-000337> <[STUCK] ExecuteThread: '23' for queue: 'weblogic.kernel.Default (self-tuning)' has been busy for "1,433" seconds working on the request "Workmanager: weblogic.kernel.Default, Version: 0, Scheduled=false, Started=true, Started time: 1434264 ms
", which is more than the configured time (StuckThreadMaxTime) of "1,200" seconds. Stack trace:
null>
Solution

1. Look through the AdminServer.log log file. Could be related to an OutOfMemory exception that occurred during deployment.
####<Mar 15, 2017 4:32:00 PM GMT> <Warning> <Deployer> <osbhost1.raastech.com> <AdminServer> <[ACTIVE] ExecuteThread: '18' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <dd3937adc7e76694:5ea517f4:15ac77c3297:-8000-0000000000004f91> <1489595520836> <BEA-149078> <Stack trace for message 149004
java.lang.Exception: java.lang.OutOfMemoryError: Java heap space
        at com.bea.wli.config.deployment.server.ServerDeploymentReceiver.notifyPrepareFailure(ServerDeploymentReceiver.java:355)
        at com.bea.wli.config.deployment.server.ServerDeploymentReceiver._prepare(ServerDeploymentReceiver.java:434)
        at com.bea.wli.config.deployment.server.ServerDeploymentReceiver.access$700(ServerDeploymentReceiver.java:70)
        at com.bea.wli.config.deployment.server.ServerDeploymentReceiver$1.run(ServerDeploymentReceiver.java:246)
        at weblogic.security.acl.internal.AuthenticatedSubject.doAs(AuthenticatedSubject.java:321)
        at weblogic.security.service.SecurityManager.runAs(SecurityManager.java:120)
        at com.bea.wli.config.deployment.server.ServerDeploymentReceiver.prepare(ServerDeploymentReceiver.java:213)
        at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.doPrepareCallback(DeploymentReceiverCallbackDeliverer.java:171)
        at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer.access$000(DeploymentReceiverCallbackDeliverer.java:13)
        at weblogic.deploy.service.internal.targetserver.DeploymentReceiverCallbackDeliverer$1.run(DeploymentReceiverCallbackDeliverer.java:46)
        at weblogic.work.SelfTuningWorkManagerImpl$WorkAdapterImpl.run(SelfTuningWorkManagerImpl.java:550)
        at weblogic.work.ExecuteThread.execute(ExecuteThread.java:263)
        at weblogic.work.ExecuteThread.run(ExecuteThread.java:221)
Caused By: java.lang.OutOfMemoryError: Java heap space
        at java.util.Arrays.copyOfRange(Arrays.java:2694)
        at java.lang.String.<init>(String.java:203)
        at java.lang.String.substring(String.java:1877)

Applicable Versions
  • Oracle Service Bus (OSB) 11g (11.1.1.9.0)

 

Friday, January 5, 2018

HTTP 500 when logging in to Siebel with SSO/X509

Problem

When accessing the Siebel application at: https://siebelhost1.raastech.com/epublicsector_enu/, you get an HTTP 500 on the browser.
 
All services are up.
 
Nothing is in any of these logs:
  • WebGate --> C:\NetPoint\Webgate\access\oblix\logs\oblog.log
  • Siebel --> C:\Siebel\15.0.0.0.0\eappweb\log\ss170601_2516.log
  • IIS --> C:\inetpub\logs\LogFiles\W3SVC1\u_ex170606.log
 
Solution

1. There was likely an IAM outage. You will need to restart the SIE01 and SIE02 webtiers.


Applicable Versions
  • Oracle Access Manager 11g

 

Monday, January 1, 2018

BEA-000403 MaxMessageSizeExceededException: Incoming message of size: '10000080' bytes exceeds

Problem

Error received during OSB deployment:
<May 2, 2017 7:13:01 PM GMT> <Error> <Socket> <BEA-000403> <IOException occurred on socket: Socket[addr=osbhost1.raastech.com/192.168.0.30,port=7001,localport=57684]
weblogic.socket.MaxMessageSizeExceededException: Incoming message of size: '10000080' bytes exceeds the configured maximum of: '10000000' bytes for protocol: 't3'.
weblogic.socket.MaxMessageSizeExceededException: Incoming message of size: '10000080' bytes exceeds the configured maximum of: '10000000' bytes for protocol: 't3'
        at weblogic.socket.BaseAbstractMuxableSocket.incrementBufferOffset(BaseAbstractMuxableSocket.java:230)
        at weblogic.rjvm.t3.MuxableSocketT3.incrementBufferOffset(MuxableSocketT3.java:369)
        at weblogic.socket.SocketMuxer.readFromSocket(SocketMuxer.java:984)
        at weblogic.socket.SocketMuxer.readReadySocketOnce(SocketMuxer.java:920)
        at weblogic.socket.SocketMuxer.readReadySocket(SocketMuxer.java:887)
        at weblogic.socket.JavaSocketMuxer.processSockets(JavaSocketMuxer.java:343)
        at weblogic.socket.SocketReaderRequest.run(SocketReaderRequest.java:29)
        at weblogic.work.ExecuteRequestAdapter.execute(ExecuteRequestAdapter.java:21)
        at weblogic.kernel.ExecuteThread.execute(ExecuteThread.java:145)
        at weblogic.kernel.ExecuteThread.run(ExecuteThread.java:117)
>
 
Solution

1. Navigate to Servers > AdminServer > Protocols and "Enable Tunneling"

2. Make this change to the envcode.properties file (or equivalent) when deploying (basically, use HTTP instead of T3):
OLD: osbAdminUrl=t3://192.168.0.30:7001

NEW: osbAdminUrl=http://192.168.0.30:7001

Applicable Versions
  • Oracle WebLogic Server 11g (11.1.1.9.0)
  • Oracle Service Bus (OSB) 11g (11.1.1.9.0)

 

Tuesday, December 26, 2017

Cloning OID 11g

Oracle does not support the changing of hostname or IP address for your Oracle Internet Directory 11g installation, rendering cloning impossible. These instructions, however, guide you through that process.


1. Set the environment:
export ORACLE_HOME=/u01/app/oracle/middleware/Oracle_OID
export ORACLE_INSTANCE=/u01/app/oracle/middleware/asinst_1
export TNS_ADMIN=$ORACLE_INSTANCE/config 
export PATH=$ORACLE_HOME/bin:$ORACLE_HOME/ldap/bin:$ORACLE_INSTANCE/bin:$PATH
2. Clear the temp folders on node 1 (for AdminServer, wls_ods1):
cd ${ASERVER_HOME}/servers/AdminServer
rm -rf cache/* logs/* tmp/*
cd ${MSERVER_HOME}/servers/wls_ods1
rm -rf cache/* logs/* tmp/* data/*
cd ${MSERVER_HOME}/servers/wls_ods2 
rm -rf cache/* logs/* tmp/* data/*
3. Replace values in red:
cd /u01/app/oracle/middleware
find . -type f \( ! -name "oidpwdr*" ! -name "*.DAT" ! -name "*.cer" ! -name "*dump1" ! -name "*.swp" ! -name "*.jks" \) -exec sed -i "s/OLDHOST/NEWHOST/g" {} \;
find . -type f \( ! -name "oidpwdr*" ! -name "*.DAT" ! -name "*.cer" ! -name "*dump1" ! -name "*.swp" ! -name "*.jks" \) -exec sed -i "s/OLDHOST/NEWHOST/g" {} \; 
find . -type f \( ! -name "oidpwdr*" ! -name "*.DAT" ! -name "*.cer" ! -name "*dump1" ! -name "*.swp" ! -name "*.jks" \) -exec sed -i "s/.olddomain.com/.newdomain.com/g" {} \;
4. Change the strings to the new database string:
FILE: /u01/app/oracle/middleware/asinst_1/config/tnsnames.ora
OLD: OIDDB=(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=prod-cluster)(PORT=1521))(CONNECT_DATA=(SERVER=DEDICATED)(SERVICE_NAME=oiddb.raastech.com)))
FILE: /u01/app/oracle/middleware/user_projects/domains/oiddomain/config/jdbc/schedulerDS-*-jdbc.xml
OLD: <url>jdbc:oracle:thin:@//prod-cluster:1521/oiddb.raastech.com</url>
OLD: <value>1521</value>
OLD: <value>prod-cluster</value>
5. Run the change-ip-host script:
nohup ${ASERVER_HOME}/bin/startWebLogic.sh >> ${ASERVER_HOME}/servers/${ADMINSERVER}/logs/${ADMINSERVER}.out & 
cd $ORACLE_HOME 
$ORACLE_HOME/chgip/scripts/chgiphost.sh -oldhost OLDHOST -newhost NEWHOST -instanceHome /u01/app/oracle/middleware/asinst_1 
$ORACLE_INSTANCE/bin/opmnctl unregisterinstance -instanceName asinst_1 -adminHost NEWHOST.raastech.com -adminPort 7001 
$ORACLE_INSTANCE/bin/opmnctl redeploy -adminHost NEWHOST.raastech.com -adminPort 7001 
ll /u01/app/oracle/middleware/user_projects/domains/oiddomain/servers/AdminServer/upload/NonJ2EEManagement/11.1.1/app/NonJ2EEManagement.ear 
$ORACLE_INSTANCE/bin/opmnctl redeploy -adminHost NEWHOST.raastech.com -adminPort 7001

$ORACLE_INSTANCE/bin/opmnctl registerinstance -adminHost NEWHOST.raastech.com -adminPort 7001
If the 'redeploy' or 'registerinstance' actions fail, do not worry. Just confirm that "NonJ2EEManagement" is a deployed application and exists in /u01/app/oracle/middleware/user_projects/domains/oiddomain/servers/AdminServer/upload/NonJ2EEManagement/11.1.1/app/NonJ2EEManagement.ear.
 6. Remove the states folder to avoid the "Failed to open locale state file" error:
rm -rf /u01/app/oracle/middleware/asinst_1/config/OPMN/states
7. Reset the ODS password in the wallet (make sure this password matches that of the ODS user in the database):
cd $ORACLE_INSTANCE/OID/admin

mv oidpwdlldap1 oidpwdlldap1.old

$ORACLE_HOME/ldap/bin/oidpasswd connect=OIDDB create_wallet=true
8. Startup the rest of the environment:
nohup ${MSERVER_HOME}/bin/startManagedWebLogic.sh wls_ods1 >> ${MSERVER_HOME}/servers/wls_ods1/logs/wls_ods1.out & 
${MW_HOME}/asinst_1/bin/opmnctl startall
9. Verify the installation:
$ORACLE_HOME/bin/ldapbind -h oidhost1.raastech.com -p 3060 -D cn=orcladmin -w welcome1
$ORACLE_HOME/bin/ldapbind -h oidhost1.raastech.com -p 3131 -D cn=orcladmin -w welcome1 -U 1 
$ORACLE_HOME/bin/ldapsearch -h oidhost1.raastech.com -p 3131 -U 1 -D cn=orcladmin -s sub "(objectclass=*)" -w welcome1

Applicable Versions
  • Oracle Internet Directory (OID) 11g (11.1.1.9.0)

 

Wednesday, December 13, 2017

Unable to Connect to Database: Incorrect location for tnsnames.ora in OID 11g

Problem

If you are trying to execute the oidpasswd command, for example:
$ORACLE_HOME/ldap/bin/oidpasswd connect=OIDDB create_wallet=true 
$ORACLE_HOME/ldap/bin/oidpasswd connect=OIDDB change_oiddb_pwd=true
You receive the following error on the prompt:
Unable to Connect to Database: Incorrect location for tnsnames.ora (derived from ORACLE_INSTANCE) or Incorrect TNS Connect string or Invalid Password

Solution

1. This is because you must set the TNS_ADMIN environment variable:
export ORACLE_HOME=/u01/app/oracle/middleware/Oracle_OID 
export ORACLE_INSTANCE=/u01/app/oracle/middleware/asinst_1 
export TNS_ADMIN=$ORACLE_INSTANCE/config 
export PATH=$ORACLE_HOME/bin:$ORACLE_HOME/ldap/bin:$ORACLE_INSTANCE/bin:$PATH
2. If this does not work, try the following:
cp $TNS_ADMIN/tnsnames.ora $ORACLE_HOME/network/admin
3. If this does not work, the problem is with the "ods" account on the database.
a. Try to connect to the "ods" user using Oracle SQL Developer.

b. Try to connect to the "ods" user using SQL*Plus on the OAM server:  
sqlplus ods/welcome1@oiddb
c. Then try resetting the password again or create a new wallet

 
Applicable Versions
  • Oracle Internet Directory (OID) 11g

 

Saturday, December 2, 2017

Test Coherence communication between nodes using 'multicast-test.sh'

This is a successful execution of the multicast-test.sh script. Any other response may indicate communication issues between the nodes.
oracle@soahost1:/u01/app/oracle/middleware/coherence_3.7/bin> ./multicast-test.sh -local soahost1 
java version "1.7.0_121"
Java(TM) SE Runtime Environment (build 1.7.0_121-b32)
Java HotSpot(TM) 64-Bit Server VM (build 24.121-b32, mixed mode)
 
2017-05-03 19:21:53.805/0.380 Oracle Coherence 3.7.1.1 <Info> (thread=main, member=n/a): Loaded operational configuration from "jar:file:/u01/app/oracle/middleware/coherence_3.7/lib/coherence.jar!/tangosol-coherence.xml"
2017-05-03 19:21:53.878/0.453 Oracle Coherence 3.7.1.1 <Info> (thread=main, member=n/a): Loaded operational overrides from "jar:file:/u01/app/oracle/middleware/coherence_3.7/lib/coherence.jar!/tangosol-coherence-override-dev.xml"
2017-05-03 19:21:53.878/0.453 Oracle Coherence 3.7.1.1 <D5> (thread=main, member=n/a): Optional configuration override "/tangosol-coherence-override.xml" is not specified
2017-05-03 19:21:53.881/0.456 Oracle Coherence 3.7.1.1 <D5> (thread=main, member=n/a): Optional configuration override "/custom-mbeans.xml" is not specified
 
Oracle Coherence Version 3.7.1.1 Build 28901
 Grid Edition: Development mode
Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.
 
Starting test on ip=soahost1/192.168.0.20, group=/237.0.0.1:9000, ttl=4
Configuring multicast socket...
Starting listener...
Wed May 03 19:21:53 GMT 2017: Sent packet 1 containing 1468 bytes.
Wed May 03 19:21:54 GMT 2017: Received test packet 1 from self (sent 7ms ago).
Wed May 03 19:21:54 GMT 2017: Received test packet 3 from ip=soahost2/192.168.0.21, group=/237.0.0.1:9000, ttl=4 containing 1468 bytes.
Wed May 03 19:21:56 GMT 2017: Sent packet 2 containing 1468 bytes.
Wed May 03 19:21:56 GMT 2017: Received test packet 2 from self (sent 1ms ago).
Wed May 03 19:21:56 GMT 2017: Received test packet 4 from ip=soahost2/192.168.0.21, group=/237.0.0.1:9000, ttl=4 containing 1468 bytes.
Wed May 03 19:21:58 GMT 2017: Sent packet 3 containing 1468 bytes.
Wed May 03 19:21:58 GMT 2017: Received test packet 3 from self (sent 3ms ago).
Wed May 03 19:21:58 GMT 2017: Received test packet 5 from ip=soahost2/192.168.0.21, group=/237.0.0.1:9000, ttl=4 containing 1468 bytes.
Wed May 03 19:22:00 GMT 2017: Sent packet 4 containing 1468 bytes.
Wed May 03 19:22:00 GMT 2017: Received test packet 4 from self (sent 1ms ago).
Wed May 03 19:22:02 GMT 2017: Sent packet 5 containing 1468 bytes.
Wed May 03 19:22:02 GMT 2017: Received test packet 5 from self (sent 1ms ago).
 
Applicable Versions
  • Oracle Coherence 11g (11.1.1.9.0)

 

Friday, December 1, 2017

Test Coherence communication between nodes using 'datagram-test.sh'

This is a successful execution of the datagram-test.sh script. Any other response may indicate communication issues between the nodes.
oracle@soahost1:/u01/app/oracle/middleware/coherence_3.7/bin> ./datagram-test.sh -polite soahost2 
java version "1.7.0_121"
Java(TM) SE Runtime Environment (build 1.7.0_121-b32)
Java HotSpot(TM) 64-Bit Server VM (build 24.121-b32, mixed mode)
2017-05-03 19:19:37.342/0.319 Oracle Coherence 3.7.1.1 <Info> (thread=main, member=n/a): Loaded operational configuration from "jar:file:/u01/app/oracle/middleware/coherence_3.7/lib/coherence.jar!/tangosol-coherence.xml"
2017-05-03 19:19:37.412/0.389 Oracle Coherence 3.7.1.1 <Info> (thread=main, member=n/a): Loaded operational overrides from "jar:file:/u01/app/oracle/middleware/coherence_3.7/lib/coherence.jar!/tangosol-coherence-override-dev.xml"
2017-05-03 19:19:37.412/0.389 Oracle Coherence 3.7.1.1 <D5> (thread=main, member=n/a): Optional configuration override "/tangosol-coherence-override.xml" is not specified
2017-05-03 19:19:37.415/0.392 Oracle Coherence 3.7.1.1 <D5> (thread=main, member=n/a): Optional configuration override "/custom-mbeans.xml" is not specified
 
Oracle Coherence Version 3.7.1.1 Build 28901
 Grid Edition: Development mode
Copyright (c) 2000, 2011, Oracle and/or its affiliates. All rights reserved.
 
creating datagram socket using provider: SystemSocketProvider
starting listener: at /192.168.0.20:9999
packet size: 1468 bytes
buffer size: 1428 packets
  report on: 100000 packets, 139 MBs
    process: 20 bytes/packet
        log: null
     log on: 139 MBs
 
starting publisher: at /192.168.0.20:10000 sending to soahost2/192.168.0.21:9999
packet size: 1468 bytes
buffer size: 32 packets
  report on: 100000 packets, 139 MBs
    process: 20 bytes/packet
      peers: 1
       rate: no limit
 
no packet burst limit
waiting for listener to be contacted before publishing
 
Recieving data from 1 publisher(s).
ioioioioioioiioioIoiOiioioioioiioioIoiioioiOiioioioiIoioioiioioioiiOioIoiioioioiioioioiIoiOioiioioioiioioIoiioioiOiioioioiIoioioioiioioiOiioIoiioioioioiioioioIiOioioiioioioiioI
Lifetime:
Rx from publisher: /192.168.0.21:10000
             elapsed: 1235ms
         packet size: 1468
          throughput: 113 MB/sec
                      80972 packets/sec
            received: 100000 of 100000
             missing: 0
        success rate: 1.0
        out of order: 0
          avg offset: 0
                gaps: 0
        avg gap size: 0
        avg gap time: 0ms
        avg ack time: 1.338602ms; acks 19992


Now:
Rx from publisher: /192.168.0.21:10000
             elapsed: 1235ms
         packet size: 1468
          throughput: 113 MB/sec
                      80972 packets/sec
            received: 100000 of 100000
             missing: 0
        success rate: 1.0
        out of order: 0
          avg offset: 0
                gaps: 0
        avg gap size: 0
        avg gap time: 0ms
        avg ack time: 1.339186ms; acks 19992
 
oioiioiOioiioioioIioioioiioioiOiioIoioioiioioioiioioIO
Tx summary 1 peers:
   life: 14 MB/sec, 10239 packets/sec
    now: 14 MB/sec, 10239 packets/sec
iioioiioioioioiIoioioiiOioioiioioIoiioioioiioiOiioIoioiioioioioiioioIOiioioioiioioioiIoioioiiOioioiioioIoiioioioiioiOioioI
Lifetime:
Rx from publisher: /192.168.0.21:10000
             elapsed: 2471ms
         packet size: 1468
          throughput: 113 MB/sec
                      80939 packets/sec
            received: 200000 of 200000
             missing: 0
        success rate: 1.0
        out of order: 0
          avg offset: 0
                gaps: 0
        avg gap size: 0
        avg gap time: 0ms
        avg ack time: 1.287305ms; acks 37107


Now:
Rx from publisher: /192.168.0.21:10000
             elapsed: 1234ms
         packet size: 1468
          throughput: 113 MB/sec
                      81037 packets/sec
            received: 100000 of 100000
             missing: 0
        success rate: 1.0
        out of order: 0
          avg offset: 0
                gaps: 0
        avg gap size: 0
        avg gap time: 0ms
        avg ack time: 1.227671ms; acks 17115
 
ioioioioiioioioiOIoiioioioioiioioioIoOoooooooooOoooooooooOoooooooooO
Tx summary 1 peers:
   life: 25 MB/sec, 17691 packets/sec
    now: 91 MB/sec, 65020 packets/sec
oooooooooOoooooooooOoooooooooO^C
 
Applicable Versions
  • Oracle Coherence 11g (11.1.1.9.0)

 

Sunday, November 26, 2017

MulticastUdpSocket failed to set receive buffer size in SOA Suite 11.1.1.9 cluster

Problem

The following appears to be repeating every 75 seconds exactly in soa_server1-diagnostic.log:
[2017-03-15T13:02:33.077+00:00] [soa_server1] [WARNING] [] [Coherence] [tid: Logger@9270387 3.7.1.13] [userId: <anonymous>] [ecid: 0000LfI3B__3b605zzT4iW1OmJeA000003,1:17737] [APP: soa-infra] 2017-03-15 13:02:33.077/235.749 Oracle Coherence GE 3.7.1.13 <Warning> (thread=PacketListenerN, member=1): MulticastUdpSocket failed to set receive buffer size to 16 packets (1023KB); actual size is 20%, 3 packets (208KB). Consult your OS documentation regarding increasing the maximum socket buffer size. Proceeding with the actual value may cause sub-optimal performance. 
[2017-03-15T13:03:48.114+00:00] [soa_server1] [WARNING] [] [Coherence] [tid: Logger@9270387 3.7.1.13] [userId: <anonymous>] [ecid: 0000LfI3B__3b605zzT4iW1OmJeA000003,1:17737] [APP: soa-infra] 2017-03-15 13:03:48.114/310.786 Oracle Coherence GE 3.7.1.13 <Warning> (thread=PacketListenerN, member=1): MulticastUdpSocket failed to set receive buffer size to 16 packets (1023KB); actual size is 20%, 3 packets (208KB). Consult your OS documentation regarding increasing the maximum socket buffer size. Proceeding with the actual value may cause sub-optimal performance. 
[2017-03-15T13:05:03.182+00:00] [soa_server1] [WARNING] [] [Coherence] [tid: Logger@9270387 3.7.1.13] [userId: <anonymous>] [ecid: 0000LfI3B__3b605zzT4iW1OmJeA000003,1:17737] [APP: soa-infra] 2017-03-15 13:05:03.182/385.854 Oracle Coherence GE 3.7.1.13 <Warning> (thread=PacketListenerN, member=1): MulticastUdpSocket failed to set receive buffer size to 16 packets (1023KB); actual size is 20%, 3 packets (208KB). Consult your OS documentation regarding increasing the maximum socket buffer size. Proceeding with the actual value may cause sub-optimal performance.  
[2017-03-15T13:06:18.221+00:00] [soa_server1] [WARNING] [] [Coherence] [tid: Logger@9270387 3.7.1.13] [userId: <anonymous>] [ecid: 0000LfI3B__3b605zzT4iW1OmJeA000003,1:17737] [APP: soa-infra] 2017-03-15 13:06:18.221/460.893 Oracle Coherence GE 3.7.1.13 <Warning> (thread=PacketListenerN, member=1): MulticastUdpSocket failed to set receive buffer size to 16 packets (1023KB); actual size is 20%, 3 packets (208KB). Consult your OS documentation regarding increasing the maximum socket buffer size. Proceeding with the actual value may cause sub-optimal performance. 
[2017-03-15T13:07:33.245+00:00] [soa_server1] [WARNING] [] [Coherence] [tid: Logger@9270387 3.7.1.13] [userId: <anonymous>] [ecid: 0000LfI3B__3b605zzT4iW1OmJeA000003,1:17737] [APP: soa-infra] 2017-03-15 13:07:33.245/535.917 Oracle Coherence GE 3.7.1.13 <Warning> (thread=PacketListenerN, member=1): MulticastUdpSocket failed to set receive buffer size to 16 packets (1023KB); actual size is 20%, 3 packets (208KB). Consult your OS documentation regarding increasing the maximum socket buffer size. Proceeding with the actual value may cause sub-optimal performance.
Solution

1. As 'root', edit /etc/sysctl.conf and add the following:
net.core.rmem_max=2096304
net.core.wmem_max=2096304
net.core.rmem_max=4194304                                                      
net.core.wmem_max=4194304
2. Bounce the SOA managed servers.

References
 
Applicable Versions
  • Oracle SOA Suite 11g (11.1.1.9.0)
  • Oracle Coherence 11g (11.1.1.9.0)

 

Friday, November 10, 2017

ORA-01017: invalid username/password when starting up OID 11g

Problem

When trying to startup the OID OPMN processes:
${ORACLE_HOME}/asinst_1/bin/opmnctl startall
In this file /u01/app/oracle/middleware/asinst_1/diagnostics/logs/OID/oid1/oidmon-0000.log, you see the following error:
[2017-04-26T21:39:25.10925+00:00] [OID] [NOTIFICATION:16] [] [OIDMON] [host: oamhost1.raastech.com] [pid: 17029] [tid: 0] Guardian: Connecting to database, connect string is oiddb 
[2017-04-26T21:39:26.57778+00:00] [OID] [NOTIFICATION:16] [] [OIDMON] [host: oamhost1.raastech.com] [pid: 17029] [tid: 0] Guardian: [gsdsiConnect] ORA-1017, ORA-01017: invalid username/password; logon denied
Solution

1. The DBA should unlock the ODS account:
ALTER USER ods ACCOUNT UNLOCK;
2. Connect as ODS and see if this query works:
CONN ods/**********;

SELECT instance_name FROM v$instance WHERE instance_number IN (SELECT MIN(instance_number) FROM v$instance);
3. If the query returns an ORA-00942 error, then log in as SYSTEM and grant these permissions:
SELECT * FROM sys.dba_role_privs WHERE grantee = "ODS";

ALTER USER ods DEFAULT ROLE SELECT_CATALOG_ROLE, RESOURCE, CONNECT;

Applicable Versions
  • Oracle Internet Directory (OID) 11g

 

Wednesday, November 8, 2017

Coherence WKA configuration changes with 11.1.1.9

With 11.1.1.9, Oracle has changed the instructions on what they recommend for the Coherence configuration for the WKA1 and WKA2 settings.

This is wrong: 

soa_server1
-Dtangosol.coherence.wka1=soahost1 -Dtangosol.coherence.wka2=soahost2 -Dtangosol.coherence.localhost=soahost1 -Dweblogic.MaxMessageSize=50000000 -Dtangosol.coherence.localport=8088 -Dtangosol.coherence.wka1.port=8088 -Dtangosol.coherence.wka2.port=8088

soa_server2
-Dtangosol.coherence.wka1=soahost2 -Dtangosol.coherence.wka2=soahost1 -Dtangosol.coherence.localhost=soahost2 -Dweblogic.MaxMessageSize=50000000 -Dtangosol.coherence.localport=8088 -Dtangosol.coherence.wka1.port=8088 -Dtangosol.coherence.wka2.port=8088

This is right:

soa_server1
-Dtangosol.coherence.wka1=soahost1 -Dtangosol.coherence.wka2=soahost2 -Dtangosol.coherence.localhost=soahost1 -Dweblogic.MaxMessageSize=50000000 -Dtangosol.coherence.localport=8088 -Dtangosol.coherence.wka1.port=8088 -Dtangosol.coherence.wka2.port=8088

soa_server2
-Dtangosol.coherence.wka1=soahost1 -Dtangosol.coherence.wka2=soahost2 -Dtangosol.coherence.localhost=soahost2 -Dweblogic.MaxMessageSize=50000000 -Dtangosol.coherence.localport=8088 -Dtangosol.coherence.wka1.port=8088 -Dtangosol.coherence.wka2.port=8088

 
References
 
Applicable Versions
  • Oracle SOA Suite 11g (11.1.1.9.0)

 

Tuesday, October 10, 2017

JPS-01013: The credential store DN cn=CredentialStore,cn=IAM,cn=JPSContext,cn=jpsroot is missing in the store

Problem

If you are trying to start up the OAM AdminServer, you see the following error in the logs, then the server dies:
<Jun 20, 2017 6:46:03 PM GMT> <Error> <Security> <BEA-090892> <The loading of OPSS java security policy provider failed due to exception, see the exception stack trace or the server log file for root cause. If still see no obvious cause, enable the debug flag -Djava.security.debug=jpspolicy to get more information. Error message: JPS-01055: Could not create credential store instance. Reason oracle.security.jps.JpsException: JPS-01013: The credential store DN cn=CredentialStore,cn=IAM,cn=JPSContext,cn=jpsroot is missing in the store; the target DN must be pre-configured.>
<Jun 20, 2017 6:46:03 PM GMT> <Critical> <WebLogicServer> <BEA-000386> <Server subsystem failed. Reason: weblogic.security.SecurityInitializationException: The loading of OPSS java security policy provider failed due to exception, see the exception stack trace or the server log file for root cause. If still see no obvious cause, enable the debug flag -Djava.security.debug=jpspolicy to get more information. Error message: JPS-01055: Could not create credential store instance. Reason oracle.security.jps.JpsException: JPS-01013: The credential store DN cn=CredentialStore,cn=IAM,cn=JPSContext,cn=jpsroot is missing in the store; the target DN must be pre-configured.
weblogic.security.SecurityInitializationException: The loading of OPSS java security policy provider failed due to exception, see the exception stack trace or the server log file for root cause. If still see no obvious cause, enable the debug flag -Djava.security.debug=jpspolicy to get more information. Error message: JPS-01055: Could not create credential store instance. Reason oracle.security.jps.JpsException: JPS-01013: The credential store DN cn=CredentialStore,cn=IAM,cn=JPSContext,cn=jpsroot is missing in the store; the target DN must be pre-configured.
        at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.loadOPSSPolicy(CommonSecurityServiceManagerDelegateImpl.java:1402)
        at weblogic.security.service.CommonSecurityServiceManagerDelegateImpl.initialize(CommonSecurityServiceManagerDelegateImpl.java:1022)
        at weblogic.security.service.SecurityServiceManager.initialize(SecurityServiceManager.java:888)
        at weblogic.security.SecurityService.start(SecurityService.java:141)
        at weblogic.t3.srvr.SubsystemRequest.run(SubsystemRequest.java:64)
        Truncated. see log file for complete stacktrace
Caused By: oracle.security.jps.JpsRuntimeException: JPS-01055: Could not create credential store instance. Reason oracle.security.jps.JpsException: JPS-01013: The credential store DN cn=CredentialStore,cn=IAM,cn=JPSContext,cn=jpsroot is missing in the store; the target DN must be pre-configured.
        at oracle.security.jps.internal.policystore.PolicyDelegationController.<init>(PolicyDelegationController.java:190)
        at oracle.security.jps.internal.policystore.TenantJavaPolicyProvider.<init>(TenantJavaPolicyProvider.java:161)
        at oracle.security.jps.internal.policystore.JavaPolicyProvider.<init>(JavaPolicyProvider.java:306)
        at oracle.security.jps.internal.policystore.JavaPolicyProvider.<init>(JavaPolicyProvider.java:279)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        Truncated. see log file for complete stacktrace
Caused By: oracle.security.jps.JpsException: JPS-01055: Could not create credential store instance. Reason oracle.security.jps.JpsException: JPS-01013: The credential store DN cn=CredentialStore,cn=IAM,cn=JPSContext,cn=jpsroot is missing in the store; the target DN must be pre-configured.
        at oracle.security.jps.internal.policystore.PolicyUtil.getDefaultPDPService(PolicyUtil.java:3180)
        at oracle.security.jps.internal.policystore.PolicyUtil.getPDPService(PolicyUtil.java:3480)
        at oracle.security.jps.internal.policystore.PolicyUtil.getPDPService(PolicyUtil.java:3466)
        at oracle.security.jps.internal.policystore.PolicyDelegationController.<init>(PolicyDelegationController.java:188)
        at oracle.security.jps.internal.policystore.TenantJavaPolicyProvider.<init>(TenantJavaPolicyProvider.java:161)
        Truncated. see log file for complete stacktrace
Caused By: oracle.security.jps.service.credstore.CredStoreException: JPS-01055: Could not create credential store instance. Reason oracle.security.jps.JpsException: JPS-01013: The credential store DN cn=CredentialStore,cn=IAM,cn=JPSContext,cn=jpsroot is missing in the store; the target DN must be pre-configured.
        at oracle.security.jps.internal.credstore.ldap.LdapCredentialStore.init(LdapCredentialStore.java:177)
        at oracle.security.jps.internal.credstore.ldap.LdapCredentialStore.<init>(LdapCredentialStore.java:166)
        at oracle.security.jps.internal.credstore.ldap.LdapCredentialStore.<init>(LdapCredentialStore.java:140)
        at oracle.security.jps.internal.credstore.ldap.LdapCredentialStoreProvider.getInstance(LdapCredentialStoreProvider.java:130)
        at oracle.security.jps.internal.credstore.rdbms.DbmsCredentialStoreProvider.getInstance(DbmsCredentialStoreProvider.java:78)
        Truncated. see log file for complete stacktrace
>
 
<Jun 20, 2017 6:46:03 PM GMT> <Error> <WebLogicServer> <BEA-000383> <A critical service failed. The server will shut itself down>
Solution

1. The DEV_OPSS schema either has no data, or it is a permission thing.

For example, running SELECT COUNT(1) FROM jps_attrs will return 0 records when there should be millions of records in that table.


References
  • Oracle Doc ID 2066916.1 


Applicable Versions
  • Oracle Access Manager 11g

 

Importing SSL user certificate to an Oracle Wallet

Oracle Wallet does not allow you to import a user certificate if the CSR was not generated in that wallet.

These instructions explain how to take your existing user cert and private key and generate a new Oracle Wallet with them. This is based off the instructions from the Oracle A-Team with some slight variation, so read that note first.

Run these instructions to create the ewallet.p12 file:
export JAVA_HOME=/u01/oracle/java 
export PATH=$JAVA_HOME/bin:$PATH

cat Trust.cer > combined.crt
cat Intermediate.cer >> combined.crt 
cat mycert.cer >> combined.crt

openssl pkcs12 -export -in combined.crt -inkey myprivatekey.key -out ewallet.p12

Afterwards, after running Oracle Wallet Manager, open the directory which contains the ewallet.p12 file (which essentially now becomes your Oracle Wallet directory), click on "Wallet > Auto Login > Exit".


References
 

Tuesday, October 3, 2017

Failed to open locale state file .locale in OID 11g

Problem

In the log file /u01/app/oracle/middleware/asinst_1/diagnostics/logs/OPMN/opmn/opmn.out, you see the following error:
[2017-04-20T09:35:22-04:00] [opmn] [ERROR:1] [752] [pm-internal] Failed to open locale state file /u01/app/oracle/middleware/asinst_1/config/OPMN/opmn/states/.locale (read: No such file or directory)
There are no permission or access issues on the .locale file.


Solution

1. Remove the "states" directory. It will be recreated upon OPMN startup.

 
Applicable Versions
  • Oracle Internet Directory (OID) 11g

 

Friday, September 29, 2017

Apache hardening

Don't forget to add these to your Apache (or OHS) configuration in httpd.conf:
Header set X-XSS-Protection "l;mode=block"Header set Content-Security-Policy "default-src *;"Header set X-Content-Type-Options "nosniff"Header set Cache-Control "no-store"Header set X-Frame-Options SAMEORIGIN

Happy hardening!


Applicable Versions
  • Apache
  • Oracle HTTP Server (OHS) 11g/12c

 

Thursday, September 28, 2017

Include multiple domains in ALLOW-FROM for X-Frame-Options (Apache)

Every single forum, blog post, and documentation online will tell you the same thing... that it's not possible to whitelist multiple domains with X-Frame-Options and to use Content-Security-Policy instead or some complicated and messy JavaScript as alternatives.

I'm here to tell you that they're all wrong.

Here is a snippet of my httpd.conf file:
# For obvious security reasonsHeader set X-Frame-Options SAMEORIGIN 
<Location /myapp>  # For the context /myapp, whitelist multiple domains  Header append X-Frame-Options "ALLOW-FROM https://raastech.com"  Header append X-Frame-Options "ALLOW-FROM https://raastechmobile.com"</Location>

If you open up Developer Tools (F12) in IE 11, you can confirm that the header will show the following:
X-Frame-Options: SAMEORIGIN, ALLOW-FROM https://raastech.com, ALLOW-FROM https://raastechmobile.com


Applicable Versions
  • Apache

 

Thursday, September 14, 2017

IAU-5047: Problem in transferring audit record

Problem

You may receive these errors in the logs when starting up the OAM AdminServer:
####<Jun 28, 2017 7:35:08 PM GMT> <Error> <oracle.jps.common> <oamhost1.raastech.com> <AdminServer> <AuditLoaderRunner> <<WLS Kernel>> <> <0000LngIa0d3V805zzc9yW1PKkHg00000T> <1498678508558> <BEA-000000> <IAU:IAU-5047: Problem in transferring audit record from file:/u01/app/oracle/middleware/user_projects/domains/oamdomain/servers/AdminServer/logs/auditlogs/OAM/audit.log position:12,485 will retry in next cycle. caught exception oracle.security.audit.AuditException: javax.naming.NameNotFoundException: Unable to resolve 'jdbc.AuditDB'. Resolved 'jdbc'; remaining name 'AuditDB'
        at oracle.security.audit.ajl.loader.OracleRepository.createConnection(OracleRepository.java:131)
        at oracle.security.audit.ajl.loader.OracleRepository.<init>(OracleRepository.java:75)
        at oracle.security.audit.ajl.loader.JDBCLogWriter.reinit(JDBCLogWriter.java:68)
        at oracle.security.audit.ajl.loader.JDBCLogWriter.write(JDBCLogWriter.java:115)
        at oracle.security.audit.ajl.loader.AuditLoader.readMessages(AuditLoader.java:516)
        at oracle.security.audit.service.AuditLoaderManager.readMessages(AuditLoaderManager.java:324)
        at oracle.security.audit.service.AuditLoaderManager$Runner.run(AuditLoaderManager.java:410)
Caused by: javax.naming.NameNotFoundException: Unable to resolve 'jdbc.AuditDB'. Resolved 'jdbc'; remaining name 'AuditDB'
        at weblogic.jndi.internal.BasicNamingNode.newNameNotFoundException(BasicNamingNode.java:1148)
        at weblogic.jndi.internal.BasicNamingNode.lookupHere(BasicNamingNode.java:258)
        at weblogic.jndi.internal.ServerNamingNode.lookupHere(ServerNamingNode.java:182)
        at weblogic.jndi.internal.BasicNamingNode.lookup(BasicNamingNode.java:206)
        at weblogic.jndi.internal.BasicNamingNode.lookup(BasicNamingNode.java:220)
        at weblogic.jndi.internal.WLEventContextImpl.lookup(WLEventContextImpl.java:254)
        at weblogic.jndi.internal.WLContextImpl.lookup(WLContextImpl.java:412)
        at javax.naming.InitialContext.lookup(InitialContext.java:411)
        at oracle.security.audit.ajl.loader.OracleRepository.createConnection(OracleRepository.java:110)
        ... 6 more
>
####<Jun 28, 2017 7:35:08 PM GMT> <Error> <oracle.jps.common> <oamhost1.raastech.com> <AdminServer> <AuditLoaderRunner> <<WLS Kernel>> <> <0000LngIa0d3V805zzc9yW1PKkHg00000T> <1498678508558> <BEA-000000> <IAU:IAU-5048: Problem in transferring audit record to Audit Store null null. Please verify your Audit Store configuration>
Solution

1. Target the AuditDB datasource to the AdminServer.

Applicable Versions
  • Oracle Access Manager 11g

 

Tuesday, September 12, 2017

Getting "An unexpected error occurred" when deploying an OSB project

Problem:

When deploying an OSB project, you may get the following undescriptive error on the OSB Console:
An unexpected error occurred.
The following error would appear in the AdminServer.out file:
####<Mar 3, 2017 5:03:21 PM GMT> <Error> <netuix> <osbhost1> <AdminServer> <[ACTIVE] ExecuteThread: '39' for queue: 'weblogic.kernel.Default (self-tuning)'> <weblogic> <> <685ea1de428af539:-5fcff8e9:82f2515a61f:-8000-0000000000003938> <1488560601009> <BEA-423137> <There was an error loading the requested URI null.>

####<Mar 3, 2017 5:03:21 PM GMT> <Error> <netuix> <osbhost1> <AdminServer> <[ACTIVE] ExecuteThread: '39' for queue: 'weblogic.kernel.Default (self-tuning)'> <weblogic> <> <de428af5685ea139:-5fcff8e9:15a61f82f25:-8000-0000000000003938> <1488560601009> <BEA-423223> <There was an error while running a lifecycle stage :: Lifecycle: UIControl.render :: for the control :: null ::.
com.bea.netuix.nf.UIControlException: com.bea.portlet.adapter.scopedcontent.ActionLookupFailedException: java.lang.NoSuchMethodError: org.apache.commons.io.IOUtils.readFully(Ljava/io/InputStream;[B)V
        at com.bea.netuix.servlets.controls.content.NetuiContent.checkPreRenderExceptions(NetuiContent.java:404)
        at com.bea.netuix.servlets.controls.content.NetuiContent.beginRender(NetuiContent.java:343)
        at com.bea.netuix.nf.ControlLifecycle$7.visit(ControlLifecycle.java:485)
        at com.bea.netuix.nf.ControlTreeWalker.walkRecursiveRender(ControlTreeWalker.java:518)

Solution:

1. OSB JAR is missing the ExportInfo file.


Applicable Version:

  • Oracle Service Bus (OSB) 11g (11.1.1.9)


Monday, September 11, 2017

"The user account is locked or disabled" when logging in with Oracle Access Manager SSO

Problem

When trying to log in with SSO against Oracle Access Manager 11g, you see the error on the screen:

The user account is locked or disabled. Please contact the System Administrator.

Solution

Check all the following.

1. OID is down or the password for cn=orcladmin password in OAM needs updating.

2. Confirm that OID is running:
oracle@oamhost1:/home/oracle> ${MW_HOME}/asinst_1/bin/opmnctl status -l
3. Confirm that OAM can connect to OID.

a. Log in to the OAM Console.

b. Navigate to Configuration > User Identity Stores.

c. Edit OIDStore.

d. Click on "Test Connection".


Applicable Versions
  • Oracle Access Manager 11g

 

Wednesday, September 6, 2017

Getting "httpd.worker': double free or corruption" when starting up OHS

Problem

Starting up OHS fails.

This error appears in ${OHS_LOGS}/ohs1.log:
--------
17/06/17 02:37:46 Start process
--------
/u01/app/oracle/middleware/Oracle_WT1/ohs/bin/apachectl startssl: execing httpd
[Sat Jun 17 02:37:46 2017] [warn] Errors will be logged into /u01/app/oracle/middleware/Oracle_WT1/instances/obiee/diagnostics/logs/OHS/ohs1/ohs1.log
[Sat Jun 17 02:37:47 2017] [warn] Errors will be logged into /u01/app/oracle/middleware/Oracle_WT1/instances/obiee/diagnostics/logs/OHS/ohs1/ohs1.log
*** Error in `/u01/app/oracle/middleware/Oracle_WT1/ohs/bin/httpd.worker': double free or corruption (out): 0x0000000001821300 ***
======= Backtrace: =========
/lib64/libc.so.6(+0x7c503)[0x7f3a1f9ab503]
/u01/app/oracle/middleware/Oracle_WT1/lib/libnnz11.so(nzumfree+0x64)[0x7f3a1f03aefa]
/u01/app/oracle/middleware/Oracle_WT1/lib/libnnz11.so(nztiFIC_Free_Identity_Contents+0x45)[0x7f3a1f06f2d3]
/u01/app/oracle/middleware/Oracle_WT1/lib/libnnz11.so(nztiDI_Destroy_Identity+0x3a)[0x7f3a1f06f268]
/u01/app/oracle/middleware/Oracle_WT1/lib/libnnz11.so(nztiFIL_Free_Identity_List+0x5a)[0x7f3a1f06f21a]
/u01/app/oracle/middleware/Oracle_WT1/lib/libnnz11.so(nztnFPC_Free_Persona_Contents+0x45)[0x7f3a1f072ec1]
If you comment out the ssl.conf line in httpd.conf, it starts up fine.


Solution

1. This was because the certificate chain was not complete in trust.jks.

For example, the identity cert was signed by a CA4, so the trust was missing the intermediate and root certs for that CA.

To import those certificates into the trust keystore:
keytool -import -alias PRODCA4 -file PRODCA4.crt -keystore /u01/app/oracle/middleware/keystore/trust.jks

keytool -import -alias PRODTreasury -file PRODTreasury.crt -keystore /u01/app/oracle/middleware/keystore/trust.jks
 
Applicable Versions
  • Oracle HTTP Server (OHS) 11g (11.1.1.9.0)

 

Wednesday, August 23, 2017

Find out what process is listening on what port

Do you want to know what process is listening on a particular port in Linux?

Using the lsof command you can confirm that a java process with a pid of 14202 is listening on port 7001:
oracle@soahost1:/home/oracle> lsof -i :7001 | grep LISTEN 
java  14202 oracle 1013u  IPv6 3652361843  0t0  TCP soahost1:afs3-callback (LISTEN)

Similarly, using the netstat command, you can confirm that the java process with a pid of 14202 is listening on port 7001:
oracle@soahost1:/home/oracle> netstat -anp | grep LISTEN | grep 7001
tcp   0  0 ::ffff:10.31.160.230:7001   :::*   LISTEN   14202/java



Thursday, August 17, 2017

Linux (process, search, size) command usage and examples

Process


View All Processes Hierarchically

ps -efaux

Shows Running Processes in Tree Format

pstree -p

Find the 20 Largest Processes (5th column)

ps -aux | sort -nk5 | grep -v USER | tail -20


Search



Search Recursively for Contents in a File

find . -type f | xargs grep "mytext"

Search Recursively for Contents in .java Files

find . -type f -name "*.java" | xargs grep "mytext"

Find All World Writable Files

find . -perm -2 ! -type l -ls

Search/Replace



Recursively Replace a String in .xml Files *dangerous*

find . -type f -name "*.xml" -exec sed -i "s%orabpel%orabpel2%" {} \;

Recursively Replace a String, but Exclude .class .jar. .zip Files *dangerous*

find . -type f \( ! -name "*.class" ! -name "*.jar" ! -name "*.zip" \) -exec sed -i "s%orabpel%orabpel2%" {} \;


Size



Directory Sizes

du -sm *

Find the 20 Largest Directories

du -S | sort -n | tail -20

Find Files > 100 MB

find . -size +100000k -exec du -h {} \;




 

Monday, August 7, 2017

WebGate 10g for Windows 2012 R2: "Could not read file" oblog config.xml

Problem

In the log file C:\NetPoint\Webgate\access\oblix\logs\oblog.log, you see this error:
2017/05/16@21:33:59.042000 2136 3108 INIT ERROR 0x000003B6 base\oblistrwutil.cpp:192 "Could not read file" filename^E:\NetPoint\Webgate\access//oblix/config/oblog_config.xml
Solution

1. Run these commands:
cd C:\NetPoint\Webgate\access\oblix\config\ 
cp oblog_config_wg.xml oblog_config.xml

Applicable Versions
  • Oracle WebGate 10g (Windows 32-bit)

 

Tuesday, August 1, 2017

Failed executing BamAdcServer.ping due to ServiceUnavailableException

Problem

Log in to the BAM Console at:
http://soahost1:9002/OracleBAM
Tried to log in to the BAM Console, and browser shows:
The BAM Server is unavailable.
These log entries show up in the bam_server1-diagnostic.log file during login:
[2017-03-10T13:10:13.846+00:00] [bam_server1] [TRACE:32] [] [oracle.bam.common.security] [tid: [ACTIVE].ExecuteThread: '47' for queue: 'weblogic.kernel.Default (self-tuning)'] [userId: weblogic] [ecid: 2913acc884e13d35:1da4e73c:15ab855a0ff:-8000-000000000000002b,0] [APP: oracle-bam#11.1.1] [SRC_CLASS: oracle.bam.common.security.authentication.WebAuthentication] [SRC_METHOD: authenticate] [60] Request: Scheme[http] AuthType[CLIENT_CERT,FORM] RemoteUser[weblogic] UserPrincipal[weblogic]
[2017-03-10T13:10:13.846+00:00] [bam_server1] [TRACE:32] [] [oracle.bam.common.security] [tid: [ACTIVE].ExecuteThread: '47' for queue: 'weblogic.kernel.Default (self-tuning)'] [userId: weblogic] [ecid: 2913acc884e13d35:1da4e73c:15ab855a0ff:-8000-000000000000002b,0] [APP: oracle-bam#11.1.1] [SRC_CLASS: oracle.bam.common.security.authentication.WebAuthentication] [SRC_METHOD: authenticate] [60] UserPrincipal: weblogic  
[2017-03-10T13:10:28.957+00:00] [bam_server1] [WARNING] [] [oracle.bam.common.remoting.BamEjbClient] [tid: [ACTIVE].ExecuteThread: '47' for queue: 'weblogic.kernel.Default (self-tuning)'] [userId: weblogic] [ecid: 2913acc884e13d35:1da4e73c:15ab855a0ff:-8000-000000000000002b,0] [APP: oracle-bam#11.1.1] [60] Failed executing BamAdcServer.ping due to ServiceUnavailableException. Configured retry count (3) has been exceeded. 
Solution

1. Wait for 15 minutes and try logging in again. It will work.
 
 
Applicable Versions
  • Oracle Business Activity Monitoring (BAM) 11g (11.1.1.9.0)

 

Saturday, July 29, 2017

WebGate 10g for Windows 2012 R2: "Could not read file" netlibmsg.xml

Problem

In this log file C:\NetPoint\Webgate\access\oblix\logs\oblog.log, you see the following:
2017/05/16@22:31:48.570000 1876 1521 INIT ERROR 0x000003B6 base\oblistrwutil.cpp:192 "Could not read file" filename^E:\NetPoint\Webgate\access//oblix/lang/en-us/netlibmsg.xml
Solution

1. No action needed. There is no adverse impact on WebGate functionality.

Applicable Versions
  • Oracle WebGate 10g (Windows 32-bit)