Monday, June 12, 2017

Example using orapki

Here is a quick example demonstrating how to list the contents of the default Oracle Wallet in Oracle WebTier 12c using orapki.

Keep in mind the following:
  • The default wallet is auto-login enabled, so it is not necessary to use the -pwd option to include the password.
  • orapki requires Java, so make sure to set your Java environment appropriately.
  • In some products, the default password for the default wallet is "welcome".


oracle@soahost1:/u01> export MW_HOME=/u01/app/oracle/products/fmw1221

oracle@soahost1:/u01> export JAVA_HOME=/u01/jdk1.8.0_102

oracle@soahost1:/u01> export PATH=$JAVA_HOME/bin:$PATH

oracle@soahost1:/u01> $MW_HOME/oracle_common/bin/orapki wallet display -wallet $MW_HOME/user_projects/domains/ohs_domain/config/fmwconfig/components/OHS/instances/ohs1/keystores/default -complete

Oracle PKI Tool : Version
Copyright (c) 2004, 2016, Oracle and/or its affiliates. All rights reserved.

Found Auto Login Only (ALO) wallet.
Option -pwd is not recognized and ignored.
Requested Certificates:
User Certificates:
Serial Number:  00
Key Length      1024
MD5 digest:     80:81:B2:49:CC:1E:1A:3C:6B:C6:23:C5:DF:BF:C3:32
SHA digest:     80:3E:47:A3:6D:8C:78:D0:00:99:48:FA:C9:1D:E2:2F:F8:FE:B0:D1

Trusted Certificates:
Serial Number:  00
Key Length      1024
MD5 digest:     80:81:B2:49:CC:1E:1A:3C:6B:C6:23:C5:DF:BF:C3:32
SHA digest:     80:3E:47:A3:6D:8C:78:D0:00:99:48:FA:C9:1D:E2:2F:F8:FE:B0:D1


Thursday, June 1, 2017

Getting OAM-02073 when WebGate agent communicates to Oracle Access Server


In this particular scenario, you are trying to single sign-on an OHS server against OAM 11g. The products currently installed are Oracle Access Manager and Oracle HTTP Server with Oracle WebGate deployed to it.

When we start up OHS, this error keeps repeating every 20 seconds in ohs1.log:
[2017-02-28T22:00:51.0079+00:00] [OHS] [ERROR:32] [OHS-9999] [core.c] [client_id: ::1] [host_id:] [host_addr:] [tid: 140187036239616] [user: oracle] [ecid: 000Fzzx0g00009Ya05IPtKHC04Fw00000a] [rid: 0] [VirtualHost: main] The Access Server has returned a fatal error with no detailed information.
This also shows up in oblog.log:
2017/02/28@21:59:51.00782 39099 39114 ACCESS_SDK ERROR 0x0000182D /ade/aime_ngamac_497961/ngamac/src/palantir/access_api/src/obresource_request.cpp:299 "The Access Server has returned a fatal error with no detailed information." raw_code^302  
2017/02/28@21:59:51.00789 39099 39114 ACCESS_GATE ERROR 0x0000151A /ade/aime_ngamac_497961/ngamac/src/palantir/webgate2/src/isprotected.cpp:296 "Failure to connect to Access Server" HTTPStatus^500 Error^The Access Server has returned a fatal error with no detailed information.  
2017/02/28@21:59:51.00793 39099 39114 WEB ERROR 0x0000151F /ade/aime_ngamac_497961/ngamac/src/palantir/commonlib/src/apache2_req_info.cpp:226 "WebGate Error Report" Message^The Access Server has returned a fatal error with no detailed information. ReqReq^HEAD /index.html HTTP/1.1 ReqProto^HTTP/1.1 ReqHost^ ReqStatLine^ ReqStatus^200 ReqRawUri^/index.html ReqUri^/index.html ReqFilename^/u01/app/oracle/middleware/Oracle_WT1/instances/ohs1/config/OHS/ohs1/htdocs/index.html ReqPath^ ReqArgs^ 
This is the error that appears in oam_server1.out managed server log:
<Feb 28, 2017 9:59:11 PM GMT> <Warning> <oracle.oam.controller> <OAM-02073> <Error while checking if the resource is protected or not. 
<Feb 28, 2017 9:59:11 PM GMT> <Error> <oracle.oam.proxy.oam> <OAM-04029> <Error in generating AMEvent. Details Event Response status is STATUS_FAIL for GET_AUTHN_SCHEME event. Error code OAM-02073 status fail isExcluded false 
<Feb 28, 2017 9:59:11 PM GMT> <Error> <oracle.oam.proxy.oam> <OAM-04020> <Exception encountered while processing the request message: Event Response status is STATUS_FAIL for GET_AUTHN_SCHEME event. Error code OAM-02073 status fail isExcluded false
at Source)
at weblogic.ejb.container.internal.MDOMethodInvoker.invoke(
at Source)
at org.apache.mina.common.DefaultIoFilterChain$TailFilter.messageReceived(
at org.apache.mina.common.DefaultIoFilterChain.callNextMessageReceived(
at org.apache.mina.common.DefaultIoFilterChain.access$1200(
at org.apache.mina.common.DefaultIoFilterChain$EntryImpl$1.messageReceived(

1. Check out these Oracle Support docs to see if it solves your problem:
  • Doc ID 1678095.1
  • Doc ID 1556223.1
  • Doc ID 1492637.1
  • Doc ID 1662513.1
2. If none of the solutions above works for you, then check out your Host Identifier configuration. You may have incorrectly deleted the row as shown in this screenshot:

Applicable Versions
  • Oracle Access Manager
  • Oracle HTTP Server
  • Oracle WebGate


Tuesday, May 30, 2017

Raastech @ BGOUG

Will you be attending BGOUG this June 2-4 in Pravets, Bulgaria? If so, then check out some of the presentations we'll be giving!

Oracle Compute Cloud vs. Amazon Web Services EC2 – A Hands-On Showdown
We walk through the end-to-end process of provisioning an Oracle Compute Cloud instance from scratch - adding storage, defining firewall rules, creating a private key, and connecting through VNC. Then we do the same thing with Amazon Web Services (AWS) EC2. Speed of provisioning is one of the benefits of the cloud, so let's embark on this live walkthrough together, see how easy it is to get up and running, and compare the IaaS offering from Oracle and Amazon.
Sat, June 3, 2017
12:30pm - 1:30pm

Developing Web Services from Scratch – For DBAs and Developers
WSDL. XSD. SOAP. Namespaces. Port types. If these terms make little sense, this presentation is for you. By the end of this presentation, you will completely understand how to dissect and decipher a web service interface, understand key design patterns, and learn how to develop top-down and bottom-up web services in technologies such as Java and Oracle SOA Suite. Want to know how to expose a PL/SQL package as a web service? This technical presentation, one of my most popular, is intended for DBAs and database developers who want to know what it takes to design and create web services.
Sat, June 3, 2017
4:00pm - 5:00pm

See you there!

Tuesday, May 16, 2017

Maven: Digging Deeper with -X

While I like many people sometimes I have to run over to StackOverflow to figure out an issue. I have made a habit of working through the issue myself first. So when I encounter an error like this one below.

A lot of times people will just take goal org.apache.maven.plugins:maven-enforcer-plugin:1.3.1:enforce failed. NullPointerException and place it into the search engine trying to figure out what the issue is, by going through a few of the links seeing if they have solutions. Maybe try them, get down a further whole, etc. Let's do it different this time.

Take the solution into our own hands. The first thing you need to do is just read the rest of the output. If you notice there's an option -X which provides full debug output, so our command should look something like mvn clean install -X where the -X it simply appended to your original command. Now as a forewarning this will output a TON, so be prepared for that. If your cli window starts clipping history too soon you may want to opt for redirecting the output to a file. For example: mvn clean install -X > output.log This should give you the ability to browse through the output and search for potential issues.

Now with the command completed again we can look for the issue. I generally like to start from the bottom. So if you are using vi you can use the :$ command to go the the last line, and begin paging up from there. In most cases since the solution can be found by seeing just the error. In more complex situation you may want to see what warnings occur further up in the output that may clue into what may actually be causing the error. So what is our issue in this case?

It appears that something is wrong with one of our libraries, and it appears to be corrupt. This was likely caused by an incomplete retrieval of the dependency. To solve this simply remove the parent folder with rm -rf /Users/harolddost/.m2/repository/com/sun/jersey/jersey-core/1.5 and then run maven again.


I hope that this guide has been helpful, and maybe a little bit inspiring to not just look for solutions others have come up with, but to come up with your own.

Happy Troubleshooting!

Monday, May 8, 2017

Getting "Invalid method in request \x16\x03\x03" after configuring SSL in OHS


You may have received the following obscure error when trying to start up OHS after configuring SSL. This error would appear in the ohs1.log file:
[2017-05-26T03:36:34.5467+00:00] [OHS] [ERROR:32] [OHS-9999] [core.c] [client_id:] [host_id:] [host_addr:] [tid: 148714440300748] [user: oracle] [ecid: 005ILyaQOA000E8d09q0yige6aMWV00028] [rid: 0] [VirtualHost: main]  Invalid method in request \x16\x03\x03 

"This error is due to a misconfiguration of VirtualHost with SSL. The server is trying to respond to a request on port 443 with unencrypted HTTP. In other words, your browser is expecting SSL, but the server is sending plain HTTP on port 443." (Source)

1. Edit ssl.conf.

2. Confirm that the Listen and VirtualHost tags are using the same hosts and ports, and correct them if necessary:
Listen 443
<VirtualHost *:8080>
Listen 443
<VirtualHost *:443>
3. Restart OHS.
4. If that does not work, try putting the fully qualified domain name:
Listen 443
<VirtualHost *:443>

Applicable Versions
  • Oracle WebTier 11g (


Thursday, May 4, 2017

Check if an XML file is well-formed in Linux

Do you want to know if a particular XML file is well-formed or malformed in Linux? Consider using xmllint.

If the return result is 0, then the XML file is well-formed:
oracle@soahost1:/tmp> xmllint --noout config.xml; echo $? 

If the return result is 1, then the XML is malformed:
oracle@soahost1:/tmp> xmllint --noout config.xml; echo $? 
config.xml:7: parser error : Opening and ending tag mismatch: domain line 2 and configuration-property 
config.xml:8: parser error : Extra content at the end of the document 

If you want to recursively check all XML files, this command will do it for you:

for i in `find . -name "*.xml"`; do echo $i; xmllint --noout $i; echo $?; done

Thursday, April 13, 2017

Installing the Oracle APM Java Agent on WebLogic

Oracle Application Performance Monitoring (APM) Cloud Service provides development and operations teams with the information that they need to find and fix application issues fast. All your application performance information (with associated application logs) are brought together into Oracle Management Cloud’s unified platform.

This blog post describes how to install the Oracle APM Java Agent for WebLogic, so that you can immediately start monitoring your WebLogic environments.

Get the Agent registration key

The Agent "registration key" is required during the installation of the agent. This identifies the APM Cloud Service that the agent is supposed to connect to.

1. Login to the Oracle Cloud My Services:<identitydomain>/faces/dashboard.jspx
2. Open the APM Service Console:

3. Click on the Application navigator icon:
4. Under Administration, click on Agents.

5. Click on Registration Keys.

6. Copy your Registration Key Value.

Download the Agent onto the Linux server running WebLogic

The Agent is normally downloaded from the Oracle Management Cloud service (it is a small zip file), and must be manually copied to your Linux server. For simplicity, the instructions below allow you to easily download it from a hosted environment directly to your server.

7. Copy the Agent to your Linux server.

Install and configure the Agent

Installing the Agent simply involves executing a few command line scripts, modifying the WebLogic startup script, and bouncing the WebLogic managed servers.

8. Set the environment in preparation of installing the APM Agent:
export STAGE_DIR=/u01/app/oracle/apm_agent
export DOMAIN_HOME=/u01/app/oracle/user_projects/domains/base_domain
export REG_KEY=<your registration key>
export GW_HOST=
export GW_PORT=
9. Install the APM Agent:
10. Provision the APM Agent:
cd ${STAGE_DIR} 
chmod +x
./ -d ${DOMAIN_HOME}
11. When prompted, enter "Y" and press ENTER.
Do you wish to proceed with these values? Y
12. Edit the WebLogic startup script:
vi ${DOMAIN_HOME}/bin/
13. Add the Java agent line right after the "" call:
JAVA_OPTIONS="${JAVA_OPTIONS} -javaagent:${DOMAIN_HOME}/apmagent/lib/system/ApmAgentInstrumentation.jar"

14. Restart the AdminServer and all managed servers:
cd $DOMAIN_HOME/bin 
nohup ${DOMAIN_HOME}/ >> ${DOMAIN_HOME}/AdminServer.out &

Here is the Oracle documentation for installing the Oracle APM Java Agent on WebLogic.


This blog post is part of a series of blog posts related to the Oracle Management Cloud:

Wednesday, April 12, 2017

Alerting in Oracle Application Performance Monitoring (APM)

Oracle has a "cloud" service that they offer as part of their Oracle Cloud ecosystem called Oracle Application Performance Monitoring or Oracle APM for short. While there are many features that may be useful, the value of monitoring is not that you can just look at the pretty graphics.

Some of the real power that can come from APM is around alerts, and being able to set them up on the various metrics which are available. To start click on Alerts on the left menu.

This will take you to a page where you hopefully won't have anything listed, and at most they will hopefully be warnings. Below is just a sandbox environment so I'm not to worried about any of these.

Now if you actually want to get notifications, you will have to create them. Start off by Select the service you want to use.

Next click Create Alert Rule so that you can start putting together the rule.

 First give it a descriptive name. In this case I chose API_SLAs. Because in this case we just want to make sure that our applications are performing within the SLAs that the business has defined.

Next you have to add the entities that you want the rule to pertain to.

For the purposes of this I am just sticking with APM Server Request. However, you can choose applications and individual paths within those applications should that be necessary.

Once you have the items that you want, start by choosing conditions under which you want to start receiving the alerts.

There are three main categories are Fixed Metric, Anomaly, and Early Warning. For our purposes here, I'll be used all three. First maintain that the error percentage is below certain percentages. Across a 5 minute window.

Next we are making sure that our Average response time remains consistent.

We are also going to check that the response times are specifically sub-second.

With all of these conditions, the last thing we need to do is send the notifications to somewhere.

That's it! Now anytime attache applications fall outside of our defined SLAs we will get notified so that we can determine what seems to be going on.

This blog post is part of a series of blog posts related to the Oracle Management Cloud:

Tuesday, April 11, 2017

Overview of Oracle APM Cloud Service features

The interface to Oracle APM Cloud Service provides access to an incredible array of features and capabilities, giving you deep insight into performance of your applications. It also includes insight into actual end-user experience, AJAX performance, log analytics, and more.


The Oracle APM Cloud Service dashboard provides what you would expect from it... a consolidated and aggregated view of your entire APM landscape. From here, you can get an executive level birds eye view of your environment.


You can group together a set of pages or server requests into a single "application" that match a particular filter criteria. Over time, it is not unusual to expect tens of thousands of individual pages and server requests to be monitored, making the need to group them that much more necessary.


Pages are exactly that. End-users access various pages of your web application, and each page is individually tracked and monitored, with full drill-down capability.


You can monitor end-user experience through "sessions". The first screen here shows you a list of all user sessions within the timeframe you selected.

Clicking on a session takes you the exact history of that session, with full performance metrics on each page the user has navigated to.


All application servers are listed here, with high-level aggregated metrics shown for the selected timeframe.

Clicking on any application server takes you to more detailed metrics specific to that server. From here, you can further dig deeper into the server requests tied to this app server, or even kick off the thread profiler to capture more detailed information.

Server Requests

From here, you can assess server-side performance down to the operation and method level. The APM agent is intelligent enough to identify the downstream links to this particular server request, as well as all callers to the request.

Detailed metrics are captured, including actual exceptions of each request.

This post is intended to give you a glimpse into the Oracle APM Cloud Service. There is a host of many other features not described here, and more information can be found in the documentation.


This blog post is part of a series of blog posts related to the Oracle Management Cloud:

Types of Oracle APM Agents

Multiple flavors of the APM Agent exist:
  • APM Java Agent
  • APM .Net Agent
  • APM Node .js Agent
  • APM on Mobile Clients
The APM Java Agent is supported on many of the leading Java application servers on the market, including:
  • Oracle WebLogic Server
  • Oracle PeopleSoft
  • Apache Tomcat
  • Red Hat JBoss
  • IBM WebSphere

For more information, check out the documentation.


This blog post is part of a series of blog posts related to the Oracle Management Cloud:

What is the Oracle Management Cloud?

"Get results when you need them."
"Built on a smart, unified platform."
"Powered by machine learning."

Forget the marketing jargon. Oracle Management Cloud is a service that consists of 4 interrelated monitoring services:
  • Application Performance Monitoring
  • Infrastructure Monitoring
  • Log Analytics
  • IT Analytics

Through the use of agents deployed to your hosts, data is collected on those target hosts and pushed in near real-time to the Oracle Management Cloud Service, providing a single, unified interface to your entire environment. It's very easy to deploy, no infrastructure costs are needed, and is easily scalable.


This blog post is part of a series of blog posts related to the Oracle Management Cloud:

Monday, April 10, 2017

Oracle Coherence requires port 7


When starting up the SOA server, you find the following repeating every 5 seconds in the logs:
[2017-03-09T18:28:22.148+00:00] [soa_server1] [WARNING] [] [Coherence] [tid: Logger@5117501122] [userId: <anonymous>] [ecid: 0000LeoJxT4iW1OkPqb000003x73b605zz,1:31280] [APP: soa-infra] 2017-03-09 18:28:22.148/296.247 Oracle Coherence GE <Warning> (thread=Cluster, member=n/a): Delaying formation of a new cluster; IpMonitor failed to verify the reachability of senior Member(Id=1, Timestamp=2017-03-09 18:24:34.601, Address=, MachineId=12345, Location=site:,machine:soahost2,process:4447, Role=WeblogicServer); if this persists it is likely the result of a local or remote firewall rule blocking either ICMP pings, or connections to TCP port 7 


1. Open up your firewall and allow port 7 between both your servers.

Port 7 is the default port of the IpMonitor component that is used for detecting hardware failure of cluster members, available in 3.6.0 and later versions of Coherence. I yourt is the default port for the Echo Protocol and, consequently, used by Coherence as the "death detector" port. This means that Coherence uses port 7 to determine if a member of the cluster is reachable (not "dead").


  • Oracle Doc ID 1530288.1
  • Oracle Doc ID 1526745.1
  • Oracle Doc ID 1472388.1

Applicable Version:

  • Oracle SOA Suite 11g (

Logging into Oracle APM Cloud Service for the first time

Logging into Oracle Application Performance Monitoring Cloud Service the first time is straightforward. This post shows you the exact screens you should expect to see when navigating to the Oracle APM Cloud Service dashboard for the first time.

1. Navigate to your identity domain. This link is available in your welcome email.<youridentitydomain>/faces/dashboard.jspx

2. Simply enter your login information:

3. You will be taken to the My Services page where you can see the various Oracle Management Cloud services, of which Oracle APM Cloud Service is one of them, here denoted as "apmtrial0796":

4. Click on the menu icon and select "Open Service Console".

5. On the Oracle Management Cloud welcome page, click on the large purple square for Application Performance Monitoring.

6. You are now taken to the default Oracle APM Cloud Service dashboard from which you can navigate to the various features and capabilities of the service.


This blog post is part of a series of blog posts related to the Oracle Management Cloud:

Tuesday, April 4, 2017

Oracle Application Performance Monitoring (APM) Cloud Service - Hands On Lab @ Collaborate 17

Application Performance Monitoring (APM) has gained steam over the past few years and has been instrumental in real-time application monitoring.

APM products have now shifted to the cloud, wherein agents are installed on your local servers and communicate to a central APM cloud service.

Oracle Application Performance Monitoring Cloud Service provides the ability to monitor applications effectively, with all the advantages of a light-touch, software-as-a-service (SaaS) solution. It integrates user experience information and server-side application metrics along with in-context application logs, so you can rapidly isolate, triage, and diagnose issues affecting application performance.

Tomorrow, the rockstar team of Ahmed Aboulnaga, Erik Benner, Harold Dost, and Alfredo Krieg have developed various hands-on labs that cover Oracle Management Cloud components that include APM, Log Analytics, and Infrastructure and is available for free for all Collaborate 17 attendees.

Here is a link to the hands-on lab:
Learn how to:
  • Install and configure the cloud agent
  • Monitor application success and failure scenarios
  • Navigate and learn about the various features and capabilities of the Oracle Management Cloud
  • Monitor the cloud agent

Check out some nice screenshots you can expect to see: