Sunday, May 8, 2011

Ant scripts for Oracle SOA Suite 11g code deployment (Windows)

This blog post provides Ant scripts that simplify the deployment, undeployment, and management of code to Oracle SOA Suite 11g. These scripts are designed for Windows (will release the Linux instructions soon). For those familiar with it, I do not use the configuration plan (long story).

Who these scripts are designed for:
  • Developers who want an automated approach to deploy/undeploy/manage code
  • App Admins who want to manage code from a Windows-based operating system

What you can do with the scripts:
  • Deploy multiple composites using a single Ant command
  • Undeploy multiple composites using a single Ant command
  • Start multiple composites using a single Ant command
  • Stop multiple composites using a single Ant command
  • List deployed composites
  • Tokenize and detokenize code
  • Import artifacts (XSDs, DVMs, WSDLs, fault policies) to the MDS
  • Export the entire MDS (for backup or browsing purposes)

To see how these scripts work, check out:


One-time Setup and Configuration

1. Download ant_soa11g_windows.zip from here

2. Unzip ant_soa11g_windows.zip to your local c:\ drive (it should exist as c:\ant)

3. Edit setEnvironmentVars.bat and modify the following:
CODE_FOLDER  <-- top-level directory where your code exists (e.g., c:\ant\code)
ORACLE_HOME  <-- directory of JDev 11g (e.g., c:\jdev11g)
JAVA_HOME    <-- ensure that the JDK path is correct
4. Edit soa-environment.properties and modify only the following variables:
wn.bea.home=C:/jdev11g              <-- directory of JDev 11g (similar to ORACLE_HOME above)
tmp.output.dir=c:/ant/temp          <-- temp directory for MDS generation, default is c:\ant\temp
local.mds.repository=c:/ant/apps    <-- location of local MDS copy (for MDS imports)
local.mds.export=c:/ant/apps.backup <-- location of local MDS backup folder (for MDS exports)
5. Double-click on winInstall.bat so that it copies the necessary JARs to the appropriate directories

6. For every SOA Suite 11g environment, create the following 3 files.
For example, if you have a DEV, TEST, and PROD environments, then create the following 9 files:
soa-build-dev.properties
soa-build-test.properties
soa-build-prod.properties
soa-cfgplan-dev.xml
soa-cfgplan-test.xml
soa-cfgplan-prod.xml
soa-token-dev.properties
soa-token-test.properties
soa-token-prod.properties
The ant_soa11g_windows.zip file includes samples for a DEV environment, so just make copies of those files for your other environments.
7. Edit all soa-build-*.properties and soa-token-*.properties files and modify the following:
USERNAME    <-- weblogic username (e.g., 'weblogic')
PASSWORD    <-- weblogic password
SOAHOST     <-- hostname or IP address of SOA Suite 11g server
SOAPORT     <-- port for the soa_server1 managed server (e.g., 8001)
DBUSERNAME  <-- MDS database username (e.g., 'dev_mds')
DBPASSWORD  <-- MDS database password
DBHOST      <-- database host
DBPORT      <-- database port
DBSID       <-- database name

Deploy, Undeploy, Start, Stop, and Detokenize Composites

1. Define the CustomProcessList.txt (see Define the Custom Process List below)

2. Open a command prompt window and type the following
cd c:\ant
call setEnvironmentVars.bat
call ant -f build.soa.xml <target> -Dtargetenv=<env>
Where <target> is deployComposites, undeployComposites, startComposites, stopComposites, or detokenizeComposites. This will loop through and perform the action against all the composites listed in the CustomProcessList.txt file.
Where <env> is your dev, test, or whatever environments you configured earlier.

List Composites

1. Open a command prompt window and type the following
cd c:\ant
call setEnvironmentVars.bat
call ant -f build.soa.xml listComposites -Dtargetenv=<env>
Where <env> is your dev, test, or whatever environments you configured earlier.

Export MDS

1. Open a command prompt window and type the following
cd c:\ant
call setEnvironmentVars.bat
call ant -f build.soa.xml exportMDS -Dtargetenv=<env>
Where <env> is your dev, test, or whatever environments you configured earlier.
2. The export file will be saved to c:\ant\apps.backup in the format of MDSbackup.<env>.YYYY-MM-DD-HHMI.jar (e.g., MDSbackup.dev.2011-01-20_1140.jar)


Import Artifacts to MDS

1. Define the CustomMDSList.txt (see Define the Custom MDS List below)

2. Add all the schemas, WSDLs, DVMs, fault policies, to the c:\ant\apps\<application> directory.
For example:
c:\ant\apps\App1MetaData\dvm\Currency.dvm
c:\ant\apps\App1MetaData\dvm\Country.dvm
c:\ant\apps\App1MetaData\config\fault-bindings.xml
c:\ant\apps\App1MetaData\config\fault-policies.xml
c:\ant\apps\App1MetaData\schemas\shared\Types.xsd
c:\ant\apps\App1MetaData\schemas\hr\Account.xsd
3. Open a command prompt window and type the following
cd c:\ant
call setEnvironmentVars.bat
call ant -f build.soa.xml importMDS -Dtargetenv=<env>
Where <env> is your dev, test, or whatever environments you configured earlier.

Define the Custom Process List

Before using any of these Ant scripts that are related to code, you must create a CustomProcessList.txt file.

1. Create the c:\ant\CustomProcessList.txt file, using the following format:
<composite_name>,<partition>,<revision>,<default_composite>,<relative_directory_of_code>

For example, the file may look like this:
HelloWorld1,default,1.0,true,hrprojects\employees\HelloWorld1
HelloWorld2,default,1.0,true,hrprojects\employees\HelloWorld2
HelloWorld3,hello99,2.0,false,hrprojects\test\synchronize\HelloWorld3
This says that there are 3 composites to loop through; HelloWorld1, HelloWorld2, and HelloWorld3. If deployed, the first 2 will be deployed to the 'default' partition and a revision of '1.0' while the last one will be deployed to the 'hello99' partition with a revision of '2.0'. The last composite will not be set as the default composite, designated by the value of 'false'.
  
The directory path shown is relative to the %CODE_FOLDER% environment variable set earlier. For example, if %CODE_FOLDER% is set to c:\ant\code, then the three projects above should exist in the following local directories:
c:\ant\code\hrprojects\employees\HelloWorld1
c:\ant\code\hrprojects\employees\HelloWorld2
c:\ant\code\hrprojects\test\synchronize\HelloWorld3

Define the Custom MDS List

Before using any of the Ant targets that are related to the MDS, you must create a CustomMDSList.txt file.

1. Create the c:\ant\CustomMDSList.txt file, using the following format:
<app_name>,false

For example, the file may look like this:
App1MetaData,false
App2MetaData,false
If you run the MDS import script, it will import the following local directory to its relative MDS directory as follows:
c:\ant\apps\App1MetaData  -->  oramds:/apps/App1MetaData
c:\ant\apps\App2MetaData  -->  oramds:/apps/App2MetaData
For example, you may have a combination of XSDs, DVMs, and fault policies as shown: 
c:\ant\apps\App1MetaData\dvm\Currency.dvm
c:\ant\apps\App1MetaData\dvm\Country.dvm
c:\ant\apps\App1MetaData\config\fault-bindings.xml
c:\ant\apps\App1MetaData\config\fault-policies.xml
c:\ant\apps\App1MetaData\schemas\shared\Types.xsd
c:\ant\apps\App1MetaData\schemas\hr\Account.xsd
When you run the MDS import script, because the CustomMDSList.txt file is referencing the local c:\ant\apps\App1MetaData directory, all files under that local directory will be imported to the MDS.

Adding Additional Tokens

The detokenizeComposites target loops through all code in your CustomProcessList.txt file, and detokenizes your code. For example, if your code has the following string @SOAServer@, the command below will replace it with whatever value is specified in the soa-token-*.properties file. This allows you to not hardcode values in your code.

If you want to add additional token variables, say @SomeEndpoint@, you must do the following:

1. Add the new token to the soa-token-*.properties file. For example:
SomeEndpoint = http://dev.raastech.com:7777
2. Edit build.soa.xml and add the token in two separate targets (detokenizeComposites and tokenizeComposites):
<replacefilter token="@SomeEndpoint@" value="${SomeEndpoint}"/> 
<replacefilter token="${SomeEndpoint}" value="@SomeEndpoint@"/>

35 comments:

SOA Technology for beginners and learners said...

Hi Ahmed nice blog...

i did all the steps as you described in the blog but i am getting error while deploying the composites..

as


scac:
[scac] Validating composite "c:\ant\code/hrprojects\employees\HelloWorldSai
lesh/composite.xml"
[scac] error: location {/ns:composite}: java.io.FileNotFoundException: c:\a
nt\code\hrprojects\employees\HelloWorldSailesh\composite.xml (The system cannot
find the file specified)
[scac] at java.io.FileInputStream.open(Native Method)
[scac] at java.io.FileInputStream.(FileInputStream.java:106)
[scac] at java.io.FileInputStream.(FileInputStream.java:66)
[scac] at oracle.soa.scac.ValidationFaultUtil.validateCompositeWithSche
ma(ValidationFaultUtil.java:101)
[scac] at oracle.soa.scac.ValidateComposite.validateWithSchema(Validate
Composite.java:1600)
[scac] at oracle.soa.scac.ValidateComposite.doValidation(ValidateCompos
ite.java:476)
[scac] at oracle.soa.scac.ValidateComposite.run(ValidateComposite.java:
150)
[scac] at oracle.soa.scac.ValidateComposite.main(ValidateComposite.java
:135)
[scac]
[echo]
[echo] ERROR IN TRYCATCH BLOCK:
[echo] C:\ant\build.soa.xml:150: The following error occurred while executi
ng this line:
[echo] C:\ant\build.soa.xml:165: The following error occurred while executi
ng this line:
[echo] C:\Oracle\Middleware\jdeveloper\bin\ant-sca-package.xml:46: The foll
owing error occurred while executing this line:
[echo] C:\Oracle\Middleware\jdeveloper\bin\ant-sca-compile.xml:269: Java re
turned: 1 Check log file : C:\DOCUME~1\SAILES~1\LOCALS~1\Temp\out.err for errors

[echo]
[echo] **************************************************************
[echo] *
[echo] * ANT TARGET NAME: deployComposites
[echo] * FAILED PROCESS: HelloWorldSailesh
[echo] *
[echo] **************************************************************
[echo]

BUILD SUCCESSFUL
Total time: 3 seconds

Ahmed Aboulnaga said...

The error you are receiving is during compilation. This is confirmed because the error is thrown by the [scac] target.

It seems that the ant script cannot find your composite. You can see this from this error:

c:\ant\code\hrprojects\employees\HelloWorldSailesh\composite.xml (The system cannot find the file specified)

If you navigate to that directory, you won't find the composite.xml file there (note that every SOA Suite 11g project must have a composite.xml).

So the problem is one of the following:

(1) You forgot to copy the composite to that directory.

(2) The code there is not a true SOA Suite 11g project.

(3) The string you used in your CustomProcessList.txt is pointing to an invalid directory.

(4) The property CODE_FOLDER you configured is pointing to the wrong source code directory.

SOA Technology for beginners and learners said...

you are awesome... thanks a lot lot lot ..its working now.. i gave the path wrong


for single bpel it is working fine i mean independent ,it is not working for dependent bpel process if i am
example if i have helloworld and another helloworld then it says it cannot take the .wsdl and xsd... fabric errror.. can you please guide me..

SOA Technology for beginners and learners said...

Hi Ahmed..

The Script work really fine..

but i have question that it works for independent composites

it is not working for dependent composites .. any idea on this..

example i have one composite with helloworld and another as callhelloworld.. this one fails...

Anonymous said...

But for my dependent composite it is working :)

SOA Technology for beginners and learners said...

Hi Ahmed..

in my project i was given an xml which contains db connections, how can i import that xml to create all the db connection required to my project

below is the code






ADVANCED


thin


apps





TRUE


oraJDBC


false


false


R12CRP


Oracle


jdbc:oracle:thin:@(DESCRIPTION=
(ADDRESS_LIST=
(LOAD_BALANCE=YES)
(FAILOVER=YES)
(ADDRESS=(PROTOCOL=tcp)(HOST=hqtdb2lx-vip.pamperedchef.com)(PORT=1536))
(ADDRESS=(PROTOCOL=tcp)(HOST=hqtdb1lx-vip.pamperedchef.com)(PORT=1536))
)
(CONNECT_DATA=
(SERVICE_NAME=erpcrp1)
)
)



05E3D91305CCD4B15AD3A2FD8E1ECE7B7B


true


oracle.jdbc.OracleDriver

Ahmed Aboulnaga said...

(1) Create a DB connection in JDeveloper manually.

See steps 1-4 in this post: http://blog.ipnweb.com/2010/10/creating-mds-connection-through.html

(2) Whatever JNDI you use in your database adapter MUST be created on the WebLogic Server on which you plan on deploying your code to.

SOA Technology for beginners and learners said...

hi ahmed.. i have two composites and i want to place those wsdls in mds and access them when i copied the ?wsld and pasted in the ie and right clicked and said view source.. and saved as .wsdk and placed in mds but when i am accessing that it says..

adf config is invalid but i have created the mds connection..

please can you help me in this

SoaExperts said...

Hi

trying to deploy the dependent composites through antscripts...however made the changes as per the blog.....but getting an the following error...

C:\ant>ant -f build.soa.xml deployComposites -Dtargetenv=soa-build-dev.prope
s
Buildfile: build.soa.xml
[taskdef] Could not load definitions from resource net/sf/antcontrib/antco
b.properties. It could not be found.
[taskdef] Could not load definitions from resource net/sf/antcontrib/antli
l. It could not be found.

deployComposites:

BUILD FAILED
C:\ant\build.soa.xml:146: Problem: failed to create task or type var
Cause: The name is undefined.
Action: Check the spelling.
Action: Check that any custom tasks/types have been declared.
Action: Check that any / declarations have taken place.


Total time: 1 second

Please help me out.....

Ahmed Aboulnaga said...

(1) Did you run step #5?

(2) Look at the contents of "winInstall.bat" to see where it's supposed to copy the files, and see if the files have actually been copied to their correct target location.

Ahmed Aboulnaga said...

Make sure that "ant-contrib-1.0b3.jar" is copied to "$MW_HOME/modules/org.apache.ant_1.7.1/lib".

SoaExperts said...

Thank you Ahmed....

It is working fine now and there is no problem. I was able to deploy the composites without any issues...

Ahmed Aboulnaga said...

What fixed your problem exactly?

SOA Technology for beginners and learners said...
This comment has been removed by the author.
SoaExperts said...

Thank you so much for your concern.... It was the same problem as u guessed...the "ant-contrib-1.0b3.jar" was not copied to the target location....
Hence the same was done....so my problem was fixed...

Anonymous said...

Hi Ahmed,

I am trying to run the deployment with a custom class reference in the BPEL in the composite. The problem is since it is a logger class and shared across multiple composites, we do not want to put the jar file having logger class in the SCA-INF/lib. How can I give the jar file to ant deployment script. I have tried below

1. CLASSPATH env set up
2. ant -lib
3. changing ant-sca-compile.xml to update the scac.tasks.class.path

But none of them is working

Anonymous said...

Ahmed,

This post helped me a lot. Really thankful to you.

Thanks,
Srini

aswathy shree said...

Hi Ahmed,

great post. thanks a ton.

I have a doubt. so as per your blog the structure of my MDS should be such that i have an apps folder is it?

when i tried exportMDS i get the below error

[exportSharedData] INFO: Received HTTP response from the server, response code=1
000
[exportSharedData] No jar file sent from server
[exportSharedData] ---->Cannot find any shared data of pattern:**.

BUILD SUCCESSFUL
Total time: 3 seconds

i tried importArtifact as well but got a response like this

D:\ant>call ant -f build.soa.xml importMDSArtifact -Dtargetenv=dev
Buildfile: build.soa.xml

importMDSArtifact:

BUILD FAILED
D:\ant\build.soa.xml:551: Directory D:\ant\ant emp\${mds.application} creation
was not successful for an unknown reason

Total time: 1 second

please help!!!

Ritu said...

Hi...

First of ALL thanks a lot for blogging such a crucial topic..

I have tried running your sample but unable to do so.

For example, for testing i am running listDeployedComposite target.When I run this ant script directly from command prompt(not from you build file i.e build.soa.xml), i get a succesful result eg.
C:\Oracle\Middleware1\Oracle_SOA1\bin>ant -f ant-sca-mgmt.xml listDeployedCompos
ites -Dhost=localhost -Dport=8001 -Duser=weblogic
Buildfile: ant-sca-mgmt.xml
[echo] oracle.home = C:\Oracle\Middleware1\Oracle_SOA1\bin/..

listDeployedComposites:
[input] skipping input as property host has already been set.
[input] skipping input as property port has already been set.
[input] skipping input as property user has already been set.
[secure-input] Please enter password:

[echo] oracle.home = C:\Oracle\Middleware1\Oracle_SOA1\bin/..

compositeMgrTask:
[java] Connecting to: service:jmx:t3://localhost:8001/jndi/weblogic.managem
ent.mbeanservers.runtime
[java] connection initiated
[java]
[java] Following 7 composites are currently deployed to the platform:
[java]
[java] 1. HelloWorldProject[1.0], partition=default, mode=active, state=on,
isDefault=true, deployedTime=2012-09-19T15:55:57.417+05:30
[java] 2. HelloWorldProject1[1.0], partition=default, mode=active, state=on
, isDefault=true, deployedTime=2012-09-21T15:37:02.660+05:30
[java] 3. Project1[1.0], partition=default, mode=active, state=on, isDefaul
t=true, deployedTime=2012-09-19T16:04:45.270+05:30
[java] 4. SimpleApproval[1.0], partition=default, mode=active, state=on, is
Default=true, deployedTime=2012-08-10T18:08:38.906+05:30
[java] 5. VC2Project[1.0], partition=default, mode=active, state=on, isDefa
ult=true, deployedTime=2012-09-24T20:28:45.412+05:30
[java] 6. VisitingCardProject[1.0], partition=default, mode=active, state=o
n, isDefault=true, deployedTime=2012-09-24T16:26:57.815+05:30
[java] 7. VisitingCard_Project[1.0], partition=default, mode=active, state=
on, isDefault=true, deployedTime=2012-09-22T15:30:17.539+05:30
[java]

BUILD SUCCESSFUL
Total time: 3 minutes 8 seconds


But when i run your scripts i get an error.

My setEnvironmentVars.bat is as follows:
REM ----------------------------------------
REM Top-level directory for your SOA 11g code
REM ----------------------------------------
set CODE_FOLDER=E:\AntApplication

REM ----------------------------------------
REM JDdevloper installation directory
REM ----------------------------------------
set ORACLE_HOME=C:\Oracle\Middleware1

REM ----------------------------------------
REM May need to be modified based on your JDK version
REM ----------------------------------------
set JAVA_HOME=%ORACLE_HOME%\jdk160_29


REM ----------------------------------------
REM No changes needed here
REM ----------------------------------------
set ANT_HOME=C:\Oracle\Middleware1\modules\org.apache.ant_1.7.1
set PATH=%ANT_HOME%\bin;%ANT_HOME%\lib;%PATH%
set CURRENT_FOLDER=%CD%
REM ----------------------------------------

Error i get:
C:\ant>ant -f build.soa.xml listDeployedComposites -Dtargetenv=dev
Buildfile: build.soa.xml

listDeployedComposites:

BUILD FAILED
C:\ant\build.soa.xml:471: The following error occurred while executing this line
:
java.io.FileNotFoundException: C:\Oracle\Middleware1\jdeveloper\bin\ant-sca-mgmt
.xml (The system cannot find the path specified)

Total time: 2 seconds
C:\ant>

I have checked the build file it reads {$oracle.home}/bin/ant....
I am not able to understand where jdeveloper is getting appended in the path
Please help ...urgent...

Ritu

Ahmed Aboulnaga said...

This value is set in "soa-environment.properties".

So update it to reflect your environment.

Ritu said...

Thank you ahmed for quick response...

Ritu

SOA Technology for beginners and learners said...

Hi Ahmed ,

Nice Post

Do u have an antscript to upgrade more than one bpel soa 10g project to soa 11g composites.

if provided it will be very usefull

thanks
saileshan.

Ahmed Aboulnaga said...

Unfortunately, I don't have anything ready that I can share.

Look at the following ant deployment scripts. Once you understand them, you can follow the same process for your upgrade.

http://blog.ipnweb.com/2011/05/ant-scripts-for-oracle-soa-suite-11g.html

http://blog.ipnweb.com/2011/05/demo-of-ant-scripts-for-oracle-soa.html

Kotresh said...

Hi Ahmed,
I did all the steps which you stated here..

while executing call ant -f build.soa.xml deployComposite -Dtargetenv=DEV

i am getting
"COuld not load defenitions from resource net/sf/antcontrib/antcontrib.properties. It Could not be found"

please help me out ... thank you in advance

Ahmed Aboulnaga said...

The "winInstall.bat" script is supposed to copy "ant-contrib-1.0b3.jar" to "%ANT_HOME%\lib".

- Did you run "winInstall.bat" as indicated in the instructions in this blog post?

- If so, does "ant-contrib-1.0b3.jar" exist in "$MW_HOME/modules/org.apache.ant_1.7.1/lib"?

- If not, then manually copy "ant-contrib-1.0b3.jar" which is located under the ~\lib subfolder to "$MW_HOME/modules/org.apache.ant_1.7.1/lib" and try again.

Kotresh said...

Thank you for the replay ahmed ,
But still the Error is same.
I am trying to deploy only one process and the same is saved in CreateProcesslist. ans i manually copied the ant-contrib-1.0b3.jar to C:\Oracle\Middleware\modules\org.apache.ant_1.7.1\lib

but still i am getting the same error
"failed to create task or type var

Cause : The name is undefined
action: check the spelling
action: check that any custom tasks/types have been declared
action : check that any / declarations have taken place"

i could not get "deployComposites" should i give my process name instead of this ?
please suggest me. thank you

Kotresh said...

Thank you vary much Ahmed, build was successful. i did mistake in copying ant-contrib-1.0b3.jar.

Anonymous said...

Great post. That is what I have been searching for. Thanks a lot.

Anonymous said...

You have fine tuned the code of Biemond very well

Anonymous said...

You have fine tuned the code of Biemond very well

Anonymous said...

Keep getting this error each time I try to undeploy. I can run listDeployedComposites fine and return all the composites. Receive the following error:

undeploy:
[input] skipping input as property serverURL has already been set.
[input] skipping input as property compositeName has already been set.
[input] skipping input as property revision has already been set.
[undeployComposite] setting user/password..., user=weblogic
[undeployComposite] compositeDN =${composite.partition}/${composite.name}!${composite.revision}
[undeployComposite] INFO: Creating HTTP connection to host:zn-prep-oraapp03.mfltest.co.uk, port:8400
[undeployComposite] INFO: Received HTTP response from the server, response code=500
[undeployComposite] ---->response code=500, error:There was an error undeploying composite on WLS_SOA1: Cannot find revision in undeploy : ${composite.partition}/${composite.name}!${composite.revision}..
[undeployComposite]

The CustomProcess.txt looks like this:

SOA_BackOffice,EnterpriseDealerRouter,1.00

I am running this on the SOA Server.

Many thanks for a great blogsite. Very useful info and I have bought your book too.

Regards
Z

Prudhvi said...

Is there any api for making a BPEL process as default ?
I am successfully able to activate and retire, after activating a process I am not able to make it as default

venkatesh a said...

Hi ,

I am getting the below error while deploying the composite. Could you please help on this.

build.xml:114: Property '${projects}.revision' is not defined

Thanks,
Venky.

Anonymous said...

Zip download link broken?

Ahmed Aboulnaga said...

The link in this blog post has been fixed, so you should now be able to download the zip file.

Please note that this was written in 2011. We have updated scripts (which includes a web front-end too!) that supports SOA, OSB, BAM, SQL/PLSQL, and Java. Please engage Raastech if you'd like to get a copy of that.

Thanks.