I fell in love with WebSphere mediations the first time I saw them. Okay, didn't fall in love but I did think... "Nice! What a cool way of going above and beyond what soap handlers offer!" For those who don't know, WebSphere mediations run in WAS runtime messaging engine, which treats any request or response flowing through as a message (could be a soap or jms message or even a request to an ejb) and any message consumer as a destination. Since we are [over] simplifying already, in simple terms WebSphere mediations are components that can intercept these messages, inspect them, route them to a different location or even change them.

Now, I wouldn't be writing this blog if everything about WebSphere mediations was so cool (yeah, I know, I have love hate relationship with IBM products!). Just note that the discussion in this blog entry refers to mediations available within WebSphere Application Server.

First of all, did you notice I call them WebSphere mediations? You won't see that term being used on any IBM website. I use that term because they use WebSphere specific APIs. You have to implement com.ibm.websphere.sib.mediation.handler.MediationHandler interface to create a mediation. This makes your code... well, not so portable. Mediations - woops, sorry - WebSphere mediations are very much like soap handlers. They can intercept the incoming requests and/or response and do things with it. There should have been some effort to make them part of the specification. I mean the whole concept of messaging engine is cool, even that should be part of the specification. The term 'messaging engine' doesn't refer to JMS messaging engine. It refers to the messaging infrastructure on which WAS runtime and service integration BUS are built. 

Second, they only support one method public boolean handle (javax.xml.rpc.handler.MessageContext context) method. Why can't they support both handleRequest and handleResponse like soap handlers do? Being able to handle response would be really convenient. For example, if I had a requirements to process an incoming message and forward it to a queue for further processing ONLY IF the initial processing was successful, I would catch the response message in the handleResponse method and then send the message for further processing. Without the ability to capture responses, you have to be little bit creative to implement this requirement (you can still do it though).

Third, they use SDO APIs. This is not a problem in itself but I don't like IBM's implementation of this API. Just try creating a soap header within mediation if no header exists already and you will know what I am talking about. In plain English, I find IBM's implementation of SDO very 'read' friendly as opposed to 'read and manipulate' friendly. It is not easy to create/add new elements.

Forth, they can't be developed. Seriously! You cannot create mediations in RAD 7.0 - not even after installing some fixpacks. Now, I know this is not a problem with the mediations themselves, it's a problem with the tooling but hey I am in the mood to rant. To develop a mediation, you create an ejb project, create a class [say] MyMediation and implement the interface, go to ejb deployment descriptor and click Add button in Mediation Handlers tab. A dialog box should pop up and the class you created should show up in it. You select the class and the necessary ejb code is generated automatically. In RAD 7, nothing happens if you click the Add button. When I used Application Server Toolkit at my last client, the dialog box popped up but my class never showed up in it. I had to create mediation project in RAD 6 (yes, it works in the previous version!) and then open it in RAD 7. Nice way to take away some functionality!

Fifth, you can't register your service in the SDO repository - something you have to do when you have an outbound service that you want to mediate using mediation. I have blogged about this in detail in this and this blog entries.

Sixth, you can customize the behavior of your mediations using mediation properties - that's a good thing. But, you have to define and change these properties using admin console. This is not flexible, and boring too if you have many properties to define. It requires a technical resource to make a change - doesn't sound like providing business agility to me.

Anyway, I want to talk about its installation. I have always used this tool on windows and it always installed itself without any problems. You can download an exe file from here. But things aren't as easy on Linux (isn't that why most of the people still find windows easier to use?). Freemind requires several packages to be installed successfully on Linux. The easiest way to deal with these dependencies is to download Freemind  freemind-0.8.0-6.noarch.rpm file from here. Become root using su command and install the file using rpm -ivh <file_name> command. It will fail with a message that looks like:

error: Failed dependencies:
        jakarta-commons-lang is needed by freemind-0.8.0-6.noarch
        relaxngDatatype is needed by freemind-0.8.0-6.noarch
        msv-xsdlib is needed by freemind-0.8.0-6.noarch
        jgoodies-forms is needed by freemind-0.8.0-6.noarch
        xml-commons-apis is needed by freemind-0.8.0-6.noarch
        ws-jaxme is needed by freemind-0.8.0-6.noarch
        jakarta-commons-codec is needed by freemind-0.8.0-6.noarch

Now, go to JPackage website. Download and install every missing package listed above and then try installing Freemind again. Do you have to go through the pain of installing all these packages when all you need is a simple mind mapping tool? Not really! In the past I have used kdissert at http://www.kde.org.uk/apps/kdissert/, which is relatively easy to install but I am used to Freemind so I will stick to it.

Starting v6.1, WebSphere separates administrative security from application security. The term administrative security means you need username and password to administer the server. This includes getting server status, stopping the server (somehow it doesn't include starting the server, you can start the server without supplying username and password) and administer the server through admin console or through wsadmin scripts. The term application security means you can protect resources in your application (e.g. you can protect /profile url), assign application security roles to those protected resources and map users or groups from ldap to those roles. It is important to note that starting WebSphere 6.1, you can choose to enable only administrative security or only application security or both. In previous versions you were required to enable or disable both at the same time. For now, we will only enable administrative security, we will talk about application security in the future when we have a sample application. This assumes that

1. you have already created a WebSphere 6.1 profile with security disabled (if you want to know how to do that Send me an email)
2. WAS is running
3. you have installed Apache DS and created users and groups
4. Apache DS ldap server is running

• Go to WAS admin console http://<host>:port/admin. For example, http://localhost:9060/admin
• Click on Security > Secure administration, applications, and infrastructure
• Select Standalone LDAP registry in Available realm definitions dropdown and click Configure
• Enter the following values in the screen:
• Primary administrative username: wasadmin (the admin user already created in ldap)
• Select Server identity that is stored in the repository radio button (because we don't want WebSphere to generate an id for us)
• Type of LDAP server: Custom (Apache DS is NOT one of the supported ldap servers)
• Host: localhost
• Port: 10389 (port on which Apache DS is listening)
• Base distinguish name (DN): dc=example,dc=com (all our groups and users are under this base dn)
• Bind distinguish name (DN): uid=wasadmin,ou=people,dc=example,dc=com (entry that uniquely identifies wasadmin user)
• Bind password: <password for wasadmin user you used in when you created user in LDAP>
• Leave default values for other options
• Click Apply
• Click Save directly to the master configuration link
• Click Advanced Lightweight Directory Access Protocol (LDAP) user registry settings link under Additional Properties section
• User filter: (&(uid=%v)(objectclass=inetOrgPerson))
• Group filter: (&(cn=%v)(|(objectclass=groupOfNames)(objectclass=groupOfUniqueNames)))
• User ID map: *:uid
• Group ID map: groupOfNames:member;groupOfUniqueNames:uniqueMember
• Click OK
• Click Save directly to the master configuration link at the top of the page
• Click OK and Save again
• Click Security > Secure administration, applications, and infrastructure link on the left hand panel
• Select Enable administrative security checkbox 
• Unselect Enable application security checkbox
• Unselect Java 2 security checkbox
• Select Standalone LDAP registry in Available realm definitions dropdown and click Set as current button
• Click Apply button 
• Click Save directly to the master configuration link

We have enabled administrative security but the change will not take effect until we restart the server. Restart the server and go to admin console. Since security is enabled, you will be asked to accept a certificate and login using the WAS admin username and password you specified in ldap. From this point on, Apache DS must be running before you can administer the server.

Edit on Sunday Feb 18, 2007: Corrected the value of Bind distinguish name (DN): to read

Bind distinguish name (DN): uid=wasadmin,ou=people,dc=example,dc=com (entry that uniquely identifies wasadmin user)

As we will see shortly, a user may be part part of an organizational unit AND may belong to a group.

Creating groups and users

The high level steps to create users and groups are:

1. Create organizational units to hold users and groups
2. Create wasadmin user
3. Create application user
4. Create groups

To create the above items, you can specify all the information in a text file, commonly known as ldif file and load the file in the ldap server. Create a file called organizational_units.ldif and put the following content in it. If you are completely unfamiliar with LDAP concepts, this article may be a good start.

In JXplorer, select LDIF >  Import File, select this file and click Open. This will result in two new organizational units, ou=people and ou=groups under dc=example,dc=com. 

We will put all users (including system users) in people ou and all groups in groups ou. Let's create users first. Create a file called users.ldif and put the following content in it.

Importing the file above creates 2 users under ou=people,dc=example,dc=com. For example, the dn of the wasadmin users becomes, uid=wasadmin,ou=people,dc=example,dc=com. We will use this dn to login to WAS admin console after we enable administrative security. 

To create groups, create a text file called groups.ldif and put the following content in it:

Notice how both the groups use the member attribute and point to the dn of its members. This attribute is required because our groups use groupOfNames object class. Since the member attribute is required, we created users first and then the groups. At this point, your ldap settings in JXplorer should look like this:

That's it! We are now ready to enable security in WebSphere 6.1. I will describe that process in the next entry.

Apache DS is a lightweight ldap server written in Java. You can think of it as an open source competitor to open ldap, IBM Tivoli Directory Server, SunOne Directory Server etc. We will install ApacheDS, make some configuration changes and use JXplorer to view/create entries in it.

Installing Apache DS is easy, configuring it after installation is little tricky:

• First of all, download apacheds-1.0.0-win32-setup.exe from here
• Double click the exe and follow the screens. I will call the installation directory <ds_home>. For example, <ds_home> could mean C:\apacheds-1.0.0
• At the end of the installation, run the configuration, which allows you to set logging behavior and server startup properties of the directory server. I accepted all the defaults, clicked Start button to start the server and clicked OK button to close configuration wizard

• Start JXplorer, click on File > Connect and enter the following values
• Host: localhost, Port: 10389, Protocol: LDAP v3, Base DN: leave blank, Level: User + Password, User DN: 'uid=admin,ou=system' (without quotes), Password: secret
• Expand System tree, select admin user and click on Table Editor tab on the right hand panel
• Click the value column on userPassword field and a User Password Data dialog pops up
• Change the password in the dialog and click OK. Remember this password, you will need it in the future!
• Click Submit button
• Disconnect from Apache DS in JXplorer using File > Disconnect
• Stop Apache DS using Service Settings menu option in the Start menu of windows
• Change the value of java.naming.security.credentials property in <apache_ds>\conf\server.xml to the new password you entered in JXplorer (yes the password is stored as clear text in Apache DS; this will change in a future version though)
• Save the file and start Apache DS
• Use JXplorer to login with new password and make sure it works

While the discussion of ldap concepts is NOT the intent of this article, here is a quick primer so that the discussion that follows will make more sense. Information is can be organized in ldap primarily using 2 styles: geographic and domain-based. Geographic style, typically used by multinational organizations, have information organized in trees that look like c=us,o=ibm etc. where c stands for country and o stands for organization. Domain based trees contain entries that look like dc=com,dc=ibm etc. where dc stands for domain component.

Once you have decided on a style, you can put your data under the tree. For example, an organization may divide its people in organizational units (represented as ou in LDAP). In that case, you will create hierarchy of ou entries. For example, ou=HR and under ou=HR, ou=benefits, ou=recruiting. 

Start JXplorer to make more sense of the information in this paragraph. Apache DS comes with ou=system, which stores the admin user for Apache DS. This is why we logged in with uid=admin,ou=system in JXplorer. Under ou=system, there are two organizational units: ou=groups and ou=users. You would put users as in bob, mike etc. under ou=users and you would create groups as in admins, users etc. under ou=groups. 

The term Access Control refers to the configuration that allows an ldap user access to a limited view of the entire tree (admin user has access to the entire tree). By default, access control is off in Apache DS, so a user (e.g. bob) created somewhere in the tree, will be able to see all other users in the tree. The term 'see all other users' means when bob logs in, he will see other users in the tree and make changes to them even though bob is not admin. This can be prevented by configuring access control in Apache DS OR by putting bob in ou=users and bob's group in ou=groups. In Apache DS, ou=users and ou=groups have special meanings. Users and groups created under those organizational units will NOT have access to other users in the tree even if access control is off and not configured. 

Why is this discussion important to us? We are going to create a WebSphere administrator in ldap and we are going to map J2EE application roles to ldap groups in WAS admin console. This is possible ONLY IF, the WebSphere admin user is able to search and view other users in the tree. To do this without configuring access control (and hence make our lives simpler), we will create ALL users and groups outside ou=users and ou=groups. Apache DS comes with dc=example,dc=com base dn entry. We will create our users and groups under this base dn. If you want to use something other than 'example' as a domain component (e.g. dc=yourorg, dc=com), replace ALL occurrences of example by yourorg in server.xml file.

Next time, we will add organizational units, users and groups to Apache DS.

A few days back, I asked IBM to give me my 3 days back. Obviously, that's not happening. So here is my contribution to the community so others don't have to loose time like I did. You can create a BUS and an outbound service as described in the series of articles at http://www-128.ibm.com/developerworks/websphere/techjournal/0512_reinitz/0512_reinitz.html. The link points to the last part of the article, which contains links to other parts.

Let's understand creating outbound service in little more detail. In WebSphere 6.0, creating an outbound service

• Automatically creates a record in the SDO repository database
• The record consists of your wsdl file inserted into the database as a blob
• The record uses a string in dest:<bus_name>:<service_name> format as the primary key. So, for an outbound service destination named MyService in bus MyBus, the wsdl is inserted using dest:MyBus:MyService as the key
• When you forward the message from queue to the outbound service, you must use a mediation to make WAS messaging engine aware that this is a soap message by applying a so called 'format string'
• The format string must be of the format SOAP:dest:[busname]:[service destination name],[service namespace],[service name],[port name]

• Out of this format string, WAS uses dest:[busname]:[service destination name] part as the key to retrieve the wsdl from the database

In WebSphere 6.1, everything else remained the same BUT format string and the key used to insert the wsdl in the database changed. In WebSphere 6.1, creating an outbound service

• Automatically creates a record in the SDO repository database
• The record consists of your wsdl file inserted into the database as a blob
• The record uses wsdl location as the primary key. Something like http://localhost:<port>/<context_root>/services/MyService/wsdl/MyService.wsdl
• When you forward the message from queue to the outbound service, you must use a mediation to make WAS messaging engine aware that this is a soap message by applying a so called 'format string'
• The format string must be of the format SOAP:<wsdlLocation>,<serviceNameSpace>,<serviceName>,<portName>
• Out of this format string, WAS uses <wsdlLocation> part as the key to retrieve the wsdl from the database 

If you use WAS 6.0 format string in WAS 6.1, you will get a validation error. The bug in WAS 6.1 is, when you create an outbound service, WAS 6.1 still uses the old format of key to insert the record BUT forces you to use the new format to retrieve the record so you will NEVER get any results back. Since WAS 6.1 uses entity beans to access SDO Repository, you will get ObjectNotFoundException.

To recreate this bug, create an outbound service following instructions in http://www-128.ibm.com/developerworks/websphere/techjournal/0506_reinitz/0506_reinitz.html

• Make sure the server is running
• Start the wsadmin tool by going to <profile_home>/bin and issuing wsadmin command (wsadmin.sh on linux/unix). If you are on a secure server, you will have to use -user and -password flags to use admin commands in wsadmin tool
• Use the following commands to list the resources in your SDO repository
• wsadmin> set sdoRep [$AdminControl queryNames *,type=SdoRepository,node=[$AdminControl getNode]] this command defines a variable called sdoRep
• wsadmin> puts [$AdminControl invoke $sdoRep listResources] this command lists the keys by which xml files are stored as blobs in your SDO repository

• If you have defined outbound service correctly, you will see an entry dest:<bus_name>:<service_name>. For example, if you had an outbound service named MyService in bus MyBus, you will see an entry dest:MyBus:MyService
• Use "SOAP:dest:MyBus:MyService" as the format string in your mediation and you will get a validation error in ffdc log files. This is because WAS 6.1 forces you to use wsdl location
• Use SOAP:<wsdlLocation>,<serviceNameSpace>,<serviceName>,<portName> as the key and you will get ObjectNotFoundException because there is no record with your wsdl location as the primary key

The workaround:

To work around this problem, you have to manually insert your wsdl in the SDO repository database using your wsdl location as the key. To do this,

• Create a scrip file called sdoWsdlImport.jacl and put the following content in it:

# script to add wsdl to SDO repository

set xsdFile [lindex $argv 0]

set xsdKey  [lindex $argv 1]

set sdoRep [$AdminControl queryNames *,type=SdoRepository,node=[$AdminControl  getNode]]

puts [$AdminControl invoke $sdoRep importResource [list $xsdKey $xsdFile]]

 

# comment the puts line above and uncomment the following $AdminControl line 

# to remove wsdl from SDO repository

#$AdminControl invoke $sdoRep removeResource [list $xsdKey false]

• Go to <profile_home>/bin directory and issue the following command:

wsadmin -f <path_to_jacl_file> <path_to_wsdl_file> <wsdl_url>

• For example,

wsadmin -f c:/temp/sdoWsdlImport.jacl C:/MySoftware/RAD7workspace/my_messaging_ejb/ejbModule/META-INF/wsdl/MyService.wsdl  http://localhost:9080/my_messaging_ejbHttpRouter/services/MyService/wsdl/MyService.wsdl where,

o        wsadmin -f invokes the wsadmin tool and runs the script

o        c:/temp/sdoWsdlImport.jacl is the full path to the jacl file (notice forward slash in windows environment)

o path        C:/MySoftware/RAD7workspace/my_messaging_ejb/ejbModule/META-INF/wsdl/MyService.wsdl is the physical location of wsdl file. On the server, you can use location of wsdl in 'installedApps' directory. Again, notice forward slash

o url        http://localhost:9080/my_messaging_ejbHttpRouter/services/MyService/wsdl/MyService.wsdl is the url to your wsdl. If your service is deployed and if the server is running, typing this url in the browser should show  you the wsdl

• Now, start the wsadmin tool as described above and use the same commands to list the contents of the SDO repository
• wsadmin> set sdoRep [$AdminControl queryNames *,type=SdoRepository,node=[$AdminControl getNode]] this command defines a variable called sdoRep
• wsadmin> puts [$AdminControl invoke $sdoRep listResources] this command lists the keys by which xml files are stored as blobs in your SDO repository

• This time you should see an entry with your wsdl location. This means there is a record in the SDO repository database with your wsdl location as the primary key
• Now run your test code to forward the soap message form the queue to outbound service using format string in SOAP:<wsdlLocation>,<serviceNameSpace>,<serviceName>,<portName> format in your mediation and your web service should be invoked correctly