In an SOA/BPM initiative, I see business processes being modeled primarily for 3 reasons:

1. To monitor the process at runtime and generate several reports
2. To perform business process analysis and identify services or derive IT architecture
3. To deploy the process in a process runtime and invoke services (service orchestration)

In each case, the level of details included in the model and the granularity of the process will be different. More often than not, you would want to do all the three. 

I tend to think of business process models at 3 different levels.

High level model

This is the level business actually cares about. Business doesn't necessarily want to see every possible task and every possible flow within the business process. In fact, if you show any non-technical person a BPMN model of a relatively complex process, he/she may get confused. A high level model of a loan approval process might include high level tasks such as: Create Application, Retrieve Credit Report, Review Application, Accept/Reject Application.

Business is usually only interested in getting runtime information at this level. In fact, if the business wants runtime information around every little task, the process server will end up invoking way too many implementations of those tasks resulting in a poor performance.

Analysis level business process model

You model business processes at detail level when:

• Current business processes are not documented
• You want to analyze the current processes and modify them
• You want to derive IT architecture and identify services using the business process model

When working at this level, you typically identify all the tasks and all possible flows in a process. Depending on your project goals, you define additional information for each task including who performs it, how much time does it take, how much does it cost, what resources are used, what data does the task operate on and more. You tie the business information with various elements in the process model. For example, you may trace use cases and/or rules with tasks in the business process. Your SOA architect will rely on this model heavily.

Deployment level business process model

If a business process has been modeled, it doesn't have to be deployed in a BPEL runtime. Modeling business processes typically means that we have recognized that there is a business process layer, the implementation of this layer may be a BPEL process, a workflow process or something entirely different. Also note that BPEL is still new and evolving. Not all scenarios modeled in the process modeled can be implemented readily using BPEL (see BPMN specification section XYZ for more information). This mismatch between model and runtime typically necessitates creating a runtime model. When BPEL is used, the runtime model is the WS-BPEL compliant xml that can be deployed in a process server. Your SOA architect or integration specialist will work on your business process model to create correct WS-BPEL artifacts.

It is important to note that I am not suggesting always creating 3 models (while that may be necessary in some cases). I am merely stressing the point that business process model may have to evolve and contain different level of details for different audience.

I like everything about SOA except the acronym and the way SOA is described. To make more sense, let me tell you a story I read in a piece of paper stuck on somebody's cube. The story goes something like this:

A person flying somewhere in a balloon gets lost. He lowers his balloon and asks a man on the ground. "Excuse me! Could you please tell me where I am?" The person on the ground responds, "You are in a balloon about 25 feet above the ground." Hearing the response, the balloon man correctly guesses, "You must be an IT person. What you said is correct but it's of no use to me." The ground man replies (also guessing correctly), "You must be a business person. You don't know what you want, you are not asking the right questions, yet you want me to solve your problem."

The story applies very well to SOA. While the idea behind SOA is to promote a set of architectural principles that, by design, support a better alignment between business and IT, we failed to include the B for Business in the acronym. Not only that, we do a poor job at explaining it to non-technical people. Those guys have absolutely no idea what we are talking about when we use terms like service, architecture, loose coupling, interoperability etc. They only care about satisfying the business requirements. They care about business agility. They care about return on investment. And they care about not having to write an enhancement request and spend a ton of money when all they want is to see an additional field on their screen.

You see, when we 'do' SOA, we knowingly or unknowingly look at it from technical perspective. Our respected architect tells us, "You should really expose this and that system as a set of services." If he knows BPM, he would probably add, "You should also analyze your business process and identify business services from them." The architecture team starts identifying the 'toolset' to do this. The team starts debating on the granularity of services and someone keeps repeating "SOA doesn't have to be implemented using web services." (What's up with that? Enough already!)

I say, for every potential SOA project, we go back to the basics and think about the value proposition of SOA and what it does for the business. I say, we use that value proposition to explain to the business how SOA is about deriving IT architecture from business. I say, we recognize and acknowledge (rather than defend) the IT pain points business typically has and try to show how SOA helps ease the pain.

While I don't see the acronym changing, I definitely see a potential to change our approach to SOA. Remember, there is a B in SOA; you just have to find it.

Part 1: SOA developer skills announcement

Part 2 (this part): Business process modeling 

Many organizations start their SOA initiative with business process analysis. Focusing on business first allows IT architecture to be derived from the business and, allows better alignment between IT and business. Analyzing business process provides an opportunity to optimize the business process and evaluate the impact of a business change on IT. 

Like you model software as a software designer, you model business processes as a process modeler. In addition to simply modeling business processes, you may also be called upon to simulate business processes and create reports of simulation results. You may be asked to derive IT architecture from the business model. For example, as an architect, you may decide that certain parts of the business process will continue to run in mainframe, others will be implemented using J2EE platform and yet others will be implemented by calling a third party service. You may also have to survey the existing IT systems and find out which parts of business process are already implemented within the organization. You may have to analyze the process for inefficiencies and suggest optimizations to achieve specific business goals (e.g. reduce cost).

To analyze a business processes, you have to model them. Business process modeling has been standardized using Business Process Modeling Notations (BPMN) specification available here. The BPMN notation is to business process what UML is to software. And like UML, BPMN is owned by Object Management Group (OMG) group. Like UML (but fewer), tools are available to model business processes, some use BPMN notation and some don't. Tools that use BPMN are eclipse plugin as part of SOA Tools Project (STP), Intalio Designer, and Microsoft Visio BPMN stencil.  Commercial tools such as BEA AquaLogic® BPM Designer also support BPMN. 

Tools that don't use BPMN notation but still provide modeling capabilities are IBM WebSphere Business Modeler (WBM). Oracle does not use BPMN in its BPM designer but makes a Visio stencil with Oracle support available through its partner. Netbeans does not seem to use BPMN notation in its designer. 

A UML model is typically turned into executable code (e.g. Java). Similarly, a business process model is turned into executable code called BPEL and deployed in an execution environment. The executable version of the process must follow WS-BPEL specification. In other words, like Java and UML are different specifications but Java is typically generated from UML, WS-BPEL and BPMN are different specifications and it is often expected that WS-BPEL will be generated from the business process model. As of this writing, Visio stencil (obviously) and eclipse plugin don't support export functionality but the rest of the tools do. Even the tools that don't use BPMN notation export their business process model as WS-BPEL.

Remember, business process modeling is as much an art as it is science. You need to know why you are modeling a business process, you need to ask the right questions to the subject matter experts to extract information needed to correctly model the process, you need to know which modeling notation to use to model a particular business scenario and you need to decide how granular the process model should be. 

Business process modeling tutorials are available here, here, here, here.  

Let's quickly cover some basics so the blog makes more sense.  

In a classic academic SOA scenario, your organization is made up of bunch of services. When you orchestrate them, you define their invocation sequence along with some other information in an xml file, which follows WS-BPEL specifications. You then deploy the process in a compliant process server. The server invokes the services turn by turn applying other rules and logic (e.g. stop the process if outcome of a service invocation is xyz) defined in the process.

In a top down scenario, you model your business process as a WS-BPEL compliant process, you define services that implement the tasks defined in the process, and you deploy the process in a compliant process server, which invokes the services to execute the tasks. 

This sounds simple on the surface but many architects don't realize that there is a performance overhead involved in having a process call different services - the overhead gets worse if the process is too fine grained.

 As can be seen from the figure above, the more 'boxes' you have, the more the invocations from the process server to the services and the more the overhead.

When I work with clients on SOA/BPM projects, I always encourage them to first identify the business need to deploy something as a process. The high level questions I ask are:

o    Do you want to collect runtime data on the process? If yes, what kind of data?

·         Time take by the process or a task to complete

·         Revenue generated by a particular activity

·         Number of times a particular task was performed

·         Revenue generated out of a task or frequency of revenue generated compared to the number of times a task was performed

o    Do you need business flexibility when you can switch tasks around?

o    Do you already have existing IT systems but you just need a central coordinator?

Sometimes, there are situations when there is a technical need to deploy something as a process. If an organization has correctly implemented SOA, it would have services that behave like black boxes, in which case you need a coordinator that will do all the plumbing work. It may be a good idea in that case to orchestrate services and deploy a process in a runtime. Some examples of technical need are:

o    You need to massage (map/transform) data between service invocations

o    You need to implement some logic to invoke different services based on different situations

It may be possible to use an ESB to satisfy some of the technical needs but that's another topic.

When you do decide to deploy something as a process, an interesting exercise it to determine the granularity of the process. Should you deploy every business task as a task in the process or should you combine some tasks making the process more granular? I will address this with an example some time in the future.

Part 1 (this part): SOA developer skills announcement

Part 2: Business process modeling 

Believe it or not, SOA is here and it's here to stay until the IT industry finds yet another magic bullet that will solve all our IT problems. SOA is gaining traction. Pretty much all IT vendors have products that make it easier to 'do' SOA. I see more and more job descriptions with SOA and related terms in them and I hear about some SOA conferences and expos happening everywhere.

Many people claim SOA is nothing more than a collection of sound architecture principles but even they cannot deny that SOA has influenced the way we conceive, plan and execute IT projects and create IT systems. Growing acceptance of SOA and its principles has resulted in new standards, tools, frameworks, programming models and methodologies.

In coming days, I am going to talk about specifications, tools and related skills typically considered under SOA umbrella. I will post a lot of links and try to provide some easy to understand description of those items. Please note that I don't plan to include every possible tool. On the other hand a mention of tool doesn't mean I endorse it. I will try to include free and open source tools for those who want to learn.

I fell in love with WebSphere mediations the first time I saw them. Okay, didn't fall in love but I did think... "Nice! What a cool way of going above and beyond what soap handlers offer!" For those who don't know, WebSphere mediations run in WAS runtime messaging engine, which treats any request or response flowing through as a message (could be a soap or jms message or even a request to an ejb) and any message consumer as a destination. Since we are [over] simplifying already, in simple terms WebSphere mediations are components that can intercept these messages, inspect them, route them to a different location or even change them.

Now, I wouldn't be writing this blog if everything about WebSphere mediations was so cool (yeah, I know, I have love hate relationship with IBM products!). Just note that the discussion in this blog entry refers to mediations available within WebSphere Application Server.

First of all, did you notice I call them WebSphere mediations? You won't see that term being used on any IBM website. I use that term because they use WebSphere specific APIs. You have to implement com.ibm.websphere.sib.mediation.handler.MediationHandler interface to create a mediation. This makes your code... well, not so portable. Mediations - woops, sorry - WebSphere mediations are very much like soap handlers. They can intercept the incoming requests and/or response and do things with it. There should have been some effort to make them part of the specification. I mean the whole concept of messaging engine is cool, even that should be part of the specification. The term 'messaging engine' doesn't refer to JMS messaging engine. It refers to the messaging infrastructure on which WAS runtime and service integration BUS are built. 

Second, they only support one method public boolean handle (javax.xml.rpc.handler.MessageContext context) method. Why can't they support both handleRequest and handleResponse like soap handlers do? Being able to handle response would be really convenient. For example, if I had a requirements to process an incoming message and forward it to a queue for further processing ONLY IF the initial processing was successful, I would catch the response message in the handleResponse method and then send the message for further processing. Without the ability to capture responses, you have to be little bit creative to implement this requirement (you can still do it though).

Third, they use SDO APIs. This is not a problem in itself but I don't like IBM's implementation of this API. Just try creating a soap header within mediation if no header exists already and you will know what I am talking about. In plain English, I find IBM's implementation of SDO very 'read' friendly as opposed to 'read and manipulate' friendly. It is not easy to create/add new elements.

Forth, they can't be developed. Seriously! You cannot create mediations in RAD 7.0 - not even after installing some fixpacks. Now, I know this is not a problem with the mediations themselves, it's a problem with the tooling but hey I am in the mood to rant. To develop a mediation, you create an ejb project, create a class [say] MyMediation and implement the interface, go to ejb deployment descriptor and click Add button in Mediation Handlers tab. A dialog box should pop up and the class you created should show up in it. You select the class and the necessary ejb code is generated automatically. In RAD 7, nothing happens if you click the Add button. When I used Application Server Toolkit at my last client, the dialog box popped up but my class never showed up in it. I had to create mediation project in RAD 6 (yes, it works in the previous version!) and then open it in RAD 7. Nice way to take away some functionality!

Fifth, you can't register your service in the SDO repository - something you have to do when you have an outbound service that you want to mediate using mediation. I have blogged about this in detail in this and this blog entries.

Sixth, you can customize the behavior of your mediations using mediation properties - that's a good thing. But, you have to define and change these properties using admin console. This is not flexible, and boring too if you have many properties to define. It requires a technical resource to make a change - doesn't sound like providing business agility to me.