In an SOA/BPM initiative, I see business processes being modeled primarily for 3 reasons:

1. To monitor the process at runtime and generate several reports
2. To perform business process analysis and identify services or derive IT architecture
3. To deploy the process in a process runtime and invoke services (service orchestration)

In each case, the level of details included in the model and the granularity of the process will be different. More often than not, you would want to do all the three. 

I tend to think of business process models at 3 different levels.

High level model

This is the level business actually cares about. Business doesn't necessarily want to see every possible task and every possible flow within the business process. In fact, if you show any non-technical person a BPMN model of a relatively complex process, he/she may get confused. A high level model of a loan approval process might include high level tasks such as: Create Application, Retrieve Credit Report, Review Application, Accept/Reject Application.

Business is usually only interested in getting runtime information at this level. In fact, if the business wants runtime information around every little task, the process server will end up invoking way too many implementations of those tasks resulting in a poor performance.

Analysis level business process model

You model business processes at detail level when:

• Current business processes are not documented
• You want to analyze the current processes and modify them
• You want to derive IT architecture and identify services using the business process model

When working at this level, you typically identify all the tasks and all possible flows in a process. Depending on your project goals, you define additional information for each task including who performs it, how much time does it take, how much does it cost, what resources are used, what data does the task operate on and more. You tie the business information with various elements in the process model. For example, you may trace use cases and/or rules with tasks in the business process. Your SOA architect will rely on this model heavily.

Deployment level business process model

If a business process has been modeled, it doesn't have to be deployed in a BPEL runtime. Modeling business processes typically means that we have recognized that there is a business process layer, the implementation of this layer may be a BPEL process, a workflow process or something entirely different. Also note that BPEL is still new and evolving. Not all scenarios modeled in the process modeled can be implemented readily using BPEL (see BPMN specification section XYZ for more information). This mismatch between model and runtime typically necessitates creating a runtime model. When BPEL is used, the runtime model is the WS-BPEL compliant xml that can be deployed in a process server. Your SOA architect or integration specialist will work on your business process model to create correct WS-BPEL artifacts.

It is important to note that I am not suggesting always creating 3 models (while that may be necessary in some cases). I am merely stressing the point that business process model may have to evolve and contain different level of details for different audience.

I like everything about SOA except the acronym and the way SOA is described. To make more sense, let me tell you a story I read in a piece of paper stuck on somebody's cube. The story goes something like this:

A person flying somewhere in a balloon gets lost. He lowers his balloon and asks a man on the ground. "Excuse me! Could you please tell me where I am?" The person on the ground responds, "You are in a balloon about 25 feet above the ground." Hearing the response, the balloon man correctly guesses, "You must be an IT person. What you said is correct but it's of no use to me." The ground man replies (also guessing correctly), "You must be a business person. You don't know what you want, you are not asking the right questions, yet you want me to solve your problem."

The story applies very well to SOA. While the idea behind SOA is to promote a set of architectural principles that, by design, support a better alignment between business and IT, we failed to include the B for Business in the acronym. Not only that, we do a poor job at explaining it to non-technical people. Those guys have absolutely no idea what we are talking about when we use terms like service, architecture, loose coupling, interoperability etc. They only care about satisfying the business requirements. They care about business agility. They care about return on investment. And they care about not having to write an enhancement request and spend a ton of money when all they want is to see an additional field on their screen.

You see, when we 'do' SOA, we knowingly or unknowingly look at it from technical perspective. Our respected architect tells us, "You should really expose this and that system as a set of services." If he knows BPM, he would probably add, "You should also analyze your business process and identify business services from them." The architecture team starts identifying the 'toolset' to do this. The team starts debating on the granularity of services and someone keeps repeating "SOA doesn't have to be implemented using web services." (What's up with that? Enough already!)

I say, for every potential SOA project, we go back to the basics and think about the value proposition of SOA and what it does for the business. I say, we use that value proposition to explain to the business how SOA is about deriving IT architecture from business. I say, we recognize and acknowledge (rather than defend) the IT pain points business typically has and try to show how SOA helps ease the pain.

While I don't see the acronym changing, I definitely see a potential to change our approach to SOA. Remember, there is a B in SOA; you just have to find it.

Part 1: SOA developer skills announcement

Part 2 (this part): Business process modeling 

Many organizations start their SOA initiative with business process analysis. Focusing on business first allows IT architecture to be derived from the business and, allows better alignment between IT and business. Analyzing business process provides an opportunity to optimize the business process and evaluate the impact of a business change on IT. 

Like you model software as a software designer, you model business processes as a process modeler. In addition to simply modeling business processes, you may also be called upon to simulate business processes and create reports of simulation results. You may be asked to derive IT architecture from the business model. For example, as an architect, you may decide that certain parts of the business process will continue to run in mainframe, others will be implemented using J2EE platform and yet others will be implemented by calling a third party service. You may also have to survey the existing IT systems and find out which parts of business process are already implemented within the organization. You may have to analyze the process for inefficiencies and suggest optimizations to achieve specific business goals (e.g. reduce cost).

To analyze a business processes, you have to model them. Business process modeling has been standardized using Business Process Modeling Notations (BPMN) specification available here. The BPMN notation is to business process what UML is to software. And like UML, BPMN is owned by Object Management Group (OMG) group. Like UML (but fewer), tools are available to model business processes, some use BPMN notation and some don't. Tools that use BPMN are eclipse plugin as part of SOA Tools Project (STP), Intalio Designer, and Microsoft Visio BPMN stencil.  Commercial tools such as BEA AquaLogic® BPM Designer also support BPMN. 

Tools that don't use BPMN notation but still provide modeling capabilities are IBM WebSphere Business Modeler (WBM). Oracle does not use BPMN in its BPM designer but makes a Visio stencil with Oracle support available through its partner. Netbeans does not seem to use BPMN notation in its designer. 

A UML model is typically turned into executable code (e.g. Java). Similarly, a business process model is turned into executable code called BPEL and deployed in an execution environment. The executable version of the process must follow WS-BPEL specification. In other words, like Java and UML are different specifications but Java is typically generated from UML, WS-BPEL and BPMN are different specifications and it is often expected that WS-BPEL will be generated from the business process model. As of this writing, Visio stencil (obviously) and eclipse plugin don't support export functionality but the rest of the tools do. Even the tools that don't use BPMN notation export their business process model as WS-BPEL.

Remember, business process modeling is as much an art as it is science. You need to know why you are modeling a business process, you need to ask the right questions to the subject matter experts to extract information needed to correctly model the process, you need to know which modeling notation to use to model a particular business scenario and you need to decide how granular the process model should be. 

Business process modeling tutorials are available here, here, here, here.  

Let's quickly cover some basics so the blog makes more sense.  

In a classic academic SOA scenario, your organization is made up of bunch of services. When you orchestrate them, you define their invocation sequence along with some other information in an xml file, which follows WS-BPEL specifications. You then deploy the process in a compliant process server. The server invokes the services turn by turn applying other rules and logic (e.g. stop the process if outcome of a service invocation is xyz) defined in the process.

In a top down scenario, you model your business process as a WS-BPEL compliant process, you define services that implement the tasks defined in the process, and you deploy the process in a compliant process server, which invokes the services to execute the tasks. 

This sounds simple on the surface but many architects don't realize that there is a performance overhead involved in having a process call different services - the overhead gets worse if the process is too fine grained.

 As can be seen from the figure above, the more 'boxes' you have, the more the invocations from the process server to the services and the more the overhead.

When I work with clients on SOA/BPM projects, I always encourage them to first identify the business need to deploy something as a process. The high level questions I ask are:

o    Do you want to collect runtime data on the process? If yes, what kind of data?

·         Time take by the process or a task to complete

·         Revenue generated by a particular activity

·         Number of times a particular task was performed

·         Revenue generated out of a task or frequency of revenue generated compared to the number of times a task was performed

o    Do you need business flexibility when you can switch tasks around?

o    Do you already have existing IT systems but you just need a central coordinator?

Sometimes, there are situations when there is a technical need to deploy something as a process. If an organization has correctly implemented SOA, it would have services that behave like black boxes, in which case you need a coordinator that will do all the plumbing work. It may be a good idea in that case to orchestrate services and deploy a process in a runtime. Some examples of technical need are:

o    You need to massage (map/transform) data between service invocations

o    You need to implement some logic to invoke different services based on different situations

It may be possible to use an ESB to satisfy some of the technical needs but that's another topic.

When you do decide to deploy something as a process, an interesting exercise it to determine the granularity of the process. Should you deploy every business task as a task in the process or should you combine some tasks making the process more granular? I will address this with an example some time in the future.

Part 1 (this part): SOA developer skills announcement

Part 2: Business process modeling 

Believe it or not, SOA is here and it's here to stay until the IT industry finds yet another magic bullet that will solve all our IT problems. SOA is gaining traction. Pretty much all IT vendors have products that make it easier to 'do' SOA. I see more and more job descriptions with SOA and related terms in them and I hear about some SOA conferences and expos happening everywhere.

Many people claim SOA is nothing more than a collection of sound architecture principles but even they cannot deny that SOA has influenced the way we conceive, plan and execute IT projects and create IT systems. Growing acceptance of SOA and its principles has resulted in new standards, tools, frameworks, programming models and methodologies.

In coming days, I am going to talk about specifications, tools and related skills typically considered under SOA umbrella. I will post a lot of links and try to provide some easy to understand description of those items. Please note that I don't plan to include every possible tool. On the other hand a mention of tool doesn't mean I endorse it. I will try to include free and open source tools for those who want to learn.

Note: The title is *Web services* infrastructural considerations as opposed to SOA infrastructural considerations. Designing SOA would take a lot more consideration, this article should give you a head start for services exposed as web services within your SOA 

Of course you have to consider a lot of things when exposing a piece of functionality as a web service or as any service within soa for that matter. But today, I am going to focus on some system considerations only.

SQL attack

This problem is not specific to web services, this could occur even for any service implementation. SQL attack refers to the search criteria that result in a long running query (e.g. select *.* or select * from customer). It might be too late to implement a mechanism to detect a sql attack in the service implementation or persistence layer because by the time the service is invoked, the system may have unnecessarily parsed xml, logged the request, gone through a mediation, gone through some kind of transformation or even done a protocol switch.

Very large or hierarchical soap request

SOAP requests could get large - especially if the service allows any valid xml document to be embedded within the soap request. XML parser built into the web service container may perform poorly or even crash (especially when it is heavily loaded) if it has to parse a very large soap request. XML documents containing heavily convoluted hierarchies are also not recommended for the same reasons.

Large attachment

This is situation is similar to the one described above. A soap request with a large attachment increases the size of the message payload adding significant processing overhead and consuming a lot of bandwidth. Plus, parsing the separating out the attachment could become the bottleneck for the message flow within the system.

Repeated requests attempts

Some applications require the system to keep trying for certain time period or for several number of attempts if the first invocation fails. This especially happens in soap over jms environments. Such recurring attempts could slow down the service or could even bring down the server.

Too much reuse negatively impacting the SLAs

The soa approach often recommends implementing a business process by combining existing services. This means without a proper governance a service may be reused by many different clients negatively impacting the SLAs of the existing clients.

SOA infrastructure as the watchdog

A well conceived and well designed soa architecture provides infrastructure components to monitor and control the soa environment. Such monitoring may include: 

• inspecting soap requests for size and input parameters before delegating them to service providers
• redirecting requests with heavy resource requirements to dedicated service provider
• detecting failure and applying retry logic (e.g. retry only for certain types of failures)
• routing requests to appropriate geographic location (e.g. routing eastern customer request to the East service center to take advantage of data processing within LAN)
• and more...

Products aside, when introducing soa in an organization, the architect must consider,

• who is calling services
• what is the volume, type and frequency of data exchange among the services
• what SLA the services must support and how that resolves to the infrastructure requirements
• where things could go wrong and how they must be handled
• what are the routing requirements between service providers and service consumers