...Because that's how long it took me to find and fix a bug in the [not so] new WebSphere V6.1 product.

Let me present some background before I start to rant! Starting V6.0, WAS has a concept of Service Integration BUS (or SIBUS). You can use this to forward an xml/soap message from a receiving queue to web service. You can also put mediations on the request and return paths to inspect/manipulate the message. This eliminates the need to parse an incoming xml message while giving a lot of flexibility in other areas. Actually, you can do a lot more with SIBUS but let me just rant for today.

WAS represents the message in Service Data Object (SDO) format because it's much easier to work with objects when you are inspecting or manipulating the message or its path. You have to install an SDO repository - a database where your wsdl and other service information is stored so that WAS can create SDO object structure – for this to work. When you create an SIBUS web service using the admin console, WAS 6.1 'should' put the service information - especially wsdl file in the SDO database using wsdl location as the primary key. So, MyService.wsdl will be stored using http://localhost:<port>/<context_root>/services/MyService/wsdl/MyService.wsdl as the key. This happens automatically when you create the SIBUS service through admin console. When you use the service, you specify a so called format string to identify the wsdl (the key of the SDO repository) the incoming soap message belongs to. Specifying the format string will give you complete SDO structure including service name, parameters, header, attachment and body in object form so that you can manipulate anything you want. You can even change the message route!

So far so good. My problem started when I successfully defined the service but couldn't retrieve it as SDO. Surprisingly, no error was reported but message simply didn't forward to my web service from the receiving queue. Upon close inspection of log files in ffdc log directory, I got a message similar to 'wsdl could not be located.' And upon closer inspection (and of course a lot time wasting), I found an ObjectNotFoundException on SDO repository. Obviously, I was using wrong key (format string) to retrieve my data. Now, according to WAS 6.1 info center, format string should be defined in the following format:

SOAP:<wsdl_location><other_parameters>

And that is exactly what I was using. This led me to reinstalling SDO repository, recreating server profile, redeploying ear files and web services and a lot more not to mention, lost sleep and eating/drinking unhealthy stuff while desperately trying to figure out what was going on.

Finally, I decided to try WAS 6.0 supported format of format string:

SOAP:dest:<bus_name>,<other_parameters>

Correctly, I got an incorrect format string error because there was a validation component watching the format string I was passing. As it turns out while the validation component was watching the format string correctly, IBM WAS developers were NOT watching the key they used to create a record in the SDO repository for an SIBUS web service.

When you create a web service in SIBUS, WAS 6.1 uses WRONG KEY to insert the record in the repository. It uses "SOAP:dest…" format. So there is a record in the repository for your web service but you can never retrieve it because of the validation component! If you use the WAS 6.1 recommended format, "SOAP:<wsdl_location>…" you will never get any results back because the data wasn't inserted with that key in the first place. In other words, creating a web service within SIBUS will NOT work correctly in WAS 6.1.

It took me 3 days to figure it out. Finally, I ended up writing some jacl script to fix this issue (I will share the scripts in near future). What is ironic is, "SOAP:dest:…" format is not even mentioned in WAS 6.0 OR WAS 6.1 info center. I found this format from an article on developerworks at http://www-128.ibm.com/developerworks/websphere/techjournal/0509_reinitz/0509_reinitz.html. Info center always mentioned "SOAP:<wsdl_location>," which makes sense because every web service within a bus will have unique wsdl location so location should be used to store objects in SDO repository.

The lesson learned? If you are using WAS 6.0, use "SOAP:dest…" format to specify format string. And if you are using WAS 6.1, good luck using web services in SIBUS environment! Well... just use the jacl script I will share soon to change the SDO repository to make the correct format string work.

This blog entry is a rant - a big one against spring, hibernate combo. 

If you run a database query and do not get any results back, here is what you would typically do:

• Check logs etc. to make sure there is no exception
• Check to make sure the data is available in the database
• Check to make sure the search criteria are being passed correctly
• Check to make sure the query you are issuing or generating is correct
• Check to make sure the code that receives the results is correctly interpreting that the result set is in fact empty
• If you haven't spent too much time debugging this issue yet, it will probably occur to you to make sure you are searching against the correct database schema

However, if you are using spring - hibernate combo, let me give you one more 

Check that all hibernate mapping files are added to the list of mapping files in the mappingResources property of LocalSessionFactoryBean provided by spring.

I didn't, and it cost me a long day and night figuring it out. But that's no ranting, that's lesson learned. The ranting comes from the fact that neither spring nor hibernate cared to tell me about a missing mapping file. If my DetachedCriteria is defined for MyDomain, I would expect hibernate to lookup its mapping file and tell me if it can't find the file. Instead, hibernate ran the query happily and returned zero results! No errors, no exceptions, no warning - not even a tiny debug statement. In fact, the behavior of the framework was misleading. Usually, when you set show_sql to true, hibernate prints all the sql queries it generates. In this case, it didn't print the query leading me to believe it couldn't generate the query. I first blamed it on my usage of projections, then on result set transformer, then on hibernate filters, then on the version of business objects jar file I was using, then on WebSphere, then on database, then on hibernate caching and the list goes on... For every possibility, I ended up changing code/configuration, adding more log statements, redeploying the application and testing. I even ran another transaction that used the same DAO method and made sure that query was printing in the log and executing correctly. 

Many will put the blame on me and they are right if they do. If the framework says you have to specify mapping files, you have to or face consequences, but I was just hoping to see the consequences in my log files.

The story doesn't end here. After I finally added the mapping file entry and ran again, I got an error similar to "no class MySubDomain found." This is valid because MyDomain would return results as MySubDomain based on a column selector BUT I think the error is misleading. In a different situation, the developer would think the class was not packaged in the domain objects jar file. In my case however, since I knew my earlier problem was a missing mapping file, it immediately occurred to me that the mapping file for MySubDomain was probably missing. After adding both mapping files, hibernate executed the query (like it did before) but this time it also returned results (thank you!). 

This incident has created yet another dent in my love for spring - hibernate as enterprise frameworks (yes, there are some dents already but let's talk about them later!). As of now, I don't know whether spring or hibernate caused this. I will try to find out when I am done with other more important things I have to do. In the mean time, my ranting continues...

In my previous blog entry , I discussed situations in which exposing web service should be treated as business/system requirement and captured accordingly. I also described situations where exposing a system as a web service (as opposed to alternatives) may not be strictly an architectural decision.

It might appear that writing use cases for web service is not any different from writing  use cases for a non web service system but as we will see in this and future blogs, there are some decisions to be made:

Where to capture the requirement that the system (or its parts) should be exposed as a web service?

Supplementary specifications: supplementary specifications typically capture non-functional requirements. One may argue that business functionality should be captured in a use case but the fact that it is available as a web service is a non functional requirement and hence should be captured in supplementary specifications. This is also the place where interoperability, security and SLAs related to the web service should be captured. On the flip side, if only parts of the system are available as web service, one may argue that supplementary specification should not have references to parts of the application and web service requirements should be captured in individual use cases for those parts

Use cases: This is another place where it could be indicated whether a given functionality is available as web service. This can be done by capturing web service client as a system user and including that user as the actor on the use case. This is valid because the web service client is typically another system and hence lives outside the boundary of the system.  Alternatively, it could be mentioned in the notes or use case description section that the functionality is available as web service

Software Architecture Document: Advocates of keeping requirements strictly implementation agnostic like this option. Typically SAD lists the architecturally significant use cases. This might be a good place to mention the use cases that are available as web services. Remember, at the end of the day, SAD is also driven by requirements! When this option is chosen, care must be exercised to make sure

Web service requirement is communicated to the developers as developers don't tend to keep up to date with SAD

Use case is web service compatible e.g. use case shouldn't say something like, "...system presents a warning: are you sure? User accepts it and submits..."

It is also possible to take a combination of these approaches. Web service related use cases can be identified in SAD, actor can be listed in web service use cases and non functional requirements related to web service can be documented in supplementary specifications.

No matter which approach you take, writing web service use cases has its own challenges, which I will discuss in a future blog.

Let's say you have an application and you are exposing it (or any of its parts) as a web service. Should such an activity be treated as requirement or should it be considered an architectural decision? Answering this question is important as it impacts how requirements are identified and how the application is tested. 

If exposing the application as a web service is a requirement, it should be captured somewhere, web service client must be identified as a user, interoperability requirement must be identified and documented and test cases must be written from web services perspective.

On the other hand, if exposing the application as a web services is an architectural decision, only SAD (Software Architecture Document) and design documents need to mention it. Use cases, test scripts and even the supplementary specifications could be web services agnostic. 

So, a natural question is, when to treat exposing web services as a requirement?

Treat web services as a requirement when

• There is a business driver to expose the system as a web service. For example, a system may expose a web service that calculates the distance between two zip codes and/or provides a list of zip codes given the center zip code and the radius. There may be a business reason (generate revenue) to make this web service available to retailers wanting to implement store locator functionality in their own environments
• Web services are to be provided to or consumed from the business partners. For example, an airline reservation system may have to invoke web services to reserve hotel and car to provide its users with a streamlined reservation experience. Since the web service invocation happens between separate businesses (typically governed by contracts), it should be treated as a business requirement
• Web service client is a system user. In other words, when web service is provided or consumed at the system boundary, it should be treated as a requirement. For example, a stock quote service may have a browser based user interface (client within the system boundary) BUT it may also have to be exposed as a web service (client outside the system boundary). In this case, there are two different types of users using the system: browser user and web service client. Since the system user the outside 

As mentioned above, when exposing web service is a business requirement, it impacts the way requirements are identified, documented and tested. I will talk more about this in a future blog.

 EDIT on December 15, 2006: Corrected some spelling and grammar mistakes

I recently received an email through my website about what is the future of certain technologies and how one should move forward especially if he/she has lost touch with some of the new technologies. I thought the email and my response to it might be useful to the community so I am posting them here as they are (without any changes)...

<email>

I was pretty impressed by your thoughts. I think your ideas and presentaion of problems was real good. I found your blog by accident on searching for my destination which direction move about the technologies in J2EE with EJB/Websphere/RAD/WSAD/DB2 and wondering about spring/hibernate/EJB3 how I can adopt them within our infrastructure. Pretty much confused. This maily because I guess I lost touch with new techs, whatz happening there. Recently came across SOA forum meeting and found it was mazing. At the same time felt very in-secured about my self. I am sorry for writng things with no clear picture but this how I am going through today. Hope I would find some light and direction from you and your site in future.

</email> 

<my response>

Thank you for your note. While I will continue to post on my website (unfortunately, it doesn't happen as frequently as I would like) let me respond to your email as well.

First, it's not your fault that you feel insecure or are confused about the direction of your career and the direction of technology market as a whole. Technology keeps changing fast. New frameworks, specifications and tools keep coming out all the time. If for some reason, you don't use them, you feel like being in a whole different world in a pretty short period of time.

This has happened to me. I was on an analysis/architecture project for some time. While that experience was very valuable, when I finished, there were Hibernate, JSF I had absolutely no idea about. Additionally, technology companies intentionally keep churning out new products and programming models frequently to continue to make consulting money. 

Here are some pointers that might help you move forward

1. Try to have a fairly good understanding of tools from at least one vendor. For example, I specialize in IBM offerings: WebSphere, RAD, DB2, Portal, Content management etc. I don't know JBoss and WebLogic as well but that has never been a problem. Knowing just Java/J2EE is not enough. You must know tools. Recently, we lost a developer because he couldn't even successfully import projects in WSAD/RAD. He was good in writing code but we had to spend a lot of time holding his hands, which was quite ineffective for the whole team 

2. Never forget to think of a problem in terms of architecture and design. I always ask my developers to think in terms of business problem --> architecture --> design --> code. This becomes more and more important as you transition from programmer to developer to designer to architect and so on 

3. As far as technologies and framework go, focus more on J2EE than on Spring and Hibernate. There are plenty of applications deployed on J2EE and they are likely to go to Java EE 5 and EJB 3 than to Spring/Hibernate. Why? Because they will be forced to migrate when they migrate to newer versions of the application servers they are using. For example, pretty soon, WebSphere 5.x will not be supported so companies will be forced to upgrade from J2EE 1.3. And it's easy to go from J2EE 1.3 to 1.4 or 5 but not to Spring/Hibernate. Even for new applications, I see Spring/Hibernate being used more in conjunction with J2EE than standalone. It takes a really good architect to create a pure Spring/Hibernate/POJO based enterprise application - very few of them exist and they are not available. Knowing Struts is a must though. It's an old framework; pretty much used everywhere - old and new projects alike. I don't see JSF replacing struts any time soon because of lack of available JSF skills 

4. SOA and web services - definitely the way to go in the future. As you may have seen on my website survey, that is what people think as well! Why are SOA and web services important? They solve the next level of complexity in IT. Remember when HTML was hot and HTML programmers were paid a lot of money? Now, you can create an html page in Microsoft word (creates ugly code though)! Remember when if you knew C language, you were considered smart (and geek)? IT industry wants to solve more complex problems and in the process, it invents the next level. Lower level languages and technologies, while important, don't remain lucrative forever. Java will probably face the same problem (not in the near future though) in the sense that writing Java programs will get easier and easier requiring lower level skills. But, creating SOA/web services based systems will require advanced skills and knowledge of the latest tools and specifications. If you have these skills, you will bring more value to the table.

To summarize,

1. Focus on at least one tool

2. Always know that business problem drives the IT solution 2. Stay with the specifications (J2EE) 3. Don't ignore SOA and web services

Hope this helps...

</my response>

Disclaimer: I am not an attorney, tax advisor or small/large business expert. Everything I write here is my personal opinion so take it with a grain of salt. I am not affiliated with any vendor so any product I mention is just for example purposes, I don't recommend any products or services. 

Yesterday, I wrote about how to estimate your utilization when you are planning to become an independent consultant. Today, I am going to give some pointers about estimating your expenses. Once you know your expenses, you can calculate the amount of money you have to make to justify quitting full time job. I used the following formula:

My income as an employee + my expenses as an independent consultant = money I have to make to justify quitting my job

How do you estimate your expenses? Consider the following expenses:

• Accountant's fee every year
• Government fees (for example, Illinois has yearly fee to run an LLC)
• Business supplies - I only considered supplies I would need for business. For example, cost of printing business cards counts but cost of printer cartridge doesn't because I would buy printer cartridge even if I didn't have a business
• Business insurances - there could be a separate blog entry about insurance but if you own a business, you would at least need general liability insurance, professional liability insurance (some clients require it), worker's compensation (if you are going to have employees) and its variants
• Personal insurances - health, life, short term and/or long term disability. I did pay for some of these insurances as an employee as well so I only calculated the difference
• Self employment tax - if you start an LLC, you will be paying so called self employment taxes. It is 15.3% on the first $90,000 you make every year. As an employee, you pay half of it and your employer pays the other half. As a self employed, you pay both the halves
• Any professional memberships or subscriptions you might need
• Any other cost you might incur as a result of quitting your job (e.g. forced to buy a second car, move to a more expensive location)
• Cost of lost benefits such as 401k match, paid vacation and more - this is an expense because now you will have to pay for these out of your own pocket
• One time expenses or startup expenses: I didn't pay much attention to startup expenses because they incur only once and I didn't want to factor them into my billing rate but here are some startup expenses you might want to consider:
• LLC articles of organization filing fee
• Laptop and related accessories such as internet security software, hardware locks etc.
• Bank account fee
• Attorney fee if you need advice on what type of company to form
• Software such as Quick Books to maintain your books

I am intentionally not putting any numbers against any of these expenses. You need to do a lot of research to estimate them. Don't take others' expenses and assume that you will incur the same amount.