Wednesday, December 15, 2004
« "The Fallacy of the Data Layer" article ... | Main | More information on "emergence" »

Earlier this week my employer, Magenic worked with the Microsoft office in Atlanta to set up a series of speaking engagements with some Atlanta-based companies. I had a good time, as I got some very good questions and discussion through some of the presentations.

 

In one of the presentations about SOA, an attendee challenged my assertion that “pure” SOA would lead to redundant business logic. He agreed with my example that validation logic would likely be replicated, but asserted that replication of validation logic was immaterial and that it was business logic which shouldn’t be replicated.

 

Now I’m the first to acknowledge that there are different types of business logic. And this isn’t the first time that I’ve had people argue that validation is not business logic. But that’s never rung true for me.

 

Every computer science student (and I assume CIS and MIS students) gets inundated with GIGO early in their career. Garbage-In-Garbage-Out. The mantra that illustrates just how incredibly important data validation really is.

 

You can write the most elegant and wonderful and perfect business logic in the world, and if you mess up the validation all your work will be for naught. Without validation, nothing else matters. Of course without calculation and manipulation and workflow the validation doesn’t matter either.

 

My point here is that validation, calculation, manipulation, workflow and authorization are all just different types of business logic. None are more important than others, and a flaw in any one of them ripples through a system wreaking havoc throughout.

 

Redundant or duplicated validation logic is no less dangerous or costly than duplicated calculation or manipulation logic. Nor is it any easier to manage, control or replicate validation logic than any other type of logic.

 

In fact, you could argue that it is harder to control replicated validation logic in some ways. This is because we often want to do validation in a web browser where the only real options are javascript or flash. But our “real” code is in VB or C# on the web server and/or application server. Due to the difference in technology, we can’t share any code here, but must manually replicate and maintain it.

 

Contrast this with calculation, manipulation or workflow, which is almost always done as part of “back end processing” and thus would be written in VB or C# in a DLL to start with. It is comparatively easy to share a well-designed DLL between a web server and app server. Not trivial I grant you, but it doesn’t require manual porting of code between platforms/languages like most validation logic we replicate.

 

So given the comparable importance of validation logic to the other types of business logic, it is clear to me that it must be done right. And given the relative complexity of reusing validation logic across different “tiers” or services, it is clear that we have a problem if we do decide to replicate it.

 

And yet changing traditional tiered applications into a set of cooperative services almost guarantees that validation logic will be replicated.

 

The user enters data into a web application. It validates the data of course, because the web application’s job is to give the user a good experience. Google’s new search beta is reinforcing the user expectation that even web apps can provide a rich user experience comparable to Windows. Users will demand decent interactivity, and thus validation (and some calculation) will certainly be on the web server and probably also replicated into the browser.

 

The web application calls the business workflow service. This service follows SOA principles and doesn’t trust the web app. After all, it is being used by other front-end apps as well, and can’t assume any of them do a good job. We’re not talking security here either – we’re talking data trust. The service just can’t trust the data coming from unknown front-end apps, as that would result in fragile coupling.

 

So the service must revalidate and recalculate any thing that the web app did. This is clear and obvious replication of code.

 

At this point we’ve probably got three versions of some or all of our validation logic (browser, web server, workflow service). We probably have some calculation/manipulation replication too, as the web server likely did some basic calculations rather than round-trip to the workflow service.

 

Now some code could be centralized into DLLs that are installed on both the web server and app server, so both the web app and workflow service can use the same code base. This is common sense, and works great in a homogenous environment.

 

But it doesn’t extend to the browser, where the replicated validation logic is in javascript. And it doesn’t work all that well if the web server is running on a different platform from the app server.

 

In any case, we have replication of code – all we’re doing is trying to figure out how to manage that replication in some decent way. I think this is an important conversation to have, because in my mind it is one of the two major hurdles SOA must overcome (the other being versioning).

Wednesday, December 15, 2004 8:48:17 PM (Central Standard Time, UTC-06:00)
That's unless you manage to encode the validations in metadata and generate code with validations for each different platform (javascript, c#/vb, t-sql).

Is not that difficult.
Wednesday, December 15, 2004 11:10:24 PM (Central Standard Time, UTC-06:00)
I've often thought of the database schema as being business logic. The constraints, the relationships, the datatypes too. Enterprise software is one big piece of interwoven business logic.
Thursday, December 16, 2004 9:25:14 AM (Central Standard Time, UTC-06:00)
Andres, I agree with you. And if someone invents a programming language (or metadata language - same thing) that enables expression of all the validation and calculation that people tend to put in browsers, UI code and server code that would be compelling. Especially if there was a related runtime that would generate appropriate code for each platform as you describe. Unfortunately I'm not aware of such a language/tool, and given people's attachment to writing thier own VB/C#/Java code, I am skeptical of its adoption.
Thursday, December 16, 2004 9:26:26 AM (Central Standard Time, UTC-06:00)
Scott, I think that is true - the schema is certainly a form of very basic business logic. Typically it is insufficient to protect applications from trashing each other's data though, because we rarely code enough constraints into the database to ensure semantic value in the data...
Thursday, December 16, 2004 10:24:03 AM (Central Standard Time, UTC-06:00)
I was there at the presentation you discussed and you missed the whole point of what was said. Instead, you just poo-pooed it in order to drive your points home and you didn't give him a chance to finish.

Furthermore, he was not disagreeing with you. It is agreed that it is business logic and very important. However, much validation logic can be reused and pushed to the client as meta data. This is what we do with much of the application architecture (e.g. Andres point). We use a series of custom attributes to validate properties, parameters, etc. And we are beginning the stage of using these customer attributes to push validation logic to the client.

The only point he was making was that in the grand schemes of things, it is more important to encapsulate the bulk of your business logic logic into your BL and/or webservice. And not sweat the small stuff. Will this require some duplication? Absolutely.

But most application architectures use a combination of client side and server side validation of data and chose to do most of their heavy lifting on the server side in order to accomodate a large amount of reuse. In most situations validation logic has more to do with min/max length, basic regular expressions, required items, etc. Is this revalidated on the server. Yes. Can you push this to the client via meta data. Not all of it, but a good chunk of it.
Thursday, December 16, 2004 10:46:02 AM (Central Standard Time, UTC-06:00)
Interesting... I was reading the post and I was thinking the very thing Andres mentioned that the javascript to do the validation on the brower can easily be generated from the same source that generates the C# source to do the validation on the middle tier - I am using "tier" instead of "layer", implying the trust boundary. See, I'm already applying the things I learned from reading your blogs Rocky! And I also agree that it's not rocket science but it won't be so simple either. Ok... all those words had no value, repeating the statements made before.

In any case, Mathew, "And not sweat the small stuff"? If I'm reading Rocky correctly, I think it is exactly this that he disagrees, that Validation is not "small stuff" Validation is huge in my opinion. Once the data can be trusted, the "business logic" that is performed is garden variety - not really, but you get the point. SOA, in my head anyway, the whole thing is about integration and for a successful integration, validation of data that gets passed around is the most important aspect.

But I would be very interested to know how you guys are doing the custom attribute thing.
Jiho Han
Thursday, December 16, 2004 11:06:57 AM (Central Standard Time, UTC-06:00)
It is his point. My point is that for the most part it is easy to reproduce and encapsulate. So you may still have to create code for a specific platform, but you can encapsulate most of it. You can create meta to drive the validation on the client, but the client does have to agree to accept it and transform it into something it can use. This is much easier to do if you have control on the client and server and/or provide different controls and/or classes to grab the meta and transform it into something useful on the client.

It's just crazy to jump on this bandwagon that the sky is falling...which is what he is saying...but then again he is a "Contrarian" (sorry Rockford I had to take the jab). Furthermore, this is NOT new. Client and Server side validation have and always will be a problem. It is NOT new to the webservice world.

-Mathew C. Nolton

Per the validation framework....Here is an older version of it.

http://www.cybral.com/solutions/tools.htm#validationframework
It has evolved a bit sense this version but I haven't had a chance to update it. Nor does this release have any meta generation.

Thursday, December 16, 2004 2:04:49 PM (Central Standard Time, UTC-06:00)
Mathew,

I didn't intentionally shut him down. Any time you are presenting to a group within a set of time-bounds you have to limit dialog. It sucks, but it is reality.

I agree with you. This is not a new problem. Nor do I think SOA offers a solution to it. That was the core point I was trying to make, and am trying to address here. SOA does not solve the problem of duplicate code. If anything it makes it worse.

And there is little to no emphasis being placed on this issue. There are no evolving standards for formatting and delivering the metadata for business logic. Many people are working on ways to describe, generate and validate the data and related schemas, but there's no public work going on around the area of making portable business logic.

I am a strong advocate of "mobile objects" or distributed agents. That's the core theme of my books for instance. As such, I am always trying to figure out how to move not only data, but also the logic associated with that data from place to place. SOA doesn't address this need, and in fact is going exactly the opposite direction.

SOA is all about passive data being passed from actor to actor. Distributed OO (agents) are all about active entities (objects) that can move to where they are needed. In this model the actors actually move.

Of course distributed OO doesn't actually address the duplicate code issue in all cases either, but it does help within the context of a single technology sphere (.NET or Java for instance). But the browser remains problematic. Eventually the limitations of the browser will be its undoing (or will force it to grow into something that's actually useful in this context).

But with its passive data mindset, SOA is going exactly in the opposite direction of distributed OO. And that's fine I guess, but it is a serious limitation - and Achilles Heel if you will - that I personally think should be addressed.
Thursday, December 16, 2004 5:59:09 PM (Central Standard Time, UTC-06:00)

Hi Rockford,

Validation do is "Business Logic" or "Business Rules". For consistency purposes throughout the corporate applications, business rules must not be replicated. BRMS (Business Rules Management Solution) enables to
externalize the business rules from the code (.net, java, ...), write them in plain natural language and reuse the business rules you need in the service of your choice. For instance Validation business rules may be used in differents applications accross the enterprise and using a BRMS helps you to maintain and share those rules in a central environment.


Marc
Marc Mouries
Thursday, December 16, 2004 8:59:01 PM (Central Standard Time, UTC-06:00)
Marc,

I agree. business rules should not be replicated. And a BRMS is a partial solution. As soon as one exists that can check the data live in the browser, live in a Windows GUI, as it comes into a web service as XML, while I'm working with the data in the web server, while I'm working with the data in the app server and before I store the data in the database then I'll be totally psyched!!

Today (as far as I know) a BRMS requires me to ferry the data off to some server so it can be validated and so forth. The reality is that this leads to a terrible user experience. Very 1980-style mainframe stuff where the user enters anything into text controls, hits the transmit key or button and waits to see if they did anything wrong.

Until a BRMS can extend the rules processing all the way out into the presentation layer we will (more often than not) have at least some duplication of code.

If you think about it, this was the promise of Java - not just write once run anywhere, but write once and reuse in every tier. But that never came true...

Maybe if Microsoft comes out with a version of the Biztalk rules processing engine that can be hosted in IE, ASP.NET and regular .NET code in an app server we can start to get closer to avoiding duplication of code.
Friday, December 17, 2004 5:17:22 PM (Central Standard Time, UTC-06:00)
Very interesting discussion...

Here are my two cents.
In one of the posts Rockford is saying that SOA doesn't address the need of moving data and also the logic associated with that data.
I think this is one of the main ideas of SOA and I would rather state it positively: SOA wants you to separate the logic and data.

In fact, the logic and implementation could be very different depending on whether you use the object on the client or server. In these cases it's even undesirable to pass "mobile objects" between tier. A hypothetically example: Order object might have a method CreateSubOrder(). On the client all it might do is to create an instance of a SubOrder object in memory. On the server this function might need to create a SubOrder and store it in the database. In SOA (since I separated logic and data) I can easily deal with this issue. How would you deal with these scenarios in "mobile objects"?

Thanks,

Dima Maltsev
Saturday, January 08, 2005 11:15:18 PM (Central Standard Time, UTC-06:00)
Here are some thoughts I had on this. WOuld be glad to read what you thought on them:
http://weblogs.asp.net/rosherove/archive/2005/01/09/349294.aspx
Comments are closed.