Rockford Lhotka - Distributed OO

An (accurate) rant on OO

Mon, 13 Mar 2006 17:44:39 GMT

Somebody woke up on the wrong side of the bed, fortunately it wasn't me this time :)

This is a nice little rant about the limitations of OO-as-a-religion. I think you could substitute any of the following for OO and the core of the rant would remain quite valid:

procedural programming/design
modular programming/design
SOA/SO/service-orientation
Java
.NET

Every concept computer science has developed over the past many decades came into being to solve a problem. And for the most part each concept did address some or all of that problem. Procedures provided reuse. Modules provided encapsulation. Client/server provided scalability. And so on...

The thing is, that very few of these new concepts actually obsolete any previous concepts. For instance, OO doesn't elminate the value of procedural reuse. In fact, using the two in concert is typically the best of both worlds.

Similarly, SO is a case where a couple ideas ran into each other while riding bicycle. "You got messaging in my client/server!" "No! You got client/server in my messaging!" "Hey! This tastes pretty good!" SO is good stuff, but doesn't replace client/server, messaging, OO, procedural design or any of the previous concepts. It merely provides an interesting lense through which we can view these pre-existing concepts, and perhaps some new ways to apply them.

Anyway, I enjoyed the rant, even though I remain a staunch believer in the power of OO.

SPOIL - interesting ideas

Wed, 01 Mar 2006 16:00:01 GMT

This recent MSDN article talks about SPOIL: Stored Procedure Object Interface Layer.

This is an interesting, and generally good idea as I see it. Unfortunately this team, like most of Microsoft, apparently just doesn't understand the concept of data hiding in OO. SPOIL allows you to use your object's properties as data elements for a stored procedure call, which is great as long as you only have public read/write properties. But data hiding requires that you will have some private fields that simply aren't exposed as public read/write properties. If SPOIL supported using fields as data elements for a stored procedure call it would be totally awesome!

The same is true for LINQ. It works against public read/write properties, which means it is totally useless if you want to use it to load "real" objects that employ basic concepts like encapsulation and data hiding. Oh sure, you can use LINQ (well, dlinq really) to load a DTO (data transfer object - an object with only public read/write properties and no business logic) and then copy the data from the DTO into your real object. Or you could try to use the DTO as the "data container" inside your real object rather than using private fields. But frankly those options introduce complexity that should be simply unnecessary...

While it is true that loading private fields requires reflection - Microsoft could solve this. They do own the CLR after all... It is surely within their power to provide a truly good solution to the problem, that supports data mapping and also allows for key OO concepts like encapsulation and data hiding.

“Distributed Objects” are bad?

Fri, 05 Aug 2005 15:12:29 GMT

Ted Neward believes that “distributed objects” are, and always have been, a bad idea, and John Cavnar-Johnson tends to agree with him.

I also agree. "Distributed objects" are bad.

Shocked? You shouldn’t be. The term “distributed objects” is most commonly used to refer to one particular type of n-tier implementation: the thin client model.

I discussed this model in a previous post, and you’ll note that I didn’t paint it in an overly favorable light. That’s because the model is a very poor one.

The idea of building a true object-oriented model on a server, where the objects never leave that server is absurd. The Presentation layer still needs all the data so it can be shown to the user and so the user can interact with it in some manner. This means that the “objects” in the middle must convert themselves into raw data for use by the Presentation layer.

And of course the Presentation layer needs to do something with the data. The ideal is that the Presentation layer has no logic at all, that it is just a pass-through between the user and the business objects. But the reality is that the Presentation layer ends up with some logic as well – if only to give the user a half-way decent experience. So the Presentation layer often needs to convert the raw data into some useful data structures or objects.

The end result with “distributed objects” is that there’s typically duplicated business logic (at least validation) between the Presentation and Business layers. The Presentation layer is also unnecessarily complicated by the need to put the data into some useful structure.

And the Business layer is complicated as well. Think about it. Your typical OO model includes a set of objects designed using OOD sitting on top of an ORM (object-relational mapping) layer. I typically call this the Data Access layer. That Data Access layer then interacts with the real Data layer.

But in a “distributed object” model, there’s the need to convert the objects’ data back into raw data – often quasi-relational or hierarchical – so it can be transferred efficiently to the Presentation layer. This is really a whole new logical layer very akin to the ORM layer, except that it maps between the Presentation layer’s data structures and the objects rather than between the Data layer’s structures and the objects.

What a mess!

Ted is absolutely right when he suggests that “distributed objects” should be discarded. If you are really stuck on having your business logic “centralized” on a server then service-orientation is a better approach. Using formalized message-based communication between the client application and your service-oriented (hence procedural, not object-oriented) server application is a better answer.

Note that the terminology changed radically! Now you are no longer building one application, but rather you are building at least two applications that happen to interact via messages. Your server doesn't pretend to be object-oriented, but rather is service-oriented - which is a code phrase for procedural programming. This is a totally different mindset from “distributed objects”, but it is far better.

Of course another model is to use mobile objects or mobile agents. This is the model promoted in my Business Objects books and enabled by CSLA .NET. In the mobile object model your Business layer exists on both the client machine (or web server) and application server. The objects physically move between the two machines – running on the client when user interaction is required and running on the application server to interact with the database.

The mobile object model allows you to continue to build a single application (rather than 2+ applications with SO), but overcomes the nasty limitations of the “distributed object” model.

Speaking of mobile objects (agents)

Tue, 26 Jul 2005 05:16:49 GMT

Here's an article that appears to provide a really good overview of the whole mobile agent/object concept. I've only skimmed through it, but the author appears to have a good grasp on the concepts and portrays them with some good diagrams.

Extending the 3-tier conversation to the web

Sun, 24 Jul 2005 14:47:26 GMT

Mike has requested my thoughts on 3-tier and the web – a topic I avoided in my previous couple entries (1 and 2) because I don’t find it as interesting as a smart/intelligent client model. But he’s right, the web is widely used and a lot of poor people are stuck building business software in that environment, so here’s the extension of the previous couple entries into the web environment.

In my view the web server is an application server, pure and simple. Also, in the web world it is impractical to run the Business layer on the client because the client is a very limited environment. This is largely why the web is far less interesting.

To discuss things in the web world I break the “Presentation” layer into two parts – Presentation and UI. This new Presentation layer is purely responsible for display and input to/from the user – it is the stuff that runs on the browser terminal. The UI layer is responsible for all actual user interaction – navigation, etc. It is the stuff that runs on the web server: your aspx pages in .NET.

In web applications most people consciously (or unconsciously) duplicate most validation code into the Presentation layer so they can get it to run in the browser. Thus is expensive to create/maintain, but is an unfortunate evil required to have a half-way decent user experience in the web environment. You must still have that logic in your actual Business layer of course, because you can never trust the browser - it is too easily bypassed (Greasemonkey anyone?). This is just the way it is on the web, and will be until we get browsers that can run complete code solutions in .NET and/or Java (that's sarcasm btw).

On the server side, the web server IS an application server. It fills the exact same role of the mainframe or minicomputer over the past 30 years of computing. For "interactive" applications, it is preferable to run the UI layer, Business layer and Data Access layer all on the web server. This is the simplest (and thus cheapest) model, and provides the best performance[1]. It can also provide very good scalability because it is relatively trivial to create a web farm to scale out to many servers. By creating a web farm you also get very good fault tolerance at a low price-point. Using ISA as a reverse proxy above the web farm you can get good security.

In many organizations the reverse proxy idea isn’t acceptable (not being a security expert I can’t say why…) and so they have a policy saying that the web server is never allowed to interact directly with the database server – thus forcing the existence of an application server that at a minimum runs the Data Access layer. Typically this application server is behind a second firewall. While this security approach hurts performance (often by as much as 50%), it is relatively easily achieved with CSLA .NET or similar architecture/frameworks.

In other situations people prefer to put the Business layer and Data Access layer on the application server behind the second firewall. This means that the web server only runs the UI layer. Any business processing, validation, etc. must be deferred across the network to the application server. This has a much higher impact on performance (in a bad way).

However, this latter approach can have a positive scalability impact in certain applications. Specifically applications where there’s not much interactive content, but instead there’s a lot of read-only content. Most read-only content (by definition) has no business logic and can often be served directly from the UI layer. In such applications the IO load for the read-only content can be quite enough to keep the web server very busy. By offloading all business processing to an application server overall scalability may be improved.

Of course this only really works if the interactive (OLTP) portions of the application are quite limited in comparison to the read-only portions.

Also note that this latter approach suffers from the same drawbacks as the thin client model discussed in my previous post. The most notable problem is that you must come up with a way to do non-chatty communication between the UI layer and the Business layer, without compromising either layer. This is historically very difficult to pull off. What usually happens is that the “business objects” in the Business layer require code to externalize their state (data) into a tabular format such as a DataSet so the UI layer can easily use the data. Of course externalizing object state breaks encapsulation unless it is done with great care, so this is an area requiring extra attention. The typical end result are not objects in a real OO sense, but rather are “objects” comprised of a set of atomic, stateless methods. At this point you don’t have objects at all – you have an API.

In the case of CSLA .NET, I apply the mobile object model to this environment. I personally believe it makes things better since it gives you the flexibility to run some of your business logic on the web application server and some on the pure application server as appropriate. Since the Business layer is installed on both the web and application servers, your objects can run in either place as needed.

In short, to make a good web app it is almost required that you must compromise the integrity of your layers and duplication some business logic into the Presentation layer. It sucks, but its life in the wild world of the web. If you can put your UI, Business and Data Access layers on the web application server that’s best. If you can’t (typically due to security) then move only the Data Access layer and keep both UI and Business layers on the web application server. Finally, if you must put the Business layer on a separate application server I prefer to use a mobile object model for flexibility, but recognize that a pure API model on the application server will scale higher and is often required for applications with truly large numbers of concurrent users (like 2000+).

[1] As someone in a previous post indirectly noted, there’s a relationship between performance and scalability. Performance is the response time of the system for a user. Scalability is what happens to performance as the number of users and/or transactions is increased.

A variety of physical n-tier options

Sat, 23 Jul 2005 17:12:25 GMT

In my last post I talked about logical layers as compared to physical tiers. It may be the case that that post (and this one) are too obvious or basic. But I gotta say that I consistently am asked about these topics at conferences, user groups and via email. The reality is that none of this is all that obvious or clear to the vast majority of people in our industry. Even for those that truly grok the ideas, there’s far from universal agreement on how an application should be layered or how those layers should be deployed onto tiers.

In one comment on my previous post Magnus points out that my portrayal of the application server merely as a place for the Data Access layer flies in the face of Magnus’ understanding of n-tier models. Rather, Magnus (and many other people) is used to putting both the Business and Data Access layers on the application server, with only the Presentation layer on the client workstation (or presumably the web server in the case of a web application, though the comment didn’t make that clear).

The reality is that there are three primary models to consider in the smart/rich/intelligent client space. There are loose analogies in the web world as well, but personally I don’t find that nearly as interesting, so I’m going to focus on the intelligent client scenarios here.

Also one quick reminder – tiers should be avoided. This whole post assumes you’ve justified using a 3-tier model to obtain scalability or security as per my previous post. If you don’t need 3-tiers, don’t use them – and then you can safely ignore this whole entry :)

There’s the thick client model, where the Presentation and Business layers are on the client, and the Data Access is on the application server. Then there’s the thin client model where only the Presentation layer is on the client, with the Business and Data Access layers on the application server. Finally there’s the mobile object model, where the Presentation and Business layers are on the client, and the Business and Data Access layers are on the application server. (Yes, the Business layer is in two places) This last model is the one I discuss in my Expert VB.NET and C# Business Objects books and which is supported by my CSLA .NET framework.

The benefit to the thick client model is that we are able to provide the user with a highly interactive and responsive user experience, and we are able to fully exploit the resources of the client workstation (specifically memory and CPU). At the same time, having the Data Access layer on the application server gives us database connection pooling. This is a very high scaling solution, with a comparatively low cost, because we are able to exploit the strengths of the client, application and database servers very effectively. Moreover, the user experience is very good and development costs are relatively low (because we can use the highly productive Windows Forms technology).

The drawback to the thick client model is a loss of flexibility – specifically when it comes to process-oriented tasks. Most applications have large segments of OLTP (online transaction processing) functionality where a highly responsive and interactive user experience is of great value. However, most applications also have some important segments of process-oriented tasks that don’t require any user interaction. In most cases these tasks are best performed on the application server, or perhaps even directly in the database server itself. This is because process-oriented tasks tend to be very data intensive and non-interactive, so the closer we can do them to the database the better. In a thick client model there’s no natural home for process-oriented code near the database – the Business layer is way out on the client after all…

Another perceived drawback to the thick client is deployment. I dismiss this however, given .NET’s no-touch deployment options today, and ClickOnce coming in 2005. Additionally, any intelligent client application requires deployment of our code – the Presentation layer at least. Once you solve deployment of one layer you can deploy other layers as easily, so this whole deployment thing is a non-issue in my mind.

In short, the thick client model is really nice for interactive applications, but quite poor for process-oriented applications.

The benefit to the thin client model is that we have greater control over the environment into which the Business and Data Access layers are deployed. We can deploy them onto large servers, multiple servers, across disparate geographic locations, etc. Another benefit to this model is that it has a natural home for process-oriented code, since the Business layer is already on the application server and thus is close to the database.

Unfortunately history has shown that the thin client model is severely disadvantaged compared to the other two models. The first disadvantage is scalability in relationship to cost. With either of the other two models as you add more users you intrinsically add more memory and CPU to your overall system, because you are leveraging the power of the client workstation. With a thin client model all the processing is on the servers, and so client workstations add virtually no value at all – their memory and CPU is wasted. Any scalability comes from adding larger or more numerous server hardware rather than by adding cheaper (and already present) client workstations.

The other key drawback to the thin client model is the user experience. Unless you are willing to make “chatty” calls from the thin Presentation layer to the Business layer across the network on a continual basis (which is obviously absurd), the user experience will not be interactive or responsive. By definition the Business layer is on a remote server, so the user’s input can’t be validated or processed without first sending it across the network. The end result is roughly equivalent to the mainframe user experiences users had with 3270 terminals, or the experience they get on the web in many cases. Really not what we should expect from an “intelligent” client…

Of course deployment remains a potential concern in this model, because the Presentation layer must still be deployed to the client. Again, I dismiss this as a main issue any longer due to no-touch deployment and ClickOnce.

In summary, the thin client model is really nice for process-oriented (non-interactive) applications, but is quite inferior for interactive applications.

This brings us to the mobile object model. You’ll note that neither the thick client nor thin client model is optimal, because almost all applications have some interactive and some non-interactive (process-oriented) functionality. Neither of the two “purist” models really addresses both requirements effectively. This is why I am such a fan of the mobile object (or mobile agent, or distributed OO) model, as it provides a compromise solution. I find this idea so compelling that it is the basis for my books.

The mobile object model literally has us deploy the Business layer to both the client and application server. Given no-touch deployment and/or ClickOnce this is quite practical to achieve in.NET (and in Java interestingly enough). Coupled with .NET’s ability to pass objects across the network by value (another ability shared with Java), all the heavy lifting to make this concept work is actually handled by .NET itself, leaving us to merely enjoy the benefits.

The end result is that the client has the Presentation and Business layers, meaning we get all the benefits of the thick client model. The user experience is responsive and highly interactive. Also we are able to exploit the power of the client workstation, offering optimal scalability at a low cost point.

But where this gets really exciting is the flexibility offered. Since the Business layer also runs on the application server, we have all the benefits of the thin client model. Any process-oriented tasks can be performed by objects running on the application server, meaning that all the power of the thin client model is at our disposal as well.

The drawback to the mobile object approach is complexity. Unless you have a framework to handle the details of moving an object to the client and application server as needed this model can be hard to implement. However, given a framework that supports the concept the mobile object approach is no more complex than either the thick or thin client models.

In summary, the mobile object model is great for both interactive and non-interactive applications. I consider it a “best of both worlds” model and CSLA .NET is specifically designed to make this model comparatively easy to implement in a business application.

At the risk of being a bit esoteric, consider the broader possibilities of a mobile object environment. Really a client application or an application server (Enterprise Services or IIS) are merely hosts for our objects. Hosts provide resources that our objects can use. The client “host” provides access to the user resource, while a typical application server “host” provides access to the database resource. In some applications you can easily envision other hosts such as a batch processing server that provides access to a high powered CPU resource or a large memory resource.

Given a true mobile object environment, objects would be free to move to a host that offers the resources an object requires at any point in time. This is very akin to grid computing. In the mobile object world objects maintain both data and behavior and merely move to physical locations in order to access resources. Raw data never moves across the network (except between the Data Access and Data Storage layers), because data without context (behavior) is meaningless.

Of course some very large systems have been built following both the thick client and thin client models. It would be foolish to say that either is fatally flawed. But it is my opinion that neither is optimal, and that a mobile object approach is the way to go.

Should all apps be n-tier?

Rockford Lhotka — Thu, 21 Jul 2005 21:51:17 GMT

I am often asked whether n-tier (where n>=3) is always the best way to go when building software.

Of course the answer is no. In fact, it is more likely that n-tier is not the way to go!

By the way, this isn’t the first time I’ve discussed this topic – you’ll find previous blog entries on this blog and an article at www.devx.com where I’ve covered much of the same material. Of course I also cover it rather a lot in my Expert VB.NET and C# Business Objects books.

Before proceeding further however, I need to get some terminology out of the way. There’s a huge difference between logical tiers and physical tiers. Personally I typically refer to logical tiers as layers and physical tiers as tiers to avoid confusion.

Logical layers are merely a way of organizing your code. Typical layers include Presentation, Business and Data – the same as the traditional 3-tier model. But when we’re talking about layers, we’re only talking about logical organization of code. In no way is it implied that these layers might run on different computers or in different processes on a single computer or even in a single process on a single computer. All we are doing is discussing a way of organizing a code into a set of layers defined by specific function.

Physical tiers however, are only about where the code runs. Specifically, tiers are places where layers are deployed and where layers run. In other words, tiers are the physical deployment of layers.

Why do we layer software? Primarily to gain the benefits of logical organization and grouping of like functionality. Translated to tangible outcomes, logical layers offer reuse, easier maintenance and shorter development cycles. In the final analysis, proper layering of software reduces the cost to develop and maintain an application. Layering is almost always a wonderful thing!

Why do we deploy layers onto multiple tiers? Primarily to obtain a balance between performance, scalability, fault tolerance and security. While there are various other reasons for tiers, these four are the most common. The funny thing is that it is almost impossible to get optimum levels of all four attributes – which is why it is always a trade-off between them.

Tiers imply process and/or network boundaries. A 1-tier model has all the layers running in a single memory space (process) on a single machine. A 2-tier model has some layers running in one memory space and other layers in a different memory space. At the very least these memory spaces exist in different processes on the same computer, but more often they are on different computers. Likewise, a 3-tier model has two boundaries. In general terms, an n-tier model has n-1 boundaries.

Crossing a boundary is expensive. It is on the order of 1000 times slower to make a call across a process boundary on the same machine than to make the same call within the same process. If the call is made across a network it is even slower. It is very obvious then, that the more boundaries you have the slower your application will run, because each boundary has a geometric impact on performance.

Worse, boundaries add raw complexity to software design, network infrastructure, manageability and overall maintainability of a system. In short, the more tiers in an application, the more complexity there is to deal with – which directly increases the cost to build and maintain the application.

This is why, in general terms tiers should be minimized. Tiers are not a good thing, they are a necessary evil required to obtain certain levels of scalability, fault tolerance or security.

As a good architect you should be dragged kicking and screaming into adding tiers to your system. But there really are good arguments and reasons for adding tiers, and it is important to accommodate them as appropriate.

The reality is that almost all systems today are at least 2-tier. Unless you are using an Access or dBase style database your Data layer is running on its own tier – typically inside of SQL Server, Oracle or DB2. So for the remainder of my discussion I’ll primarily focus on whether you should use a 2-tier or 3-tier model.

If you look at the CSLA .NET architecture from my Expert VB.NET and C# Business Objects books, you’ll immediately note that it has a construct called the DataPortal which is used to abstract the Data Access layer from the Presentation and Business layers. One key feature of the DataPortal is that it allows the Data Access layer to run in-process with the business layer, or in a separate process (or machine) all based on a configuration switch. It was specifically designed to allow an application to switch between a 2-tier or 3-tier model as a configuration option – with no changes required to the actual application code.

But even so, the question remains whether to configure an application for 2 or 3 tiers.

Ultimately this question can only be answered by doing a cost-benefit analysis for your particular environment. You need to weigh the additional complexity and cost of a 3-tier deployment against the benefits it might bring in terms of scalability, fault tolerance or security.

Scalability flows primarily from the ability to get database connection pooling. In CSLA .NET the Data Access layer is entirely responsible for all interaction with the database. This means it opens and closes all database connections. If the Data Access layer for all users is running on a single machine, then all database connections for all users can be pooled. (this does assume of course, that all users employ the same database connection string include the same database user id – that’s a prerequisite for connection pooling in the first place)

The scalability proposition is quite different for web and Windows presentation layers.

In a web presentation the Presentation and Business layers are already running on a shared server (or server farm). So if the Data Access layer also runs on the same machine database connection pooling is automatic. In other words, the web server is an implicit application server, so there’s really no need to have a separate application server just to get scalability in a web setting.

In a Windows presentation the Presentation and Business layers (at least with CSLA .NET) run on the client workstation, taking full advantage of the memory and CPU power available on those machines. If the Data Access layer is also deployed to the client workstations then there’s no real database connection pooling, since each workstation connects to the database directly. By employing an application server to run the Data Access layer all workstations offload that behavior to a central machine where database connection pooling is possible.

The big question with Windows applications is at what point to use an application server to gain scalability. Obviously there’s no objective answer, since it depends on the IO load of the application, pre-existing load on the database server and so forth. In other words it is very dependant on your particular environment and application. This is why the DataPortal concept is so powerful, because it allows you to deploy your application using a 2-tier model at first, and then switch to a 3-tier model later if needed.

There’s also the possibility that your Windows application will be deployed to a Terminal Services or Citrix server rather than to actual workstations. Obviously this approach totally eliminates the massive scalability benefits of utilizing the memory and CPU of each user’s workstation, but does have the upside of reducing deployment cost and complexity. I am not an expert on either server environment, but it is my understanding that each user session has its own database connection pool on the server, thus acting the same as if each user has their own separate workstation. If this is actually the case, then an application server would have benefit by providing database connection pooling. However, if I’m wrong and all user sessions share database connections across the entire Terminal Services or Citrix server then having an application server would offer no more scalability benefit here than it does in a web application (which is to say virtually none).

Fault tolerance is a bit more complex than scalability. Achieving real fault tolerance requires examination of all failure points that exist between the user and the database – and of course the database itself. And if you want to be complete, you just also consider the user to be a failure point, especially when dealing with workflow, process-oriented or service-oriented systems.

In most cases adding an application server to either a web or Windows environment doesn’t improve fault tolerance. Rather it merely makes it more expensive because you have to make the application server fault tolerant along with the database server, the intervening network infrastructure and any client hardware. In other words, fault tolerance is often less expensive in a 2-tier model than in a 3-tier model.

Security is also a complex topic. For many organizations however, security often comes down to protecting access to the database. From a software perspective this means restricting the code that interacts with the database and providing strict controls over the database connection strings or other database authentication mechanisms.

Security is a case where 3-tier can be beneficial. By putting the Data Access layer onto its own application server tier we isolate all code that interacts with the database onto a central machine (or server farm). More importantly, only that application server needs to have the database connection string or the authentication token needed to access the database server. No web server or Windows workstation needs the keys to the database, which can help improve the overall security of your application.

Of course we must always remember that switching from 2-tier to 3-tier decreases performance and increases complexity (cost). So any benefits from scalability or security must be sufficient to outweigh these costs. It all comes down to a cost-benefit analysis.

Is Indigo compelling or boring?

Tue, 19 Apr 2005 04:13:06 GMT

Indigo is Microsoft’s code name for the technology that will bring together the functionality in today’s .NET Remoting, Enterprise Services, Web services (including WSE) and MSMQ. Of course knowing what it is doesn’t necessarily tell us whether it is cool, compelling and exciting … or rather boring.

Ultimately beauty is in the eye of the beholder. Certainly the Indigo team feels a great deal of pride in their work and they paint this as a very big and compelling technology.

Many technology experts I’ve talked to outside of Microsoft are less convinced that it is worth getting all excited.

Personally, I must confess that I find Indigo to be a bit frustrating. While it should provide some absolutely critical benefits, in my view it is merely laying the groundwork for the potential of something actually exciting to follow a few years later.

Why do I say this?

Well, consider what Indigo is again. It is a technology that brings together a set of existing technologies. It provides a unified API model on top of a bunch of concepts and tools we already have. To put it another way, it lets us do what we can already do, but in a slightly more standardized manner.

If you are a WSE user, Indigo will save you tons of code. But that’s because WSE is experimental stuff and isn’t refined to the degree Remoting, Enterprise Services or Web services are. If you are using any of those technologies, Indigo won’t save you much (if any) code – it will just subtly alter the way you do the things you already do.

Looking at it this way, it doesn’t sound all that compelling really does it?

But consider this. Today’s technologies are a mess. We have at least five different technologies for distributed communication (Remoting, ES, Web services, MSMQ and WSE). Each technology shines in different ways, so each is appropriate in different scenarios. This means that to be a competent .NET architect/designer you must know all five reasonably well. You need to know the strengths and weaknesses of each, and you must know how easy or hard they are to use and to potentially extend.

Worse, you can’t expect to easily switch between them. Several of these options are mutually exclusive.

But the final straw (in my mind) is this: the technology you pick locks you into a single architectural world-view. If you pick Web services or WSE you are accepting the SOA world view. Sure you can hack around that to do n-tier or client/server, but it is ugly and dangerous. Similarly, if you pick Enterprise Services you get a nice set of client/server functionality, but you lose a lot of flexibility. And so forth.

Since the architectural decisions are so directly and irrevocably tied to the technology, we can’t actually discuss architecture. We are limited to discussing our systems in terms of the technology itself, rather than the architectural concepts and goals we’re trying to achieve. And that is very sad.

By merging these technologies into a single API, Indigo may allow us to elevate the level of dialog. Rather than having inane debates between Web services and Remoting, we can have intelligent discussions about the pros and cons of n-tier vs SOA. We can apply rational thought as to how each distributed architecture concept applies to the various parts of our application.

We might even find that some parts of our application are n-tier, while others require SOA concepts. Due to the unified API, Indigo should allow us to actually do both where appropriate. Without irrational debates over protocol, since Indigo natively supports concepts for both n-tier and SOA.

Now this is compelling!

As compelling as it is to think that we can start having more intelligent and productive architectural discussions, that isn’t the whole of it. I am hopeful that Indigo represents the groundwork for greater things.

There are a lot of very hard problems to solve in distributed computing. Unfortunately our underlying communications protocols never seem to stay in place long enough for anyone to really address the more interesting problems. Instead, for many years now we’ve just watched as vendors reinvent the concept of remote procedure calls over and over again: RPC, IIOP, DCOM, RMI, Remoting, Web services, Indigo.

That is frustrating. It is frustrating because we never really move beyond RPC. While there’s no doubt that Indigo is much easier to use and more clear than any previous RPC scheme, it is also quite true that Indigo merely lets us do what we could already do.

What I’m hoping (perhaps foolishly) is that Indigo will be the end. That we’ll finally have an RPC technology that is stable and flexible enough that it won’t need to be replaced so rapidly. And being stable and flexible, it will allow the pursuit of solutions to the harder problems.

What are those problems? They are many, and they include semantic meaning of messages and data. They include distributed synchronization primitives and concepts. They include standardization and simplification of background processing – making it as easy and natural as synchronous processing is today. They include identity and security issues, management of long-running processes, simplification of compensating transactions and many other issues.

Maybe Indigo represents the platform on which solutions to these and other problems can finally be built. Perhaps in another 5 years we can look back and say that Indigo was the turning point that finally allowed us to really make distributed computing a first-class concept.

Teaser

Fri, 01 Apr 2005 05:10:08 GMT

I just spent three days getting inundated with the details of Indigo as it stands today (in its pre-beta state). For those that don’t know, Indigo is the combined next generation of .NET Remoting, Enterprise Services, Web services (asmx and WSE) and MSMQ all rolled into one big ball.

I also had some conversations about Avalon, though not nearly in so much detail. For those that don’t know, Avalon is the next generation display technology that uses full 3D rendering and should allow us to (finally) escape the clutches of GDI. Avalon is the display technology related to XAML. XAML is an XML markup language to describe Avalon interfaces.

My interest in these technologies spans quite a range of concerns, but top on my list is how they might impact my Business Objects books and the related CSLA .NET framework.

It turns out that the news is pretty good. Seriously good actually. In fact, it looks like people using CSLA .NET today are going to be very happy over the next few years.

Within the context of CSLA .NET, Indigo is essentially a drop-in replacement for Remoting. I will have to change the DataPortal to use Indigo, but that change should have no impact on an application’s UI, business logic or data access code. In other words (cross your fingers), a business application based on CSLA .NET should move to Indigo with essentially no code changes.

[Disclaimer: Indigo isn’t even in beta, so anything and everything could change. My statements here are based on what I’ve seen and heard, and thus may change over time as well.]

One of my primary goals for CSLA .NET 2.0 is to alter the DataPortal to make it easier to adapt it to various transports. In the short term this means Remoting, asmx, WSE and Enterprise Services (DCOM). But it also means I’ll be able to add Indigo support with relative ease once Indigo actually exists.

Avalon is a different story. Avalon is a new UI technology, which means that moving to Avalon means tossing out your existing UI and building a new one. But if you are using Windows Forms today, with CSLA .NET business objects for your logic and databinding to connect the two together your life will be better than most. It appears that Avalon will also support databinding against objects just like (hopefully better than) Windows Forms.

Since a well-written CSLA-based Windows Forms application doesn’t have any business logic (not even validation) in the UI itself, switching to Avalon should merely be a rip-and-replace of the UI, with little to no impact on the underlying business or data access layers. I keep telling people that the “UI is expendable”, and here’s the proof :-)

I just thought I’d share these observations. Indigo and Avalon (together under the label of WinFX) won’t show up for quite a long time, so none of this is of immediate interest. Still it is nice to know that when it does show up sometime in the future that CSLA .NET will have helped people to move their applications more easily to the new technologies.

DataAdapter.Fill preferable to DataReader?

Fri, 25 Feb 2005 20:28:57 GMT

In this article, Bill Vaughn voices his view that DataAdapter.Fill should typically be a developer's first choice for getting data rather than using a DataReader directly.

In my VB.NET and C# Business Objects books I primarily use the DataReader to populate objects in the DataPortal_Fetch methods. You might infer then, that Bill and I disagree.

While it is true that Bill and I often get into some really fun debates, I don't think we disagree here.

Bill's article seems to be focused on scenarios where UI developers or non-OO business developers use a DataReader to get data. In such cases I agree that the DataAdapter/DataTable approach is typically far preferable. Certainly there will be times when a DataReader makes more sense there, but usually the DataAdapter is the way to go.

In the OO scenarios like CSLA .NET the discussion gets a bit more complex. In my books I discussed why the DataReader is a good option - primarily because it avoids loading the data into memory just to copy that data into our object variables. For object persistence the DataReader is the fastest option.

Does that mean it is the best option in some absolute sense?

Not necessarily. Most applications aren't performance-bound. In other words, if we lost a few milliseconds of performance it is likely that our users would never notice. For most of us, we could trade a little performance to gain maintainability and be better off.

As a book author I am often stuck between two difficult choices. If I show the more maintainable approach the performance Nazis will jump on me, while if I show the more performant option the maintainability Nazis shout loudly.

So in the existing books I opted for performance at the cost of maintainability. Which is nice if performance is your primary requirement, and I don’t regret the choice I made.

(It is worth noting that subsequent to publication of the books, CSLA .NET has been enhanced, including enhancing the SafeDataReader to accept column names rather than ordinal positions. This is far superior for maintenance, with a very modest cost to performance.)

Given some of the enhancements to the strongly typed DataAdapter concept (the TableAdapter) that we’ll see in .NET 2.0, it is very likely that I’ll switch and start using TableAdapter objects in the DataPortal_xyz methods.

While there’s a performance hit, the code savings looks to be very substantial. Besides, it is my opportunity to make the maintenance-focused people happy for a while and let the performance nuts send me nasty emails. Of course nothing I do will prevent the continued use the DataReader for those who really need the performance.

List of ORM tools, code generators and related stuff

Wed, 23 Feb 2005 20:51:21 GMT

Rich put together this valuable list of tools - thank you!!

Serialization of events in .NET 1.x and 2.0

Wed, 23 Feb 2005 18:06:27 GMT

Sahil has started a discussion on a change to the BinaryFormatter behavior in .NET 2.0.

Serialization is just the process of taking complex data and converting it into a single byte stream. Motivation is a separate issue :)

There are certainly different reasons to "serialize" an object.

One for what I do in CSLA .NET, which is to truly clone an object graph (not just an object, but a whole graph), either in memory or across the network. This is a valid reason, and has a set of constraints that a serializer must meet to be useful. This is what the BinaryFormatter is all about today.

Another is to easily convert some or all of an object’s data into actual data. To externalize the object’s state. This is what the XmlSerializer is all about today. The purpose isn’t to replicate a .NET type, it is to convert that type into a simple data representation.

Note that in both cases the formatter/serializer attempts to serialize the entire object graph, not just a single object. This is because the “state” of an object is really the state of the object graph. That implies that all references from your object to any other objects are followed, because they collectively constitute the object graph. If you want to “prune” the graph, you mark references such that the formatter/serializer doesn’t follow them.

In the case of the BinaryFormatter, it does this by working with each object’s fields, and it follows references to other objects. An event inside an object is just another field (though the actual backing field is typically hidden). This backing field is just a delegate reference, which is just a type of object reference – and thus that object reference is followed like any other.

In the case of the XmlSerializer, only the public fields and read/write properties are examined. But still, if one of them is a reference the serializer attempts to follow that reference. The event/delegate issue doesn’t exist here only because the backing field for events isn’t a public field. Make it a public field and you’ll have issues.

In .NET 1.x the BinaryFormatter attempts to follow all references unless the field is marked as [NonSerializable]. In C# the field: target provided (what I consider to be) a hack to apply this attribute to the backing field. It also has a better approach, which is to use a block structure to declare the event so you get to manually declare the backing field and can apply the attribute yourself.

The trick is that the default behavior is for the backing field to be serializable, and things like Windows Forms or other nonserializable objects might subscribe to the object’s events. When the BinaryFormatter follows the references to those objects it throws an exception (as it should).

I made a big stink about this when I wrote my VB Business Objects book though, because I discovered this issue and there was no obvious solution. This is because VB.NET has neither the Field: target hack, nor the block structure declaration for events. I was forced to implement a C# base class to safely declare my events.

I spent a lot of time talking to both the VB and CLR (later Indigo) teams about this issue.

The VB team provided a solution by supporting the block structure for declaring events, thus allowing us to have control over how the backing field is serialized. This is a very nice solution as I see it.

I am not familiar with the change Sahil is seeing in .NET 2.0. But that is probably because I’ve been using the block structure event declarations as shown in my C# and VB 2005 links above, so I’m manually ensuring that only serializable event handlers are being traced by the BinaryFormatter.

But I should point out that the C# code in the example above is for .NET 1.1 and works today just as it does in .NET 2.0.

Fire bad, tree pretty

Fri, 18 Feb 2005 18:11:26 GMT

In a recent online discussion the question came up “If ‘the middle tier’ has no knowledge about the actual data it's transporting, then what value is it adding?”

The answer: database connection pooling.

Pure and simple, in most rich client scenarios the only reason for a "middle tier" is to pool database connections. And the benefit can be tremendous.

Consider 200 concurrent clients all connecting to your database using conventional coding. They'll have at least 200 connections open, probably more. This is especially true since each client does its own "connection pooling", so typically a client will never close its connection once established for the day.

Then consider 200 clients going through a middle tier "app server" that does nothing but ferry the data between the clients and database. But it has the code to open the connections. Now those 200 clients might use just 3-4 connections to the database rather than 200, because they are all pooled on the server.

Was there a performance hit? Absolutely. Was there a scalability gain? Absolutely. Is it more expensive and harder to build/maintain? Absolutely.

This middle tier stuff is not a panacea. In fact its cost is typically higher than the benefit, because most applications don't actually have enough concurrent users to make it worth the complexity. But people are enamored of the idea of "n-tier", thinking it requires an actual physical tier...

I blame it on Windows DNA and those stupid graphic representations. They totally muddied the waters in people's understanding the difference between n-layer (logical separation) and n-tier (physical separation).

People DO want n-layer, because that provides reuse, maintainability and overall lower costs. Logical separation of UI, business logic, data access and data storage is almost always of tremendous benefit.

People MIGHT want n-tier if they need the scalability or security it can offer, and if those benefits outweigh the high cost of building a physically distributed system. But the cost/benefit isn’t there as often as people think, so a lot of people build physical n-tier systems for no good reason. They waste time and money for no real gain. This is sad, and is something we should all fight against.

I make my living writing books and articles and speaking about building distributed systems. And my primary message is just say NO!

You should be forced into implementing physical tiers kicking and screaming. There should be substantial justification for using tiers, and those justifications should be questioned at every step along the way.

At the same time, you should encourage the use of logical layers at all times. There should be substantial justification for not using layers, and any argument against layering of software should be viewed with extreme skepticism.

Layering is almost always good, tiers are usually bad.

Indigo: the future of asmx/wse/es/msmq/remoting

Tue, 08 Feb 2005 18:35:52 GMT

I just finished watching Eric Rudder’s keynote on Indigo at VS Live in San Francisco . As with all keynotes, it had glitz and glamour and gave a high-level view of what Microsoft is thinking.

(for those who don’t know, Eric is the Microsoft VP in charge of developer-related stuff including Indigo)

Among the various things discussed was the migration roadmap from today’s communication technologies to Indigo. I thought it was instructive.

From asmx web services the proposed changes are minor. Just a couple lines of code change and away you go. Very nice.

From WSE to Indigo is harder, since you end up removing lots of WSE code and replacing it with an attribute or two. The end result is nice because your code is much shorter, but it is more work to migrate.

From Enterprise Services (COM+, ServicedComponent) the changes are minor – just a couple lines of changed code. But the semantic differences are substantial because you can now mark methods as transactional rather than the whole class. Very nice!

From System.Messaging (MSMQ) to Indigo the changes are comparable in scope to the WSE change. You remove lots of code and replace it with an attribute or two. Again the results are very nice because you save lots of code, but the migration involves some work.

From .NET Remoting to Indigo the changes are comparable to the asmx migration. Only a couple lines of code need to change and away you go. This does assume you listened to advice from people like Ingo Rammer, Richard Turner and myself and avoided creating custom sinks, custom formatters or custom channels. If you ignored all this good advice then you’ll get what you deserve I guess :-)

As Eric pointed out however, Indigo is designed for the loosely coupled web service/SOA mindset, not necessarily for the more tightly coupled n-tier client/server mindset. He suggested that many users of Remoting may not migrate to Indigo – directly implying that Remoting may remain the better n-tier client/server technology.

I doubt he is right. Regardless of what Indigo is designed for, it is clear to me that it offers substantial benefits to the n-tier client/server world. These benefits include security, reliable messaging, simplified 2-phase transactions and so forth. The fact that Indigo can be used for n-tier client/server even if it is a bit awkward or not really its target usage won’t stop people. And from today’s keynote I must say that it looks totally realistic to (mis)use Indigo for n-tier client/server work.

Remoting vs Web Services vs ES/COM+/DCOM

Fri, 04 Feb 2005 04:11:20 GMT

I’ve hashed and rehashed this topic numerous times. In particular, read this and this. But the debate rages on, paralyzing otherwise perfectly normal development teams in a frenzy of analysis paralysis over something that is basically not all that critical in a good architecture.

I mean really. If you do a decent job of architecting, it really doesn’t matter a whole hell of a lot which RPC protocol you use, because you can always switch away to another one when required.

And while web services a pretty obvious choice for SOA, the reality is that if you are debating between remoting, web services and DCOM/ES/COM+ then you are looking for an RPC protocol – end of story.

If you were looking to do service-oriented stuff you’d reject remoting and DCOM/ES/COM+ out of hand because they are closed, proprietary, limited technologies that just plain don’t fit the SO mindset.

In an effort to summarize to the finest point possible, I’ll try once again to clarify my view on this topic. In particular, I am going to focus on why you might use remoting and how to use it if you decide to. Specifically I am focusing on the scenario where remoting works better than either of its competitors.

First, the scenario that remoting does that web services and ES/COM+/DCOM don’t do (without hacking them):

1. You want to pass rich .NET types between AppDomains, Processes or Machines (we’re talking Hashtables, true business objects, etc.)

2. You are communicating between layers of an application that are deployed in different tiers (We’re not talking services here, we’re talking layers of a single application. We’re talking client-server, n-tier, etc.)

3. You want no-touch deployment on the client

If you meet the above criteria then remoting is the best option going today. If you only care about 1 and 2 then ES/COM+/DCOM is fine – all you lose is no-touch deployment (well, and a few hours/days in configuring DCOM, but that’s just life :-) ).

If you don’t care about 1 then web services is fine. In this case you are willing to live within the confines of the XmlSerializer and should have no pretense of being object-oriented or anything silly like that. Welcome to the world of data-centric programming. Perfectly acceptable, but not my personal cup of tea. To be fair, it is possible to hack web services to handle number 1, and it isn't hard. So if you feel that you must avoid remoting but need 1, then you aren't totally out of luck.

But in general, assuming you want to do 1, 2 and 3 then you should use remoting. If so, how should you use remoting?

1. Host in IIS

2. Use the BinaryFormatter

3. Don’t create custom sinks or formatters, just use what Microsoft gave you

4. Feel free to use SSL if you need a secure line

5. Wrap your use of the RPC protocol in abstraction objects (like my DataPortal or Fowler’s Gateway pattern)

Hosting in IIS gives you a well-tested and robust process model in which your code can run. If it is good enough for www.microsoft.com and www.msn.com it sure should be good enough for you.

Using the BinaryFormatter gives you optimum performance and avoids the to-be-deprecated SoapFormatter.

By not creating custom sinks or formatters you are helping ensure that you’ll have a relatively smooth upgrade path to Indigo. Indigo, after all, wraps in the core functionality of remoting. They aren’t guaranteeing that internal stuff like custom sinks will upgrade smoothly, but they have been very clear that Indigo will support distributed OO scenarios like remoting does today. And that is what we’re talking about here.

If you need a secure link, use SSL. Sure WSE 2.0 gives you an alternative in the web service space, but there’s no guarantee it will be compatible with WSE 3.0, much less Indigo. SSL is pretty darn stable comparatively speaking, and we’ve already covered the fact that web services doesn’t do distributed OO without some hacking.

Finally, regardless of whether you use remoting, web services or ES/COM+/DCOM make sure – absolutely sure – that you wrap your RPC code in an abstraction layer. This is simple good architecture. Defensive design against a fluid area of technology.

I can’t stress this enough. If your business or UI code is calling web services, remoting or DCOM directly you are vulnerable and I would consider your design to be flawed.

This is why this whole debate is relatively silly. If you are looking for an n-tier RPC solution, just pick one. Wrap it in an abstraction layer and be happy. Then when Indigo comes out you can easily switch. Then when Indigo++ comes out you can easily upgrade. Then when Indigo++# comes out you are still happy.

C# event issue resolved

Fri, 07 Jan 2005 02:58:17 GMT

OK, I figured it out (I think)

[NonSerialized]

EventHandler _nonSerializableHandlers;

EventHandler _serializableHandlers;

/// <summary>

/// Declares a serialization-safe IsDirtyChanged event.

/// </summary>

public event EventHandler IsDirtyChanged

{

add

{

if (value.Target.GetType().IsSerializable)

_serializableHandlers =

(EventHandler)Delegate.Combine(_serializableHandlers, value);

else

_nonSerializableHandlers =

(EventHandler)Delegate.Combine(_nonSerializableHandlers, value);

}

remove

{

if (value.Target.GetType().IsSerializable)

_serializableHandlers =

(EventHandler)Delegate.Remove(_serializableHandlers, value);

else

_nonSerializableHandlers =

(EventHandler)Delegate.Remove(_nonSerializableHandlers, value);

}

/// <summary>

/// Call this method to raise the IsDirtyChanged event.

/// </summary>

virtual protected void OnIsDirtyChanged()

{

if (_nonSerializableHandlers != null)

_nonSerializableHandlers(this, EventArgs.Empty);

if (_serializableHandlers != null)

_serializableHandlers(this, EventArgs.Empty);

}

I temporarily forgot that C# makes you invoke the delegate directly anyway, so having a separate clause in the manual event declaration isn’t required. I still think it makes the code easier to read, but functionality is king in the end.

Limitation on manual declaration of C# events?

Fri, 07 Jan 2005 02:42:22 GMT

Several months ago I posted an entry about the new VB syntax for manual declaration of events – specifically showing how it can be used to solve today’s issue with attempted serialization of nonserializable event listeners.

Subsequent to that post, I posted a follow-up with a better version of the code, thanks to input from a reader.

At the moment I am working through some eventing issues in both VB and C#, and I’ve found what appears to be a troubling limitation in C#.

In the new VB manual declaration scheme we have the ability to manually raise the event using the backing field. This allows me to have two backing fields - one serialized and one not as shown in my follow-up post. This is really nice, because it means we can retain serializable delegate references, while dropping nonserializable references.

Unfortunately, it doesn’t appear that the C# syntax allows us to control how the backing field is invoked. It appears that only one backing field is possible, and it is invoked automatically such that we don't have control over the invocation.

If we can’t control the invocation, then we can’t invoke both the serialized and nonserialized set of delegates. This will force the C# code to treat all the delegates as nonserialized, even those that could be serialized.

Of course I’m researching this for CSLA .NET 2.0. Wouldn’t it be a joke if this time the C# framework had to have some VB code (since last time it was the other way around)?

Anyone have any insight into a solution on the C# side?

Indigo and SOA in San Francisco

Thu, 06 Jan 2005 22:48:49 GMT

Want to hear about SOA and/or Indigo? Eric Rudder, Don Box, Doug Purdy and Rich Turner are all speaking at VS Live (as is yours truly).

Even if you like me, are rather skeptical about SOA, the fact is that Indigo is coming.

Don't let the SOA hype fool you. Indigo will impact you if you use .NET.

Personally I look at Indigo much more as a replacement for remoting and DCOM, along with integrating the WSE stuff into Web services. Because of this, Indigo is a very important thing to me - and to anyone building client/server or n-tier distributed systems in .NET.

Indigo alters the way objects are serialized, the way data is marshalled across networks and more. It is pretty extensive, and is going to be harder to abstract away than either asmx or remoting have been. This means we, as consumers of the technology, will need to understand more of it than we have needed to with existing technologies.

Since VS Live has a whole day on Indigo, this is a chance to get a good look at what's coming and assess what it is going to do to you.

And of course while you are at VS Live, you can attend my distributed object-oriented workshop :-)

Egyptian mythology and object-oriented design

Sun, 19 Dec 2004 04:27:43 GMT

I was relaxing today, visiting with my wife and she read me the text from this link. It got me thinking… (and yes, I am way too geeky for my own good sometimes ;-) )

The ancient Egyptians had a pretty solid understanding of decomposition into object-oriented models al la CRC-style analysis.

(Stargate SG-1’s general misuse of the goddess Maat notwithstanding, and totally ignoring the real world speculation that Maat was an alien)

Look at the description of the goddess Maat’s role. She has a very clearly delineated and focused responsibility. As an object, Maat’s job is to measure justice vs evil. Arguably this is the job of the scale, but given that there’s a goddess involved, I’m giving more weight (pun intended) to her than to the inanimate scale.

She doesn’t decide what to do about evil, or how to mete out justice. This is not her role.

Osiris on the other hand, doesn’t measure justice or evil, nor does he perform punishment. His responsibility is also clearly delineated as a single “business rule” wherein he passes the Judged off to the appropriate “handler“.

To do this he collaborates with Maat (and technically with the scale) and with the Judged. He has a using relationship with Ahemait, where he passes the Judged if they fail to meet the business requirement by having an excess quantity of evil (as measured by Maat).

The fact that the Egyptians realized that justice had two parts, or two responsibilities, is very interesting. They properly separated the responsibility of measuring evil (as a single, discrete concept) from the responsibility of judging outcomes based on that measurement. A distinction that was fortunately carried forward into the US justice system. The measurement of evil is the responsibility of the jury, while the judging is a separate concern as is the actual implementation of punishment. Next time you are called for jury duty, keep in mind that you are filling the role of the goddess.

Not only did the Egyptians get the separation of responsibilities, but they understood the idea of object collaboration. This is illustrated by the fact that Osiris can’t do his job without collaborating with other “objects” in the system.

And to think. All this time I’ve been saying that object-orientation has been around for about 30 years. I guess I was off by a couple orders of magnitude. It is more like 3000 years :-)

Conversations about services, remoting and Santa

Thu, 09 Dec 2004 19:24:47 GMT

Last night a group of top developers from the Minneapolis area had dinner with David Chappell. Great company, great food and great conversation; what more could you want?

The conversation ranged wide, from whether parents should perpetuate the myth of the Tooth Fairy and Santa all the way down to how Whidbey’s System.Transactions compares to the transactional functionality in BizTalk Server.

At one point during the conversation David mentioned that there’s not only an impedance mismatch between objects and relational data, but also between users and objects and between services and objects. While the user-object mismatch is somewhat understood, we’re just now starting to learn the nature of the service-object mismatch.

(The OO-relational impedance mismatch is well-known and is discussed in many OO books, perhaps most succinctly in David Taylor’s book)

I think that his observation is accurate, and it is actually the topic of an article I submitted to TheServerSide.net a few days ago, so look for that to stir some controversy in the next few weeks :-)

I also heard that some people have read my blog entries on remoting (this one in particular I think) and somehow came to the conclusion that I was indicating that Indigo would be binary compatible with remoting. To be clear, as far as I know Indigo won’t be binary compatible with remoting. Heck, remoting isn’t binary compatible with remoting. Try accessing a .NET 1.1 server from a 1.0 client and see what happens!

What I said is that if you are careful about how you use remoting today you’ll have a relatively easy upgrade path to Indigo. Which is no different than using asmx or ES today. They’ll all have an upgrade path to Indigo, but that doesn’t imply that Indigo will natively interoperate with remoting or ES (DCOM).

(Presumably Indigo will interop with asmx because that’s just SOAP, but even then you’ll want to change your code to take advantage of the new Indigo features, so it is still an upgrade process)

What does “be careful how you use remoting” mean? This:

Use the Http channel
Host in IIS
Use the BinaryFormatter
Do not create or use custom sinks, formatters or channels

If you follow these rules you should have a relatively direct upgrade path to Indigo. Will you have to change your code? Of course. That’s what an upgrade path IS. It means you can, with relative ease, update your code to work with the new technology.

Technologies for crossing tiers vs crossing trust boundaries

Thu, 21 Oct 2004 13:34:31 GMT

I’ve been involved in yet another discussion about whether people should use Remoting (vs DCOM/Enterprise Services or ASMX/WSE), and I thought I’d share some more of my thoughts in this regard.

One key presupposition in this discussion is often than we’re talking about communication across service boundaries. The train of thought goes that Microsoft (at least the Indigo team) seems to indicate that the only cross-network communication we’ll ever do in the future is across service boundaries. In short, n-tier is dead and will never happen. Anywhere n-tier would exist should be treated as crossing a service (and thus trust) boundary.

I consider that assumption to be fundamentally flawed.

Just because the Indigo team sees a future entirely composed of services doesn't mean that is a correct answer. When MSMQ first came out the MSMQ team was telling everyone that all communication should be done through MSMQ. Filter through the absurd bias of the product teams and apply some basic judgment and it is clear that not all cross-network communication will be service-oriented.

The idea of crossing trust boundaries between parts of my application incurs a terribly high cost in terms of duplicate logic and overhead in converting data into/out of messages. The complexity and overhead is tremendous and makes no sense for communication between what would have been trusted layers in my application…

In a world where service-orientation exists, but is not the only story, there's a big difference between creating a n-layer/n-tier application and creating a set of applications that work together via services. They are totally different things.

I totally agree that neither Remoting nor DCOM/ES are appropriate across service boundaries. Those technologies are not appropriate between applications in general.

However, I obviously do not for a minute buy the hype that service-orientation will obviate the need for n-tier architectures. And between tiers of your application, ASMX/WSE is a very limited and poor choice. As an example, ASMX/WSE uses the highly limited XmlSerializer, which is virtually useless for transferring objects by value. Between tiers of your application either Remoting or DCOM/ES are good choices.

Application tiers are just application layers that happen to be separated across a network boundary. By definition they are tightly coupled, because they are just layers of the same application. There are many techniques you can use here, all of which tend to be tightly coupled because you are inside the same service and trust boundary even though you are crossing tiers.

I’ve personally got no objection to using DCOM/ES between tiers running on different servers (like web server to app server). The deployment and networking complexity incurred is relatively immaterial in such scenarios.

But if you are creating rich or smart client tiers that talk to an app server then the loss of no-touch deployment and the increased networking complexity incurred by DCOM/ES is an overwhelming negative. In most cases this complexity and cost is far higher than the (rather substantial) performance benefit. In this scenario Remoting is usually the best option.

Of course, if you can’t or won’t use Remoting in this case, you can always hack ASMX/WSE to simulate Remoting to get the same result (with a small performance penalty). You can wrap both ends of the process manually so you can use the BinaryFormatter and actually get pass-by-value semantics with ASMX/WSE. The data on the wire is a Base64 byte array, but there's no value in being "open" between layers of your own application anyway so it doesn't matter.

There's a performance hit due to the Base64 encoding, but it may be worth it to gain the benefits of WSE. And it may be worth it so you can say that you are conforming to recommendations against Remoting.

For better or worse, Remoting does some things that neither DCOM/ES nor ASMX/WSE do well. You have the choice of either using Remoting (with its commensurate issues) or of hacking to make DCOM or ASMX/WSE do what Remoting can do. Neither choice is particularly appealing, and hopefully Indigo will provide a better solution overall. Only time will tell.

Browser as a "client tier"?

Sat, 09 Oct 2004 04:32:33 GMT

The adoguy is wondering if the browser is a "client tier". I think the web world is a flip-flopper.

To many the browser is a terminal. Pure and simple. Using it for anything outside of the basic capabilities is non-standard and should be avoided. In this worldview the browser is no tier. It is just a terminal, and granting it “tier“ status would mean granting 3270 mainframe terminals that same status - thus saying that the COBOL guys from 30 years ago did n-tier development. Go there if you want, but I'm skeptical.

To many others the browser is a programming platform. The “new rich client”, against which we can and should target rich, interactive programs. In this worldview the browser is a place to run code. It is not a terminal, it is a programming platform just like Windows, Linux or anything else. In this worldview you bet the browser can be a tier. Heck, you could probably write your UI and business layers into the browser tier if you really wanted to...

Honestly I thought this debate died with the dot-bomb. The debate was a growing concern back in 2000 or so, and then it rather died off when the excess money evaporated at the end of the Clinton era boom.

Though certainly some organizations have undertaken the massive pain and cost associated with targeting the browser as a programming platform (examples include Microsoft with Outlook Web Access, and as someone pointed out, Google with gmail)... But it is a losing battle for most of us, because there are no development tools.

If Microsoft wanted to support the browser as a programming platform, we'd have Internet Explorer project types in Visual Studio and the experience would be comparable to programming Windows Forms (though probably in JavaScript of all things).

But the only truly viable tool today (for developers who want productivity and reasonable development cost) is Flash. And with the recent advent of laszlo it looks like Flash is seriously on the move as a community effort. Now if only there was a laszlo equivelent that could be hosted in ASP.NET we'd be talking!!

So my take on it is this.

If you are willing to actually write UI code in JavaScript or Flash then you can consider the browser to be a tier. To be a tier it has to DO something beyond basic terminal functions. (and the ASP.NET validation controls just can't count as enough to elevate it to tier status)

But if you are like most people, the browser is a terminal - a glorified, colorful mainframe terminal just like we've had for decades. In this worldview it can't be considered a tier because no meaningful code runs closer to the user than your web server.

Frameworks, O/R mappers and other plumbing

Fri, 01 Oct 2004 20:54:40 GMT

It seems that TheServerSide.net has a debate going this week about O/R Mapping. As part of the discussion, one of the invited debaters made the following comment about CSLA .NET:

“…everyone just "loves" CSLA, afterall its got every feature imaginable, was created by one of the best "experts", is the subject of books and user groups -- and yes there are code gen templates for it. Now don't get me wrong, I think Rocky is a great speaker and teacher (and great in person too -- I have met him), but I think he would agree that CSLA is an example for teaching how to do something and should not really be the expected to be perfect out-of-the-box -- and its not. I've personally seen a project spend way too much time and effort doing things because they chose this route -- they had to constantly go and add/remove/modify pieces to their chosen framework -- and I've heard many others that have similar problems.” (link)

- Paul Wilson

It is not the most professionally phrased comment, but this gentleman has previously made similar comments in other forums as well, so I thought I should provide an answer.

I think it is important to note up front that Mr. Wilson sells an O/R Mapper tool, and so has some bias. It is also important to note that I have written numerous books on distributed object-oriented architecture, and so I have some bias.

CSLA .NET is intended to be a learning or teaching vehicle. It is often also useful in its own right as a development framework.

Different people write different types of books. For instance, Martin Fowler’s excellent Patterns of Enterprise Application Architecture came out at about the same time as my .NET Business Objects book. In many ways he and I are targeting the same demographic, but in different ways. Fowler’s book stays at a more abstract, conceptual level and teaches from there. My book is more pragmatic and applied, teaching from there.

As an aside, I find it wonderful how some of the patterns in Fowler’s book can be found in CSLA .NET. I think this goes to show the power of the design pattern concept – how patterns emerge in the wild and are documented to help provide a more common language we can all use in future discussions.

But back to books. We need books that give us broad theory and concepts. We also need reference-oriented books that give us basic usage of .NET, C# and VB. And we need books that try to span the gulf between the two. This is where I’ve tried to live for the most part – between the theory and reference content. To do this is challenging, and requires that the theory be distilled down into something concrete, but yet rich enough that it isn’t just more reference material.

The way I’ve chosen to do this is to pick a way of creating distributed object-oriented applications. Are there other ways? Oh yeah, tons. Are some of them good? Absolutely! Are some of them bad? Again, absolutely. Is CSLA .NET good? It depends.

Certainly I think CSLA .NET and the rest of the content in my business objects books are good for learning how to apply a lot of theory into a complex problem space. I think that many readers use ideas and concepts from my books in conjunction with ideas from Fowler’s books, the GoF book and lots of other sources.

A side-effect of writing a book on applied theory is that the theory does actually get applied. This means that the book includes a pretty functional framework that can help build distributed object-oriented applications. This framework, CSLA .NET, can be useful as-is in some scenarios. There’s a vibrant online forum for discussion of my books and CSLA .NET where you can find numerous examples where people have applied the framework to help solve their problems.

Many of those people have also modified the framework to better suit their needs or requirements. This is nothing but good. In several cases, modifications made by readers of the books have been applied back into the framework, which is why the framework is currently at version 1.4, where the book described version 1.0.

Does CSLA .NET fit every need? Of course not. No tool meets every need.

It is particularly good at creating performant, scalable line-of-business systems that involve a lot of data entry and business rules and relationships. It supports people who build a good object-oriented business entity model, because it provides for virtually unlimited flexibility in translating the object data to arbitrary (relational and/or non-relational) persistence targets (databases, etc).

It is not particularly good at creating reporting systems, large batch processing systems or systems that are very data-centric and have little business logic. There’s also a cost to the data mapping flexibility I mentioned as a benefit – because with that flexibility comes more work on the developer. If you have either a non-complex data source or you are willing to tie your object model to your data model then CSLA .NET isn’t ideal because it requires that you do more work. If you want your object model to mirror your data model then use a DataSet, that’s what it is designed for.

But the real thing to keep in mind, above all else, is this: there is a set of functionality that must exist to build distributed object-oriented enterprise applications. This includes data persistence, UI design support, business logic, organization of layers and tiers, state management and more.

The issues that I address in my business objects books and CSLA .NET are addressed in every enterprise application. Whether formally or informally, whether through reusable or ad-hoc implementations – everything I address happens in every enterprise app. (And a whole lot more things I don't address are in every enterprise app!!)

The idea that CSLA .NET has “got every feature imaginable” seems to imply that it has extra stuff you don’t need. The fact is that you will address every issue it covers one way or another, plus numerous other issues. You might address them differently that I did, and that’s great. But you will address them.

You can address them in an ad-hoc manner in each app you write. You can address them through a framework, or you can address them through tools. You might address them through a combination of the above.

But I’ll say this: the ad-hoc approach is poor. You should opt for a solution that provides reuse and consistency. There are numerous ways to do this, CSLA .NET is one and I imagine that Mr. Wilson’s O/R mapper tool is another. And there are various commercial frameworks and other O/R mappers out there as well. You have a lot of choices – which is indicative that this is a healthy segment of our industry.

The fact is that at some point you need to make hard decisions within your organization that move from the theory of design patterns and architecture down to the practical and pragmatic level of code, and you only want to do that once. It is expensive, and it does nothing to solve your actual business needs. All this code that does data persistence, layering, tiering, data binding and whatever else is just plumbing. It is necessary, it is costly, it is time-consuming, and it provides no direct business value.

And be warned – if you buy or otherwise acquire a framework or tool set (CSLA .NET, O/R mappers, whatever) you will almost certainly need to modify, extend, add or remove bits and pieces to make it fit your needs. If you do this by buying a black-box component of any sort, you’ll end up extending or modifying around the component, while if you acquire one that includes code you may opt to make the changes inside or outside the component.

Obviously you can acquire a component that does less work for you – something targeted at a narrow task. In that case you might not need to modify it, but you must face up to the fact that you will need to write all the other stuff that it doesn’t do. It isn’t like you can escape the requirement for all this functionality. Every app has it, whether you do it in a reusable form or not is up to you!

So when it comes to frameworks, O/R mappers and all this other plumbing: do it once – build it, buy it, cobble together products to do it. Whatever suits your fancy.

Then get on with building business apps and providing actual value to your organization.

An answer to the Remoting confusion

Fri, 27 Aug 2004 04:32:31 GMT

OK, now I feel better. Perhaps I jumped the gun with my previous post.

Rich Turner gave an awesome presentation – totally on the mark from start to end.

He was very, very clear that the prescriptive guidance is to use asmx (web services) to cross service boundaries and to use Enterprise Services (COM+), MSMQ, Remoting or asmx inside a service boundary.

Note that inside a service might be multiple tiers. Multiple physical tiers. You might cross network boundaries (though that should be minimized), but that’s OK. This is all inside your service, within your control. Since it is inside your control, you should choose the appropriate technology based on all criteria (such as performance, transactional support, security, infrastructure support, deployment and so forth).

This is the best and most clear guidance I’ve heard from Microsoft yet. Very nice!

More confusion about Microsoft's guidance for Remoting

Fri, 27 Aug 2004 03:29:00 GMT

I’d laugh except that it makes me want to cry.

I’m at a Microsoft training event, being briefed on various technologies by people on the product teams – including content on Indigo.

The unit manager gave an overview, and someone asked about the recommended architecture guidance around today’s Remoting technology. He reiterated that the recommendation is to only use it within a process. This, after he’d just finished pointing out that there are scenarios today that are only solved by remoting.

Say what?

Then several other Indigo team members covered various features of Indigo and how they map to today’s technology and how we may get from today to Indigo. Numerous times it comes up that Indigo incorporates much of the Remoting model (because it is good), and that most code using Remoting today will transparently migrate to Indigo when it gets here.

So what now?

First, the prescriptive guidance is nuts. They are saying conflicting things and just feeding confusion. Remoting is sometimes the only answer, but don't use it?

I'm sorry, I have to build real apps between now and when ever Indigo shows up. If Remoting is the answer, then it is the answer. End of story.

Second, it turns out that you are fine with Remoting as long as you don’t create custom sinks or formatters for Remoting your code will move to Indigo just as easily as any asmx code you write today (which is to say with minimal code changes).

And of course you should avoid the TCP channel and custom hosts – use IIS, the HttpChannel and the BinaryFormatter and life is good.

Finally, (as I’ve discussed before), Remoting is for communication between tiers of your own application. If you are communicating across trust boundaries (between separate applications) then you should use web services – or better yet use WSE 2.0.

Conversely, if you are using web services or WSE 2.0, then you have inserted a trust boundary and you shouldn’t be pretending that you are communicating between tiers – you are now communicating between separate applications.

Middle-tier hosting: Enterprise Services, IIS, DCOM, Web services and Remoting

Wed, 11 Aug 2004 23:45:39 GMT

There is this broad-reaching debate that has been going on for months about remoting, Web services, Enterprise Services, DCOM and so forth. In short, it is a debate about the best technology to use when implementing client/server communication in .NET.

I’ve weighed in on this debate a few times with blog entries about Web services, trust boundaries and related concepts. I’ve also had discussions about these topics with various people such as Clemens Vasters, Juval Lowy, Ingo Rammer, Don Box and Michele Leroux Bustamante. (all the “experts” tend to speak at many of the same events, and get into these discussions on a regular basis)

It is very easy to provide a sound bite like “if you aren’t doing Enterprise Services you are creating toys” or something to that effect. But that is a serious oversimplification of an important issue.

Because of this, I thought I’d give a try at summarizing my thoughts on the topic, since it comes up with Magenic’s clients quite often as well.

Before we get into the article itself, I want to bring up a quote that I find instructive:

“The complexity is always in the interfaces” – Craig Andrie

Years ago I worked with Craig and this was almost like a mantra with him. And he was right. Within a small bit of code like a procedure, nothing is ever hard. But when that small bit of code needs to use or be used by other code, we have an interface. All of a sudden things become more complex. And when groups of code (objects or components) use or are used by other groups of code things are even more complex. And when we look at SOA we’re talking about entire applications using or being used by other applications. Just think what this does to the complexity!

Terminology

I think a lot of the problem with the debate comes because of a lack of clear terminology. So here are the definitions I’ll use in the rest of this article:

Term	Meaning
Layer	A logical grouping of similar functionality within an application. Often layers are separate .NET assemblies, though this is not a requirement.
Tier	A physical grouping of functionality within an application. There is a cross-process or cross-network boundary between tiers, providing physical isolation and separation between them.
Application	A complete unit of software providing functionality within a problem domain. Applications are composed of layers, and may be separated into tiers.
Service	A specific type of application interface that specifically allows other applications to access some or all of the functionality of the application exposing the service. Often this interface is in the form of XML. Often this XML interface follows the Web services specifications.

I realize that these definitions may or may not match those used by others. The fact is that all of these terms are so overloaded that intelligent conversation is impossible without some type of definition/clarification. If you dislike these terms, please feel free to mentally mass-substitute them for your favorite overloaded term in the above table and throughout the remainder of this article J.

First, note that there are really only three entities here: applications, tiers and layers.

Second, note that services are just a type of interface that an application may expose. If an application only exposes a service interface, I suppose we could call the application itself a service, but I suggest that this only returns us to overloading terms for no benefit.

A corollary to the above points is that services don’t provide functionality. Applications do. Services merely provide an access point for an application’s functionality.

Finally, note that services are exposed for use by other applications, not other tiers or layers within a specific application. In other words, services don’t create tiers, they create external interfaces to an application. Conversely, tiers don’t create external interfaces, they are used exclusively within the context of an application.

Layers

In Chapter 1 of my .NET Business Objects books I spend a fair amount of time discussing the difference between physical and logical n-tier architecture. By using the layer and tier terminology perhaps I can summarize here more easily.

An application should always be architected as a set of layers. Typically these layers will include:

Presentation
Business logic
Data access
Data management

The idea behind this layering concept is two-fold.

First, we are grouping similar application functionality together to provide for easier development, maintenance, reuse and readability.

Second, we are grouping application functionality such that external services (such as transactional support, or UI rendering) can be provided to specific parts of our code. Again, this makes development and maintenance easier, since (for example) our business logic code isn’t contaminated by the complexity of transactional processing during data access operations. Reducing the amount of external technology used within each layer reduces the surface area of the API that a developer in that layer needs to learn.

In many cases each layer will be a separate assembly, or even a separate technology. For instance, the data access layer may be in its own DLL. The data management layer may be the JET database engine.

Tiers

Tiers represent a physical deployment scenario for parts of an application. A tier is isolated from other tiers by a process or network boundary. Keeping in mind that cross-process and cross-network communication is expensive, we must always pay special attention to any communication between tiers to make sure that it is efficient given these constraints. I find it useful to view tier boundaries as barriers. Communication through the barriers is expensive.

Specifically, communication between tiers must be (relatively) infrequent, and coarse-grained. In other words, send few requests between tiers, and make sure each request does a relatively large amount of work on the other side of the process/network barrier.

Layers and Tiers

It is important to understand the relationship between layers and tiers.

Layers are deployed onto tiers. A layer does not span tiers. In other words, there is never a case where part of a layer runs on one tier and part of the layer runs on another tier. If you think you have such a case, then you have two layers – one running on each tier.

Due to the fact that layers are a discrete unit, we know that we can never have more tiers than layers. In other words, if we have n layers, then we have n or less tiers.

Note that thus far we have not specified that communication between layers must be efficient. Only communication between tiers is inherently expensive. Communication between layers could be very frequent and fine-grained.

However, notice also that tier boundaries are also layer boundaries. This means that some inter-layer communication does need to be designed to be infrequent and coarse-grained.

For all practical purposes we can only insert tiers between layers that have been designed for efficient communication. This means that it is not true that n layers can automatically be deployed on n tiers. In fact, the number of potential tiers is entirely dependant on the design of inter-layer communication.

This means we have to provide terminology for inter-layer communication:

Term	Meaning
Fine-grained	Communication between layers involves the use of properties, methods, events, delegates, data binding and so forth. In other words, there’s a lot of communication, and each call between layers only does a little work.
Coarse-grained	Communication between layers involves the use of a very few methods. Each method is designed to do a relatively large amount of work.

If we have n layers, we have n-1 layer interfaces. Of those interfaces, some number m will be course-grained. This means that we can have at most m+1 tiers.

In most applications, the layer interface between the presentation and business logic layers is fine-grained. Microsoft has provided us with powerful data binding capabilities that are very hard to give up. This means that m is virtually never n-1, but rather starts at n-2.

In most modern applications, we use SQL Server or Oracle for data management. The result is that the layer interface between data access and data management is typically course-grained (using stored procedures).

I recommend making the layer interface between the business logic and data access also be course-grained. This provides for flexibility in placement of these layers into different tiers so we can achieve different levels of performance, scalability, fault-tolerance and security as required.

In a web environment, the presentation is really just the browser, and the actual UI code runs on the web server. Note that this is, by definition, two tiers – and thus two or more layers. The interaction between web presentation and web UI is coarse-grained, so this works.

In the end, this means we have some clearly defined potential tier boundaries that map directly to the course-grained layer interfaces in our design. These include:

Presentation <-> UI (web only)
Business logic <-> Data access
Data access <-> Data management

Thus, for most web apps m is 3 and for most Windows apps m is 2. So we’re talking about n layers being spread (potentially) across 3 or 4 physical tiers.

Protocols and Hosts

Now that we have an idea how layers and tiers are related, let’s consider this from another angle. Remember that layers are not only logical groupings of domain functionality, but also are grouped by technological dependency. This means that, when possible, all code required database transactions will be in the same layer (and thus the same tier). Likewise, all code consuming data binding will be in the same layer, and so forth.

The net result of this is that a layer must be deployed somewhere that the technological dependencies of that layer can be satisfied. Conversely, it means that layers that have few dependencies have few hard restrictions on deployment.

Given that tiers are physical constructs (as opposed to the logical nature of layers), we can bind technological capabilities to tiers. What we’re doing in this case is defining a host for tiers, which in turn contain layers. In the final analysis, we’re defining host environments in which layers of our application can run.

We also know that we have communication between tiers, which is really communication between layers. Communication occurs over specific protocols that provide appropriate functionality to meet our communication requirements. The requirements between different layers of our application may vary based on functionality, performance, scalability, security and so forth. For the purposes of this article, the word protocol is a high-level concept, encompassing technologies like DCOM, Remoting, etc.

It is important to note that the concept of a host and a protocol are different but interrelated. They are interrelated because some of our technological host options put restrictions on the protocols available.

In .NET there are three categorical types of host: Enterprise Services, IIS or custom. All three hosts can accommodate ServicedComponents, and the IIS and custom hosts can accommodate Services Without Components (SWC).

The following table illustrates the relationships:

Host	Protocols	Technologies
Enterprise Services (Server Application)	DCOM	Simple .NET assembly ServicedComponent
IIS	Web services Remoting	Simple .NET assembly ServicedComponent SWC
Custom	Remoting Web services (w/ WSE) DCOM	Simple .NET assembly ServicedComponent SWC

The important thing to note here is that we can easily host ServicedComponent objects or Services Without Components in an IIS host, using Web services or Remoting as the communication protocol.

The all three hosts can host simple .NET assemblies. For IIS and Remoting this is a native capability. However, Enterprise Services can host normal .NET assemblies by having a ServicedComponent dynamically load .NET assemblies and invoke types in those assemblies. Using this technique it is possible to create a scenario where Enterprise Services can act as a generic host for .NET assemblies. I do this in my .NET Business Objects books for instance.

Hosts

What we’re left with is a choice of three hosts. If we choose Enterprise Services as the host then we’ve implicitly chosen DCOM as our protocol. If we choose IIS as a host we can use Web services or Remoting, and also choose to use or not use the features of Enterprise Services. If we choose a custom host we can choose Web services, Remoting or DCOM as a protocol, and again we can choose to use or not use Enterprise Services features.

Whether you need to use specific Enterprise Services features is a whole topic unto itself. I have written some articles on the topic, the most broad-reaching of which is this one.

However, there are some things to consider beyond specific features (like distributed transactions, pooled objects, etc.). Specifically, we need to consider broader host issues like stability, scalability and manageability.

Of the three hosts, Enterprise Services (COM+) is the oldest and most mature. It stands to reason that it is probably the most stable and reliable.

The next oldest host is IIS, which we know is highly scalable and manageable, since it is used to run a great many web sites, some of which are very high volume.

Finally there’s the custom host option. I generally recommend against this except in very specific situations, because writing and testing your own host is hard. Additionally, it is unlikely that you can match the reliability, stability and other attributes of Enterprise Services or IIS.

So do we choose Enterprise Services or IIS as a host? To some degree this depends on the protocol. Remember that Enterprise Services dictates DCOM as the protocol, which may or may not work for you.

Protocols

Our three primary protocols are DCOM, Web services and Remoting.

DCOM is the oldest, and offers some very nice security features. It is tightly integrated with Windows and with Enterprise Services and provides very good performance. By using Application Center Server you can implement server farms and get good scalability.

On the other hand, DCOM doesn’t go through firewalls or other complex networking environments well at all. Additionally, DCOM requires COM registration of the server components onto your client machines. Between the networking complexity and the deployment nightmares, DCOM is often very unattractive.

However, as with all technologies it is very important to weigh the pros of performance, security and integration against the cons of complexity and deployment.

Web services is the most hyped of the technologies, and the one getting the most attention by key product teams within Microsoft. If you cut through the hype, it is still an attractive technology due to the ongoing work to enhance the technology with new features and capabilities.

The upside to Web services is that it is an open standard, and so is particularly attractive for application integration. However, that openness has very little meaning between layers or tiers of a single application. So we need to examine Web services using other criteria.

Web services is not high performance, or low bandwidth.

Web services use the XmlSerializer to convert objects to/from XML, and that serializer is extremely limited in its capabilities. To pass complex .NET types through Web services you’ll need to manually use the BinaryFormatter and Base64 encode the byte stream. While achievable, it is a bit of a hack to do this.

However, by using WSE we can get good security and reliability features. Also Web services are strategic due to the focus on them by many vendors, most notably Microsoft.

Again, we need to evaluate the performance and feature limitations of Web services against the security, reliability and strategic direction of the technology. Additionally keeping in mind that hacks exist to overcome the worst of the feature limitations in the technology, allowing Web services to have similar functionality to DCOM or Remoting.

Finally we have Remoting. Remoting is a core .NET technology, and is very comparable to RMI in the Java space.

Remoting makes it very easy for us to pass complex .NET types across the network, either by reference (like DCOM) or by value. As such, it is the optimal choice if you want to easily interact with objects across the network in .NET.

On the other hand, Microsoft recommends against using Remoting across the network. Primarily this is because Remoting has no equivalent to WSE and so it is difficult to secure the communications channel. Additionally, because Microsoft’s focus is on Web services, Remoting is not getting a whole lot of new features going forward. Thus, it is not a long-term strategic technology.

Again, we need to evaluate this technology be weighing its superior feature set for today against its lack of long-term strategic value. Personally I consider the long-term risk manageable assuming you are employing intelligent application designs that shield you from potential protocol changes.

This last point is important in any case. Consider that DCOM is also not strategic, so using it must be done with care. Also consider that Web services will undergo major changes when Indigo comes out. Again, shielding your code from specific implementations is of critical importance.

In the end, if you do your job well, you’ll shield yourself from any of the three underlying protocols so you can more easily move to Indigo or something else in the future as needed. Thus, the long-term strategic detriment on DCOM and Remoting is minimized, as is the strategic strength of Web services.

So in the end what do you do? Choose intelligently.

For the vast majority of applications out there, I recommend against using physical tiers to start with. Use layers – gain maintainability and reuse. But don’t use tiers. Tiers are complex, expensive and slow. Just say no.

But if you must use physical tiers, then for the vast majority of low to medium volume applications I tend to recommend using Remoting in an IIS host (with the Http channel and BinaryFormatter), potentially using Enterprise Services features like distributed transactions if needed.

For high volume applications you are probably best off using DCOM with an Enterprise Services host – even if you use no Enterprise Services features. Why? Because this combination is more than two times older than Web Services or Remoting and its strengths and limitations and foibles are well understood.

Note that I am not recommending the use of Web services for cross-tier communication. Maybe I’ll change my view on this when Indigo comes out – assuming Indigo provides the features of Remoting with the performance of DCOM. But today it provides neither the features nor performance that make it compelling to me.

Distributed Object-Oriented systems, SOA and Java?

Wed, 07 Jul 2004 17:52:56 GMT

I rarely read magazines or articles - I just skim them to see if there's anything I actually do want to invest the time to read. This is useful, because it means I can get the gist of a lot of content and only dive into the bits that I find actually interesting. It also means that I can (and do) read a lot of Java related stuff as well as .NET related material.

If you've followed my blog or my books, you know that I have a keen interest in distributed object-oriented systems - which naturally has spilled over into this whole SOA/service fad that's going on at the moment. So I do tend to actually read stuff dealing with these topic areas in both the .NET and Java space.

Interestingly enough, there's no difference between the two spaces. Both the .NET and Java communities are entirely confused and both have lots of vendors trying to convince us that their definitions are correct, so we'll buy the “right” products to solve our theoretical problems. What a mess.

Ted Neward has been engaged in some interesting discussions about the message-based or object-based nature of services. Now Daniel F. Savarese has joined the fray as well. And all this in the Java space, where they claim to be more mature and advanced than us lowly .NET guys... I guess not, since their discussions are identical to those happening in the .NET space.

I think the key point here is that the distributed OO and SOA issues and confusion totally transcend any technical differences between .NET and Java. What I find unfortunate is that most of the discussions on these topics are occuring within artificial technology silos. Too few discussions occur across the .NET/Java boundary.

When it comes to SOA and distributed object-oriented systems (which are two different things!), every single concept, issue, problem and solution that applies to .NET applies equally to Java and visa versa. Sure, the vendor implementations vary, but that's not where the problem (or the solution) is found. The problems and solutions are more abstract and are common across platforms.

What's needed is an intelligent, honest and productive discussion of these issues that is inclusive of people from all the key technology backgrounds. I imagine that will happen about the same time that we have any sort of intelligent, honest or productive discussion between Republicans and Democrats in the US government...

Middle-tier Hosting: Remoting vs. Enterprise Services

Thu, 24 Jun 2004 22:38:44 GMT

I got this question via email. I get variations on this question a lot, so I thought I’d blog my answer.

Hope you don't mind imposing on you for a second. I actually spoke to you very briefly after the one of the sessions and you seemed to concur with me that for my scenario - which is a typical 3-tier scenario, all residing on separate machines, both internal and external clients - hosting my business components on IIS using HTTP/binary was a sensible direction. I've recently had a conversation with someone suggesting that Enterprise Services was a far better platform to pursue. His main point in saying this is the increased productivity - leveraging all the services offered there (transactions, security, etc.). And not only that, but that ES is the best migration path for Indigo, which I am very interested in. This is contrary to what I have read in the past, which has always been that ES involves interop, meaning slower (which this person also disputes, by the way), and Don Box's explicit recommendation that Web Services were the best migration path. I just thought I'd ask your indulgence for a moment to get your impressions. Using DCOM is a little scary, we've had issues with it in the past with load-balancing etc. Just wondering if you think this is a crazy suggestion or not, and if not, do you know of any good best-practices examples or any good resources.

Here are some thoughts from a previous blog post.

The reason people are recommending against remoting is because the way you extend remoting (creating remoting sinks, custom formatters, etc) will change with Indigo in a couple years. If you aren't writing that low level type of code then remoting isn't a problem.

Indigo subsumes the functionality of both web services and remoting. Using either technology will get you to Indigo when it arrives. Again, assuming you aren't writing low-level plug-ins like custom sinks.

Enterprise Services (ES) provides a declarative, attribute-based programming model. And this is good. Microsoft continues to extend and enhance the attribute-based models, which is good. People should adopt them where appropriate.

That isn't to say, however, that all code should run in ES. That's extrapolating the concept beyond its breaking point. Just because ES is declarative, doesn’t make ES the be-all and end-all for all programming everywhere.

It is true that ES by itself causes a huge performance hit - in theory. In reality, that perf hit is lost in the noise of other things within a typical app (like network communication or the use of XML in any way). However, specific services of ES may have larger perf hits. Distributed transactions, for instance, have a very large perf hit - which is essentially independent of any interop issues lurking in ES. That perf hit is just due to the high cost of 2-phase transactions. Here's some performance info from MSDN supporting these statements.

The short answer is to use the right technology for the right thing.

If you need to interact between tiers that are inside your application but across the network, then use remoting. Just avoid creating custom sinks or formatters (which most people don't do, so this is typically a non-issue).
If you need to communicate between applications (even .NET apps) then use web services. Note this is not between tiers, but between applications – as in SOA.
If you need ES, then use it. This article may help you decide if you need any ES features in your app. The thing is, if you do use ES, your client still needs to talk to the server-side objects. In most cases remoting is the simplest and fastest technology for this purpose. This article shows how to pull ES and remoting together.

Note that none of the above options use DCOM. The only case I can see for DCOM is where you want the channel-level security features it provides. However, WSE is providing those now for web services, so even there, I'm not sure DCOM is worth all the headaches. Then the only scenario is where you need stateful server-side objects and channel-level security, because DCOM is the only technology that really provides both features.

SOA, CSLA .NET and trust boundaries

Rockford Lhotka — Sat, 12 Jun 2004 00:25:44 GMT

I recently had a reader of my business objects book ask about combining CSLA .NET and SOA concepts into a single application.

Certainly there are very valid ways of implementing services and client applications using CSLA such that the result fits nicely into the SOA world. For instance, in Chapter 10 I specifically demonstrate how to create services based on CSLA-style business objects, and in some of my VS Live presentations I discuss how consumption of SOA-style services fits directly into the CSLA architecture within the DataPortal_xyz methods (instead of using ADO.NET).

But the reader put forth the following, which I wanted to discuss in more detail. The text in italics is the reader’s view of Microsoft’s “standard” SOA architecture. My response follows in normal text:

1. Here is the simplified MS’s “standard” layering,

(a) UI Components

(b) UI Process Components [UI never talk to BSI/BC directly, must via UIP]

(c) Service Interfaces

(d) Business Components [UIP never talk to D directly, must via BC]

(e) Business Entities

(f) Data Access Logic Components

I’m aware of its “transaction script” and “anemic domain model” tendency, but it is perhaps easier for integration and using tools.

To be clear, I think this portrayal of ‘SOA’ is misleading and dangerous. SOA describes the interactions between separate applications, while this would lead one into thinking that SOA is used inside an application between tiers. I believe that is entirely wrong – and if you pay attention to what noted experts like Pat Helland are saying, you’ll see that this approach is invalid.

SOA describes a loosely-coupled, message-based set of interactions between applications or services. But in this context, the word ‘service’ is synonymous with ‘application’, since services must stand alone and be entirely self-sufficient.

Under no circumstance can you view a ‘tier’ as a ‘service’ in the context of SOA. A tier is part of an application, and implements part of the application’s overall functionality. Tiers are not designed to stand alone or be used by arbitrary clients – they are designed for use by the rest of their application. I discussed this earlier in my post on trust boundaries, and how services inherently create new trust boundaries by their very nature.

So in my view you are describing at least two separate applications. You have an application with a GUI or web UI (a and b or ab), and an application with an XML interface (c, d, e and f or cdef).

To fit into the SOA model, each of these applications (ab and cdef) is separate and must stand alone. They are not tiers in a single application, but rather are two separate applications that interact. The XML interface from cdef must be designed to service not only the ab application, but any other applications that need its functionality. That’s the whole point of an SOA-based model – to make this type of functionality broadly available and reusable.

Likewise, application ab should be viewed as a full-blown application. The design of ab shouldn’t necessarily eschew business logic – especially validation, but likely also including some calculations or other data manipulation. After all, it is the job of the designer of ab to provide the user with a satisfactory experience based on their requirements – which is similar, but not the same, as the set of requirements for cdef.

Yes, this does mean that you’ll have duplicate logic in ab and cdef. That’s one of the side-effects of SOA. If you don’t like it, don’t use SOA.

But there’s nothing to stop you from using CSLA .NET as a model to create either ab or cdef or both.

To create ab with CSLA .NET, you’d design a set of business objects (with as little or much logic as you felt was required). Since ab doesn’t have a data store of its own, but rather uses cdef as a data store, the DataPortal_xyz methods in the business objects would call the services exposed by cdef to retrieve and update data as needed.

To create cdef with CSLA .NET, you’d follow the basic concepts I discuss in Chapter 10 in my VB.NET and C# Business Objects books. The basic concept is that you create a set of XML web services as an interface that provides the outside world with data and/or functionality. The functionality and data is provided from a set of CSLA .NET objects, which are used by the web services themselves.

Note that the CSLA-style business objects are not directly exposed to the outside world – there is a formal interface definition for the web services which is separate from the CSLA-style business objects themselves. This provides separation of interface (the web services) from implementation (the CSLA-style business objects).

Remoting, web services, SOA and trust boundaries

Fri, 09 Apr 2004 01:13:51 GMT

Totally unrelated to my previous blog entry, I received an email asking me about the ‘future of remoting’ as it relates to web services and SOA.

It seems that the sender of the email got some advice from Microsoft indicating that the default choice for all network communication should be web services, and that instead of using conventional tiers in applications, that we should all be using SOA concepts instead. Failing that, we should use Enterprise Services (and thus presumably DCOM (shudder!)), and should only use remoting as a last resort.

Not surprisingly, I disagree. My answer follows…

Frankly, I flat out disagree with what you've been told. Any advice that starts with "default choices" is silly. There is no such thing - our environments are too complex for such simplistic views on architecture.

Unless you are willing to rearchitect your system to be a network of linked _applications_ rather than a set of tiers, you have no business applying SOA concepts. Applying SOA is a big choice, because there is a very high cost to its application (in terms of complexity and performance).

SOA is very, very cool, and is a very good thing when appropriately applied. Attempting to change classic client/server scenarios into SOA scenarios is typically the wrong approach.

While SOA is a much bigger thing than web services, it turns out that web services are mostly only useful for SOA models. This is because they employ an open standard, so exposing your 'middle tier' as a set of web services really means you have exposed the 'middle tier' to any app on your network. This forces the 'middle tier' to become an application (because you have created a trust boundary that didn't previously exist). As soon as you employ web services you have created a separate application, not a tier. This is a HUGE architectural step, and basically forces you to employ SOA concepts - even if you would prefer to use simpler and faster client/server concepts.

Remoting offers superior features and performance when compared to web services. It is the most appropriate client/server technology available in .NET today. If you aren't crossing trust boundaries (and don't want to artificially impose them with web services), then remoting is the correct technology to use.

Indigo supports the _concepts_ of remoting (namely the ability to pass .NET types with full fidelity, and the ability to have connection-based objects like remoting's activated objects). Indigo has to support these concepts, because it is supposed to be the future of both the open web service technology and the higher performance client/server technology for .NET.

The point I'm making, is that using remoting today is good if you are staying within trust boundaries. It doesn't preclude moving to Indigo in the future, because Indigo will also provide solutions for communication within trust boundaries, with full .NET type fidelity (and other cool features).

Web services force the use of SOA, even if it isn't appropriate. They should be approached with caution and forethought. Specifically, you should give serious thought to where you want or need to have trust boundaries in your enterprise and in your application design. Trust boundaries have a high cost, so inserting them between what should have been tiers of an application is typically a very, very bad idea.

On the other hand, using remoting between applications is an equally bad idea. In such cases you should always look at SOA as a philosophy, and potentially web services as a technical solution (though frankly, today the better SOA technology is often queuing (MQ Series or MSMQ), rather than web services).