MCsla is a prototype that takes SQL Server Modeling (aka “Oslo” and “M”) and CSLA .NET, using them to create an end-to-end story. This story goes like this:

1. Developer creates a business database for business data
2. Developer writes concise DSL code
3. Developer “compiles” DSL code into the SQL Server Modeling repository
4. End user executes a runtime application, which dynamically creates a UI, business layer and data layer that uses the business database – all this based on the compiled DSL metadata stored in the repository

This saves the developer from a lot of work. In fact the developer writes perhaps 5% of the code they would have written to create the UI, business layer and data layer by hand.

See my discussion of the concepts and prototype in a three part video series:

Exploring SQL Server Modeling through MCsla

You can see the MCsla code in the CSLA .NET repository (web view) or grab it using any svn client from svn://svn.lhotka.net/csla/mcsla/tags/Video1001.

I’m heading to Atlanta to talk about “Microsoft Oslo” – oh wait – to talk about the SQL Server Modeling at ALEMUG on Wednesday, December 2.

Actually I’ll be talking about MCsla, which is the prototype domain specific language and runtime I created for CSLA .NET using the SQL Server Modeling tools. It is a good topic, because I get to talk about designing and defining a DSL with a custom grammar, using the m.exe and mx.exe tools to compile DSL code into instance data, and then defining a schema so that data can be loaded into SQL Server tables. Tables that you can view (and edit) using “Quadrant”, or any other SQL Server tools of your choice.

In short, nearly every part of SQL Server Modeling gets used, and the DSL code actually executes as a WPF application – so it is an end-to-end prototype solution.

So if you are in the Atlanta area, maybe I’ll see you there!

I’ve updated my prototype MCsla project to work on the “Olso” May CTP. The update took some effort, because there are several subtle changes in the syntax of “Oslo” grammars and instance data. What complicated this a little, is that I am using a custom DSL compiler because the standard mgx.exe utility can’t handle my grammar.

Still, I spent less than 8 hours getting my grammar, schemas, compiler and runtime fixed up and working with the CTP (thanks to some help from the “Oslo” team).

I chose at this point, to put the MCsla project into my public code repository. You can use the web view to see the various code elements if you are interested.

The prototype has limited scope – it supports only the CSLA .NET editable root stereotype, which means it can be used to create simple CRUD screens over single records of data. But even that is pretty cool I think, because it illustrates the end-to-end flow of the whole “Oslo” platform concept.

A business developer writes DSL code like this:

Object Product in Test
{
  Public ReadOnly int Id;
  Public string Name;
  Public double ListPrice;
} Identity Id;

(this is the simplest form – the DSL grammar also allows per-type and per-property authorization rules, along with per-property business and validation rules)

Then they run a batch file to compile this code and insert the resulting metadata into the “Oslo” repository.

The user runs the MCslaRuntime WPF application, which reads the metadata from the repository and dynamically creates a XAML UI, CSLA .NET business object and related data access object that talks to a SQL Server database.

The basic functionality you get automatically from CSLA .NET is all used by the runtime. This includes application of authorization, business and validation rules, automatic enable/disable for the Save/Cancel buttons based on the business object’s rules and so forth.

If the business developer “recompiles” their DSL code, the new metadata goes into the repository. The user can click a Refresh App button to reload the metadata, immediately enjoying the new or changed functionality provided by the business developer.

The point is that the business developer writes that tiny bit of DSL code instead of pages of XAML and C#. If you calculate the difference in terms of lines of code, the business developer writes perhaps 5% of the code they’d have written by hand. That 95% savings in effort is what makes me so interested in the overall “Oslo” platform story!

In working on MCsla I have learned quite a few things. Obviously I had to learn the ins and outs of MGrammar, MSchema, MGraph and the various related command line tools. But that’s not really the interesting part, because that sort of knowledge is specific to a bunch of unreleased tools and so is transitory.

What is more interesting is learning about the process of building a runtime that works off metadata. Which leads to learning about what metadata is needed for a runtime. Which leads to learning about how you get that metadata – where it comes from.

In my simplistic worldview I thought I’d create a single DSL to create metadata for my runtime. And this is what I’ve done in my MCsla prototype, but it really isn’t sufficient for any long-term real effort.

Why?

Because my DSL describes business objects. It doesn’t describe details about the UI, or about the data access layer (DAL) or any data mappings into the database. In my prototype I am inferring enough information from the metadata to create a basic UI, a limited DAL and some data mappings, but this is only useful for a prototype.

The conclusion I’ve come to is that any real CSLA .NET application runtime will require a number of related metadata schemes:

1. Business object description
2. UI description
3. DAL description
4. Data mapping description

If you create a single DSL to cover all these at once I think you’ll be in trouble. The trouble will come in at least two forms. First, this single DSL will almost certainly approach the complexity of C# or other 3GLs, and at that point why use a DSL at all? Second, such a DSL would almost certainly break down walls around separation of concerns. Let me talk about this second point further.

One of the primary strengths of CSLA .NET is that it encourages clear separation of concerns. The business layer supports numerous UI layers at one time – Web Forms, WPF, ASP .NET MVC, Silverlight, Windows Forms, WCF services, Workflows, etc. Similarly, the business layer doesn’t know or care how or where the DAL gets the data, as long as the DAL can provide create/read/insert/update/delete operations.

It is a virtual certainty that you’ll need a different DSL for ADO.NET EF than for raw ADO.NET (connection/command/datareader) data access layers. Similarly, you’ll probably need a different DSL to describe a web UI, as opposed to a XAML-based UI. The technology semantics are so different that it is hard to imagine a single DSL that encompasses both UI styles (at least one that is rich enough to make the end users happy).

As a result, I suspect we need a different DSL for each technology of each layer of the application architecture. And that’s OK, because it is a good bet that Microsoft will create a DSL for ADO.NET EF (as an example), so we don’t need to. That’ll probably cover at least item number 4 in my earlier list.

The role of the “DAL description DSL” is probably to map the items in the business DSL to the items in the data mapping DSL.

Already it becomes clear that some DSLs (like any DSL for ADO.NET EF, or for a framework like MCsla) will exist in their own world, for their own narrow domain. And that’s fine, because that is the point of a DSL.

And at the same time, it becomes clear that we’ll need to create cross-DSL DSLs to bridge the gaps between different DSLs. So that hypothetical “DAL description DSL” would be specific to both MCsla and the data mapping DSL.

To put it another way, it seems to me that we’re looking at two types of DSL. “Real” domain languages, and “glue” domain languages. Domain languages that target a specific domain, and domain languages that target the gap between other domain languages.

I suspect the end result, is that to create a comprehensive runtime, the runtime will need to pull in metadata for each layer of the application, and for the description on how the layers interact. And a developer will need to learn 4 or more DSLs to describe the various layers of the application. And tooling will need to evolve to validate the various inter-connected bits of DSL code so we aren’t left catching mis-matched bits of DSL input at runtime.

To summarize: this DSL/runtime stuff is very cool, but it seems to me that it will take years of maturation before everything comes together in a really productive manner.

One caveat: clearly you could limit the flexibility of the architecture/runtime and simplify the problem space and thus this multi-DSL issue. My fear though, is then you are no different from CASE and many other similar failed concepts of the past – so that seems rather uninteresting to me.

One of the issues I’ve been struggling with in building MCsla is that the mgx tool that is used to compile a DSL into M doesn’t output valid M in all cases.

A couple of the issues are simply unsolvable (I mentioned these in my last blog post as well).

1. If your grammar includes any Boolean values, they come out as ‘ true’ and ‘false’ (including the single quotes), when they should come out as true and false (no quotes).
2. If your grammar includes any lists of values, a single-item list generates invalid M, while multi-item lists generate valid M (though I’m not sure you can describe the result using an MSchema – but at least the result is “valid”).

As I say, these problems are “unsolvable” because the M rendering API is what creates the bad result, and that’s a black box.

The solution, such as it is, requires taking the text output, the M code, and doing some text processing to fix the issues. In the olden days I’d have whipped up an awk script to do this, but these days C# is the tool (even if it is a lot more verbose). Here’s the code I’m using to fix the two issues – kind of a post-processor to the mgx tool:

public static class MFix
{
  /// <summary>
  /// Fix single-quote and single-list-item
  /// issues with a generated M file.
  /// </summary>
  /// <param name="fileName">Path to the file to fix</param>
  public static void FixFile(string fileName)
  {
    var output = new System.Text.StringBuilder();
    File.ReadAllLines(fileName).ToList().ForEach((line) =>
    {
      output.AppendLine(FixQuotes(FixSingleNode(line)));
    });
    File.WriteAllText(fileName, output.ToString());
  }

  /// <summary>
  /// Fix single-quote issue with M generated by
  /// mgx.exe.
  /// </summary>
  /// <param name="input">One line of text input.</param>
  /// <returns>Fixed line of text from input.</returns>
  private static string FixQuotes(string input)
  {
    var sb = new System.Text.StringBuilder(input.Length);
    int state = 0;
    for (int i = 0; i < input.Length; i++)
    {
      var sub = input.Substring(i, 1);
      if (sub == "\"")
      {
        if (state == 0)
          state = 1;
        else
          state = 0;
        sb.Append(sub);
      }
      else if (sub == "\\" && state == 1)
      {
        sb.Append(sub);
        if (i < input.Length - 1)
        {
          i++;
          sb.Append(input.Substring(i, 1));
        }
      }
      else if (sub == "'" && state == 0)
      {
        // do nothing and eat next char if it is a ' '
        i++;
        if (input.Substring(i, 1) != " ")
          i--;
      }
      else
        sb.Append(sub);
    }
    return sb.ToString();
  }

  /// <summary>
  /// Fix single element array issue with M generated by
  /// mgx.exe.
  /// </summary>
  /// <param name="input">One line of text input.</param>
  /// <returns>Fixed line of text from input.</returns>
  private static string FixSingleNode(string input)
  {
    var trm = input.TrimStart();
    if (!trm.StartsWith("=")) return input;

    var pos = input.IndexOf("=");
    var sb = new System.Text.StringBuilder(input.Length);

    sb.Append(input.Substring(0, pos));
    sb.Append(" {");
    sb.Append(input.Substring(pos + 1, input.Length - pos - 1));
    sb.Append(" } ");

    return sb.ToString();
  }
}

It is kind of ironic that I had to write a subset of an M parser to fix issues with the M generator API.

Working with Microsoft Oslo, the goal is to enable the creation of a domain specific language (DSL) using MGrammar, so a developer can write code in that grammar, compile the code into M and then load that M into the Oslo repository.

This is turning out to be a bigger challenge that you’d expect, though I think that’s largely because I’m dealing with such early builds of all the Oslo tools.

The tool chain for compiling DSL source into M is this:

1. Use mg.exe to compile the MGrammar
2. Use mgx.exe to compile the DSL source against the grammar to create M
3. Use m.exe to compile the M
4. Use mx.exe to load the compiled M (an mx file) into the repository

Unfortunately this breaks down at step 2, because mgx.exe has some issues.

The first issue I encountered was that mgx creates hierarchical M, not relational M. What’s the difference?

Relational M is defined by an MSchema where child entities define foreign key relationships to their parent. This is the normal model you’d find when creating any relational database.

Hierarchical M is defined by an MSchema where parent entities define relationships with their children. This is more like what you’d find in XML, and doesn’t completely map to a normal relational model. While it is true that this type of MSchema can result in valid SQL table definitions, you can’t define all the relationship restrictions you’d normally want to have in your database.

The thing is, M code itself must match the shape of the MSchema you want to use. So if you define your entities using a relational MSchema, the M must also be relational. Conversely, this means that when mgx generations hierarchical M, you must use a hierarchical MSchema to map that M metadata into your database (or the Oslo repository).

This is an unfortunate limitation, that hopefully will be resolved at some point.

The second and third issues I encountered appear to be bugs in mgx. But not just in mgx – they are bugs in the underlying API, so even if you try to write your own “mgx replacement” (which is what I’m doing btw), the API that helps you generate M will create invalid M output.

The two bugs are:

1. When a node has a Boolean value, true comes out as ‘ true’ and false comes out as ‘ false’. The single quotes and space after the first single quote should not be there, but that’s what gets output. Oddly enough, Int32 and other primitive types work fine – just Boolean types seem to be broken.
2. When a list of items is output, and that list contains one item, the output is invalid M. Multiple items or no items in the list work fine – just the single-item case is broken.

The only solution I’ve found for these bugs is to do a post-processing step on the resulting M – literally doing text processing to identify and fix the issues. Again, hopefully these issues get resolved in the API itself, so valid M gets produced by mgx or any other tool using the API.

The fourth issue, and the reason I’m writing my own mgx replacement, is that my MCsla grammar allows arbitrary placement of various elements, like authorization and business rules within a property:

Public string Name {
  Allow Write (“Admin”, “Supervisor”);
  Rule StringRequired;
  Deny Read (“Guest”);
}

Because the grammar allows arbitrary placement, the resulting M is a collection of mixed types of entity. In other words, the compiled M is effectively invalid, because the M parser can’t figure out whether the collection contains auth rules, or business rules (because it contains both).

I really have two choices – either change my grammar to put auth rules and business rules in different block structures, or at least to require one to come before the other. Or write my own mgx replacement to rearrange the nodes of the compiled DSL code into discrete collections of homogeneous types.

Since I don’t want to “ruin” my pretty grammar, I’m trying to fix the nodes by creating my own mgx replacement. And this seems quite realistic, though it doesn’t address the first three issues in any way – those are still problematic.

I’m working on a number of things at the moment. Most notably the CSLA .NET for Silverlight video series that will be available in the very near future (it turns out that creating a video series is a lot of work!). And of course I’m prepping for numerous upcoming speaking engagements, and I’d love to see you at them – please come up and say hi if you get a chance.

But I’m also working on some content around Microsoft Oslo, MGrammar and related concepts. To do this, I’m creating a prototype “MCsla” language (DSL) that allows the creation of CSLA .NET business objects (and more) with very concise syntax. I’ll probably be blogging about this a bit over the next couple weeks, but the goal is to end up with some in-depth content that really walks through everything in detail.

My goal is for the prototype to handle CSLA editable root objects, but covers business rules, validation rules and per-type and per-property authorization rules. Here’s a conceptual example:

Object foo
{
    // per-type authz
    Allow Create ("Admin");
    Allow Edit ("Admin", "Clerk" );

    // property definitions

    Public ReadOnly int Id;

    Public Integer Value {
       // per-property authz
       Allow Write ("Clerk" );
       // business/validation rules
       Rule MinValue (1 );
       Rule MaxValue (50 );
    }
}

But what I’ve realized as I’ve gotten further into this, is that I made a tactical error.

A lot of people, including myself, have been viewing MGrammar as a way to create a DSL that is really a front-end to a code generator. My friend Justin Chase has done a lot of very cool work in this space, and he’s not alone.

But it turns out that if you want to really leverage Microsoft Oslo and not just MGrammar, then this is not about creating a code generator. And what I’m finding is that starting with the DSL is a terrible mistake!

In fact, the idea behind MOslo is that the DSL is just a way to get metadata into the Oslo repository. And you can use other techniques to get metadata into the repository as well, including the graphical “Quadrant” tool.

But my next question, and I’m guessing yours too, is that if all we do is put metadata into the repository, what good is that???

This is where a runtime comes into play. A runtime is a program that reads the metadata from the repository and basically executes the metadata.

I always had a mental picture of MOslo “projecting” the metadata into the runtime. But that’s not accurate. It is the runtime that “pulls” metadata from the repository and then uses it.

And that’s OK. The point is that the runtime is this application that interprets/compiles/uses/consumes the metadata to end up with something that actually does some work.

What I’m learning through this process, is that the DSL exists to service the metadata in the repository. But it is the runtime that defines the metadata. The runtime consumes the metadata, so the metadata exists to serve the needs of the runtime.

In other words, I should have started with the runtime first so I knew what metadata was required, so I could design a DSL that was capable of expressing and capturing that metadata.

The example MCsla code you see above is OK, but it turns out that it is missing some important bits of information that my runtime needs to function. So while the DSL expresses the important CSLA .NET concepts, it doesn’t express everything necessary to generate a useful runtime result…

So at the moment I’ve stopped working on the DSL, and am focusing on creating a working runtime. One that can execute the metadata in a way that the user gets a dynamically created UI (in WPF) that is bound to a dynamically created CSLA .NET business object, that leverages a dynamically created data access layer (using the CSLA .NET 3.6 ObjectFactory concept). I’m not dynamically creating the database, because I think that’s unrealistic in any real-world scenario. We all have pre-existing databases after all.

Once I get the runtime working (and it is close – here’s a screenshot of a simple object, showing how the dynamic WPF UI is running against an object with business rules, buttons bound to the CslaDataProvider through commanding and all the other nice CSLA features.

Not that my UI is a work of art, but still :)

I have a lot more to do, but it is now clear that starting with the runtime makes a lot more sense than starting with the DSL.