Rockford Lhotka

 Wednesday, December 12, 2018

CSLA .NET 4.9.0 includes some exciting new features, primarily focused on server-side capabilities for containers, .NET Core, and ASP.NET Core. There are a number of data portal enhancements, as well as powerful new configuration options based around the .NET Core configuration subsystem.

Data Portal Enhancements

Many of the enhancements are focused on enabling powerful cloud/container based scenarios with the data portal. Most notably, the data portal now supports two different types of routing to enable the use of multiple server instances. It also has the option to track recent activity with the intent of supporting a basic health dashboard, the ability to force a client "offline", and support for the .NET Core IoC/DI model.

Client-side data portal routing

One common request is that it would be nice if a subset of data portal requests could be routed to some server endpoint other than the default. This idea supports a number of scenarios, including security, logging, multi-tenant server farms, and I'm sure many others.

When you configure the client-side data portal on a web server, mobile device, PC, Mac, or Linux desktop you can now provide mappings so specific business domain types (root object types) cause the data portal to call server-side endpoints other than the default. This can be done on a per-type basis, or by applying the DataPortalServerResource attribute to a root domain type.

As the client app starts up, the data portal must be configured with mappings from domain types or DataPortalServerResource attribute values to specific server endpoint definitions. For example:

      // set up default data portal for most types
      ApplicationContext.DataPortalProxy = typeof(Csla.DataPortalClient.HttpProxy).AssemblyQualifiedName;
      ApplicationContext.DataPortalUrlString = "";

      // add mapping for DataPortalServerResource attribute
        new Csla.DataPortalClient.DataPortalProxyDescriptor
        { ProxyTypeName = typeof(Csla.DataPortalClient.HttpProxy).AssemblyQualifiedName, DataPortalUrl = "" });

      // add mapping for specific business type
        new Csla.DataPortalClient.DataPortalProxyDescriptor
        { ProxyTypeName = typeof(Csla.DataPortalClient.HttpProxy).AssemblyQualifiedName, DataPortalUrl = "" });

Notice that the server-side data portal endpoint is defined not just by a URL, but also by the proxy type, so you can use different network transport technologies to access different data portal servers.

Server-side data portal routing

One of the most common issues we all face when hosting server-side endpoints is dealing with client-side app versioning and having server endpoints that support clients running older software. This is particularly important when deploying mobile apps through the Apple/Google/Microsoft stores, when you can't directly control the pace of rollout for client apps.

Additionally, in a clustered server-side environment such as Kubernetes or Cloud Foundry it is likely that you'll want to organize your containers based on various characteristics, such as CPU consumption, memory requirements, or the need for specialized hardware.

The HTTP data portal (Csla.Server.Hosts.HttpPortalController) now supports a solution for both scenarios via server-side data portal routing. It is now possible to set up a public gateway server that hosts a data portal endpoint, which in turn routes all calls to other data portal endpoints within your server environment. Typically these other endpoints are not public, forcing all client apps to route all data portal requests through that gateway router endpoint.

Note that using technologies like Kubernetes it is entirely realistic for the gateway router to be composed of multiple container instances to provide scaling and fault tolerance.

To minimize overhead, the data portal router does not deserialize the inbound data stream from the client. Instead, it uses a header value to determine how to route the request to a worker node where the data stream is deserialized and processed. That header value is created by the client-side data portal by combining two concepts.

First is an optional application version value. If this value is set by the client-side app it is then passed through to the server-side data portal where the value is used to route requests to a worker node running a corresponding version of the server-side app. This allows you to leave versionA nodes running while also running versionB nodes, and once all client devices have migrated off versionA then you can shut down those versionA nodes with no disruption to your users.

Second is a DataPortalServerRoutingTag attribute you can apply to root domain types. This attribute provides a routing tag value that is also passed to the server-side data portal. The intent of this attribute is that you can specify that certain root domain types have characteristics that should be used for server-side routing. For example, you might tag domain types that you know will require a lot of CPU or memory on the server so they are routed to Kubernetes pods that are hosted on fast or large physical servers (Kubernetes nodes), while all other requests go to pods running on normal sized Kubernetes nodes.

Another example would be where the server-side processing requires access to specialized hardware. In that case your Kubernetes nodes would have a taint indicating that they have that hardware, and the DataPortalServerRoutingTag is used to tell the data portal router to send requests for that root domain type only to server-side data portal instances running on those nodes.

What makes this data portal feature a little complex is that the data portal router uses both the version and routing tag to route calls. This is obviously required to support both scenarios at the same time, but it does mean you need to define your routes such that they consider both version and routing tag elements.

The client-side data portal passes a routing tag to the server with the following format:

  • null - neither routing tag nor version are specified
  • routingTag-versionTag - both routing tag and version are specified
  • routingTag- - a routing tag is specified, but no version
  • -versionTag - a version is specified, but no routing tag

Your server-side data portal gateway router is configured on startup with code like this (in the data portal controller class):

  public class DataPortalController : HttpPortalController
    public DataPortalController()
      RoutingTagUrls.Add("routingTag-versionTag", "https://serviceName:36123/api/DataPortal");

You can specify as many routing tag/version keys as necessary to handle routing to the data portal endpoints running in your server environment.

If a key can't be found, or if the target URL is localhost then the request will be processed directly on the data portal router instance. Of course you may choose not to deploy any of your business DLLs to the router instance, in which case all such requests will fail, returning an exception to the client.

Offline mode

A common feature request, especially for mobile devices and laptops, is for the application to force itself into "offline mode" where all data portal calls are directed only to the local data portal. The client-side data portal now supports this via the Csla.ApplicationContext.IsOffline property.

If you set IsOffline to true the client-side data portal will immediately route all data portal requests to the local app rather than a remote server. It is as if all your data portal methods had the RunLocal attribute applied.

Clearly you need to build your DataPortal_XYZ methods to behave properly when they are running on the client (Csla.ApplicationContext.ExecutionLocation == ExecutionLocations.Client) vs the server (Csla.ApplicationContext.ExecutionLocation == ExecutionLocations.Server). Typically you'd have your code delegate to a DAL provider specific to the client or server based on execution location using the encapsulated invocation model as described in Using CSLA 4: Data Access.

Save and Merge

CSLA has included the GraphMerger type for some time now. This type allows you to merge the results of a SaveAsync call back into the existing domain object graph. The BusinessBase and BusinessListBase types now implement a simpler SaveAndMergeAsync method you can use to save a root domain object such that the results are automatically merged into the object graph.

DataPortalFactory in ASP.NET Core dependency injection

ASP.NET Core includes a dependency injection model based on IServiceCollection, with dependencies defined in the ConfigureServices method of the Startup.cs file. CSLA .NET now supports that scenario via an AddCsla extension method for IServiceCollection.

    public void ConfigureServices(IServiceCollection services)
      // ...

The only service that is defined is a new IDataPortalFactory, implemented by DataPortalFactory. The factory is a singleton, and is used to create instances of the data portal for use in a page. For example, in a Razor Page you might have code like this:

  public class EditModel : PageModel
    private IDataPortal<Customer> dataPortal;

    public EditModel(IDataPortalFactory factory)
      dataPortal = factory.GetPortal<Customer>();
    // ...

Configuration Enhancements

An important design pattern for success in both DevOps and container-based server deployment is to store config in the environment. The .NET Core configuration subsystem supports this pattern, and now CSLA .NET does as well. Not only that, but we now support an optional fluent configuration model if you want to set configuration in code (common in Xamarin apps for example), and integration with the .NET Core and ASP.NET Core configuration models.

.NET Core configuration subsystem integration

The .NET Core configuration subsystem supports modern configuration concepts, while still supporting legacy concepts like config files. It is also extensible, allowing providers to load configuration values from many different sources.

The base type used in .NET Core configuration is a ConfigurationBuilder. An instance of this type is provided to you by ASP.NET Core, or you can create your own instance for use in console apps. CSLA .NET provides an extension method to integrate into this configuration model. For example, in a console app you might write code like this:

  var config = new ConfigurationBuilder()

In ASP.NET Core you'd simply use the ConfigurationBuilder instance already avalable to invoke the ConfigureCsla method.

The result of the ConfigureCsla method is that the configuration values are loaded for use by CSLA .NET based on any configuration sources defined by the ConfigurationBuilder. In many cases that includes a config file, environment values, and command line parameters. It might also include secrets loaded from Azure, Docker, or whatever environment is hosting your code. That's all outside the scope of CSLA itself - those are features of .NET Core. The point is that the ConfigureCsla method takes the results of the Build method and maps the config values for use by CSLA .NET.

Note that nothing we've done here has any impact on .NET Framework code. All your existing .NET 4.x code will continue to function against web.config/app.config files.

Fluent configuration API

Although it is best to follow the 12 Factor guidance around config, there are times when you just need to set config values through code. This is particularly true in Xamarin and WebAssembly scenarios, where the configuration systems are weak or non-existant.

In the past you've been able to directly set propery values on Csla.ApplicationContext, various data portal types, and elsewhere in CSLA. That's confusing and inconsistent, and is a source of bugs in many people's code.

There's a new fluent API for configuration you can use when you need to set configuration through code. The entry point for this API is Csla.Configuration.CslaConfiguration, which implements Csla.Configuration.ICslaConfiguration. The model uses extension methods for sub-configuration, extending the ICslaConfiguration type.

Basic configuration is done like this:

      new Csla.Configuration.CslaConfiguration()

Data portal configuration can be chained like this:

      new Csla.Configuration.CslaConfiguration()
        .DataPortal().DefaultProxy(typeof(HttpProxy), "")

Client-side data portal descriptors for client-side data portal routing (see above) can be configured like this:

      new CslaConfiguration()
        .DataPortal().ProxyDescriptors(new List<Tuple<string, string, string>>

Configuration of Csla.Data options:

      new CslaConfiguration()

Security options:

      new CslaConfiguration()

You should consider using this new fluent API instead of the old/fragmented configuration options at all times - for setting config values. CSLA .NET itself continues to rely on Csla.ApplicationContext to read config values for use throughout the code.


CSLA .NET version 4.9 is very exciting for server-side developers, or people using a remote data portal where versioning or advanced routing scenarios are important. Between the data portal enhancements and more modern configuration techniques available in this version, your life should be improved as a CSLA .NET developer.

Wednesday, December 12, 2018 1:38:51 PM (Central Standard Time, UTC-06:00)  #    Disclaimer
 Friday, November 30, 2018

For some time now it has been the case that people are all excited about "microservices".

I want to start by saying that I think this is a horrible name for what we're trying to accomplish, because the term attempts to put the concept of size onto services.

Calling a service is expensive. You are creating a message, serializing that message, sending it over a network via http or a queue (or both), where it is deserialized, and finally the service can do some work. Any response goes through all that same overhead to get back to the caller. That overhead is measured in milliseconds - usually 10's of milliseconds.

So if your "micro"service does work that takes less than 10-20 milliseconds to complete, then you've paid this massive performance cost to get to/from the service over the network, only to have the actual work take less time than the overhead.

That's like paying your auto mechanic for a full hour, just so they can do a 5 minute fix to your car. Nobody would do that without being forced (and sometimes we are - due to specialized skills or tools/parts availability).

This analogy translates to the services world too. Calling a service is fine if the service:

  1. does enough work to be worth the overhead of calling the service (the work took longer than the network overhead)
  2. needs to run in a secure location
  3. is required to run in a geo-bounded location (think GDPR)
  4. must run on specialized hardware that doesn't exist elsewhere
  5. contains code that is so valuable that, if reverse engineered, would destroy your organization (variation on security)

I've tried to put those in order from most to least likely, but everyone's scenario is a bit different.

The point I'm really getting at, is that in the majority of cases it is important to make sure a service does enough work to make it worth the non-trivial overhead of getting to/from the service over http and/or a queue.

In those other less common scenarios there's a business decision to be made. If the service is very small and fast, you will be paying massive overhead for very little work to be done. Presumably the business is saying that those other concerns outweigh the fact that your app will be far slower than it could be.

(hint: as a developer you should make this clear to the business stakeholders up front - head off the question you know is coming later: "why is this app so slow???" - you want to be able to pull out the email thread where you explained that it will be slow because the business decision maker chose to put some tiny-but-critical bit of functionality in a service)

Friday, November 30, 2018 11:21:25 AM (Central Standard Time, UTC-06:00)  #    Disclaimer
 Tuesday, October 2, 2018

For many months (maybe years) I tried to use the Netflix app on my Surface Pro 3 and then Surface Pro 4.

The audio would always get out of sync with the video when streaming shows. Once Netflix allowed download of shows then it worked ok - but only with downloaded content.

This was true as recently as February 2018.

Searching the web revealed that other people had the issues too, and the concensus was something to do with the audio driver for the chipset used in Surface (and maybe other devices).

Recently I got a Surface Go, and to my surprise Netflix worked fine.

So I tried it again on my Surface Pro 4, and it works fine there too now.

I assume that either a Surface driver update or a Netflix app update (or both) occurred since February that finally resolved the issue with streaming and audio playback sync.

In any case, the news is good, and the Netflix app is working great on both my Surface Pro and Surface Go. This makes life in hotels a whole lot nicer, as the last thing I want is to be stuck with cable! 😃

Tuesday, October 2, 2018 4:12:24 PM (Central Standard Time, UTC-06:00)  #    Disclaimer
 Tuesday, September 25, 2018

As people reading my blog know, I'm an advocate of container-based deployment of server software. The most commonly used container technology at the moment is Docker. And the most popular way to orchestrate or manage clusters of Docker containers is via Kubernetes (K8s).

Azure, AWS, and GCE all have managed K8s offerings, so if you can use the public cloud there's no good reason I can see for installing and mananaging your own K8s environment. It is far simpler to just use one of the pre-existing managed offerings.

But if you need to run K8s in your own datacenter then you'll need some way to get a cluster going. I thought I'd figure this out from the base up, installing everything. There are some pretty good docs out there, but as with all things, I ran into some sharp edges and bumps on the way, so I thought I'd write up my experience in the hopes that it helps someone else (or me, next time I need to do an install).

Note: I haven't been a system admin of any sort since 1992, when I was admin for a Novell file server, a Windows NT server, and a couple VAX computers. So if you are a system admin for Linux and you think I did dumb stuff, I probably did 😃

The environment I set up is this:

  1. 1 K8s control node
  2. 2 K8s worker nodes
  3. All running as Ubuntu server VMs in HyperV on a single host server in the Magenic data center

Install Ubuntu server

So step 1 was to install Ubuntu 64 bit server on three VMs on our HyperV server.

Make sure to install Ubuntu server, not client, otherwise this article has good instructions.

This is pretty straightforward, the only notes are:

  1. When the server install offers to pre-install stuff, DON'T (at least don't pre-install Docker)
  2. Make sure the IP addresses won't change over time - K8s doesn't like that
  3. Make sure the MAC addresses for the virtual network cards won't change over time - Ubuntu doesn't like that

Install Docker

The next step is to install Docker on all three nodes. There's a web page with instructions

⚠ Just make sure to read to the bottom, because most Linux install docs are seem to read like those elementary school trick tests where the last step in the quiz is to do nothing - you know, a PITA.

In this case, the catch is that there is a release version of Docker so don't use the test version. Here's the bash steps to get stable Docker for Ubuntu 18.04:

sudo apt install apt-transport-https ca-certificates curl software-properties-common
curl -fsSL | sudo apt-key add -
sudo add-apt-repository "deb [arch=amd64] bionic stable"
sudo apt update
sudo apt install docker-ce

Later on the K8s instructions will say to install, but that's for Ubuntu desktop, while docker-ce is for Ubuntu server.

You will probably want to grant your current user id permission to interact with docker:

sudo usermod -aG docker $USER

Repeat this process on all three nodes.

Install kubeadm, kubectl, and kubelet

All three nodes need Docker and also the Kubernetes tools: kubeadm, kubectl, and kubelet.

There's a good instruction page on how to install the tools. Notes:

  1. Ignore the part about installing Docker, we already did that
  2. In fact, you can read-but-ignore all of the page except for the section titled Installing kubeadm, kubelet and kubectl. Only this one bit of bash is necessary:

Become root:

sudo su -

Then install the tools:

apt-get update && apt-get install -y apt-transport-https curl
curl -s | apt-key add -
cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb kubernetes-xenial main
apt-get update
apt-get install -y kubelet kubeadm kubectl
apt-mark hold kubelet kubeadm kubectl

I have no doubt that all the other instructions are valuable if you don't follow the default path. But in my case I wanted a basic K8s cluster install, so I followed the default path - after a lot of extra reading related to the non-critical parts of the doc about optional features and advanced scenarios.

Install Kubernetes on the master node

One of my three nodes is the master node. By default this node doesn't run any worker containers, only containers necessary for K8s itself.

Again, there's a good instruction page on how to create a Kubernetes cluster. The first part of the doc describes the master node setup, followed by the worker node setup.

This doc is another example of read-to-the-bottom. I found it kind of confusing and had some false starts following these instructions: hence this blog post.

Select pod network

One key thing is that before you start you need to figure out the networking scheme you'll be using between your K8s nodes and pods: the pod network.

I've been unable to find a good comparison or explanation as to why someone might use any of the numerous options. In all my reading the one that came up most often is Flannel, so that's what I chose. Kind of arbitrary, but what are you going to do?

Note: A colleague of mine found that Flannel didn't interact well with the DHCP server when installing on his laptop. He used Weave instead and it seemed to resolve the issue. I guess your mileage may vary when it comes to pod networks.

Once you've selected your pod network then you can proceed with the instructions to set up the master node.

Set up master node

Read the doc referenced earlier for more details, but here are the distilled steps to install the master K8s node with the Flannel pod network.

Kubernetes can't run if swap is enabled, so turn it off:

swapoff -a

This command only affects the current session. You also need to turn off swap after a reboot. To do this you need to edit the /etc/fstab file and comment out the lines regarding the swap file. For example, I've added a # to comment out these two lines in mine:

#UUID=xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx / ext4 defaults 0 0
#/swap.img	none	swap	sw	0	0

You can, optionally, pre-download the required container images before continuing. They'll get downloaded either way, but if you want to make sure they are local before initializing the cluster you can do this:

kubeadm config images pull

Now it is possible to initialize the cluster:

kubeadm init --pod-network-cidr=

💡 The output of kubeadm init includes a lot of information. It is important that you take note of the kubeadm join statement that's part of the output, as we'll need that later.

Next make kubectl work for the root user:

export KUBECONFIG=/etc/kubernetes/admin.conf

Pass bridged IPv4 traffic to iptables (as per the Flannel requirements):

sysctl net.bridge.bridge-nf-call-iptables=1

Apply the Flannel v0.10.0 pod network configuration to the cluster:

kubectl apply -f

⚠ Apparently once v0.11.0 is released this URL will change.

Now the master node is set up, so you can test to see if it is working:

kubectl get pods --all-namespaces

Optionally allow workers to run on master

If you are ok with the security ramifications (such as in a dev environment), you might consider allowing the master node to run worker containers.

To do this run the following command on the master node (as root):

kubectl taint nodes --all

Configure kubectl for admin users

The last step in configuring the master node is to allow the use of kubectl if you aren't root, but are a cluster admin.

⚠ This step should only be followed for the K8s admin users. Regular users are different, and I'll cover that later in the blog post.

First, if you are still root, exit:


Then the following bash is used to configure kubectl for the K8s admin user:

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

What this does is take the K8s keys that are in a secure location and copy them to the admin user's ~/.kube directory. That's where kubectl looks for the config file with the information necessary to run as cluster admin.

At this point the master node should be up and running.

Set up worker nodes

In my case I have 2 worker nodes. Earlier I talked about installing Docker and the K8s tools on each node, so all that work is done. All that remains is to join each worker node to the cluster controlled by the master node.

Like the master node, worker nodes must have swap turned off. Follow the same instructions from earlier to turn off swap on each worker node before joining it to the cluster.

That 'kubeadm join' statement that was display as a result of 'kubeadm init' is the key here.

Log onto each worker node and run that bash command as root. Something like:

sudo kubeadm join --token <token> <master-ip>:<master-port> --discovery-token-ca-cert-hash sha256:<hash>

That'll join the worker node to the cluster.

Once you've joined the worker nodes, back on the master node you can see if they are connected:

kubectl get nodes

That is it - the K8s cluster is now up and running.

Grant access to developers

The final important requirement is to allow developers access to the cluster.

Earlier I copied the admin.conf file for use by the cluster admin, but for regular users you need to create a different conf file. This is done with the following command on the master node:

sudo kubeadm alpha phase kubeconfig user --client-name user > ~/user.conf

The result is a user.conf file that provides non-admin access to the cluster. Users need to put that file in their own '/.kube/' directory with the file name ofconfig:~/.kube/config`.

If you plan to create user accounts going forward, you can put this file into the /etc/skel/ directory as a default for new users:

sudo mkdir /etc/skel/.kube
sudo cp ~/user.conf /etc/skel/.kube/config

As you create new users (on the master node server) they'll now already have the keys necessary to use kubectl to deploy their images.

Recovering from a mistake

If you somehow end up with a non-working cluster (which I did a few times as I experimented), you can use:

kubeadm reset

Do this on the master node to reset the install so you can start over.


There are a lot of options and variations on how to install Kubernetes using kubeadm. My intent with this blog post is to have a linear walkthrough of the process based as much as possible on defaults; the exception being my choice of Flannel as the pod network.

Of course the world is dynamic and things change over time, so we'll see how long this blog post remains valid into the future.

Tuesday, September 25, 2018 9:17:32 AM (Central Standard Time, UTC-06:00)  #    Disclaimer
 Monday, September 17, 2018

I'm not 100% sure of the cause here, but today I ran into an issue getting the latest Visual Studio 2017 to work with Docker for Windows.

My device does have the latest VS preview installed too, and I suspect that's the core of my issue, but I don't know for sure.

So here's the problem I encountered.

  1. Open VS 2017 15.8.4
  2. Create a new ASP.NET Core web project with Docker selected
  3. Press F5 to run
  4. The docker container gets built, but doesn't run

I tried a lot of stuff. Eventually I just ran the image from the command line

docker run -i 3247987a3

By using -i I got an interactive view into the container as it failed to launch.

The problem turns out to be that the container doesn't have Microsoft.AspNetCore.App 2.1.1, and apparently the newly-created project wants that version. The container only has 2.1.0 installed.

It was not possible to find any compatible framework version
The specified framework 'Microsoft.AspNetCore.App', version '2.1.1' was not found.
  - Check application dependencies and target a framework version installed at:
  - Installing .NET Core prerequisites might help resolve this problem:
  - The .NET Core framework and SDK can be installed from:
  - The following versions are installed:
      2.1.0 at [/usr/share/dotnet/shared/Microsoft.AspNetCore.App]

The solution turns out to be to specify the version number in the csproj file.

    <PackageReference Include="Microsoft.AspNetCore.App" Version="2.1.0" />

Microsoft's recent guidance has been to not specify the version at all and their new project template reflects that guidance. Unfortunately there's something happening on my machine (I assume the VS preview) that makes things fail if the version is not explicitly marked as 2.1.0.

Monday, September 17, 2018 10:49:26 PM (Central Standard Time, UTC-06:00)  #    Disclaimer
 Wednesday, September 12, 2018

I've recently become a bit addicted to Quora. It is probably because of their BNBR (be nice, be respectful) policy, so it isn't as nasty as Twitter and Facebook have become over the past couple years.

It also turns out that there are tech communities found on the site, and I've answered some questions recently. Stuff I probably would have (should have?) put on my blog, but wrote there instead.

Wednesday, September 12, 2018 3:13:35 PM (Central Standard Time, UTC-06:00)  #    Disclaimer
 Thursday, August 30, 2018

Software deployment has been a major problem for decades. On the client and the server.

On the client, the inability to deploy apps to devices without breaking other apps (or sometimes the client operating system (OS)) has pushed most business software development to relying entirely on the client's browser as a runtime. Or in some cases you may leverage the deployment models of per-platform "stores" from Apple, Google, or Microsoft.

On the server, all sorts of solutions have been attempted, including complex and costly server-side management/deployment software. Over the past many years the industry has mostly gravitated toward the use of virtual machines (VMs) to ease some of the pain, but the costly server-side management software remains critical.

At some point containers may revolutionize client deployment, but right now they are in the process of revolutionizing server deployment, and that's where I'll focus in the remainder of this post.

Fairly recently the concept of containers, most widely recognized with Docker, has gained rapid acceptance.


Containers offer numerous benefits over older IT models such as virtual machines. Containers integrate smoothly into DevOps; streamlining and stabilizing the move from source code to deployable assets. Containers also standardize the deployment and runtime model for applications and services in production (and test/staging). Containers are an enabling technology for microservice architecture and DevOps.

Virtual Machines to Containers

Containers are somewhat like virtual machines, except they are much lighter weight and thus offer major benefits. A VM virtualizes the hardware, allowing installation of the OS on "fake" hardware, and your software is installed and run on that OS. A container virtualizes the OS, allowing you to install and run your software on this "fake" OS.

In other words, containers virtualize at a higher level than VMs. This means that where a VM takes many seconds to literally boot up the OS, a container doesn't boot up at all, the OS is already there. It just loads and starts our application code. This takes fractions of a second.

Where a VM has a virtual hard drive that contains the entire OS, plus your application code, plus everything else the OS might possibly need, a container has an image file that contains your application code and any dependencies required by that app. As a result, the image files for a container are much smaller than a VM hard drive.

Container image files are stored in a repository so they can be easily managed and then downloaded to physical servers for execution. This is possible because they are so much smaller than a virtual hard drive, and the result is a much more flexible and powerful deployment model.

Containers vs PaaS/FaaS

Platform as a Service and Functions as a Service have become very popular ways to build and deploy software, especially in public clouds such as Microsoft Azure. Sometimes FaaS is also referred to as "serverless" computing, because your code only uses resources while running, and otherwise doesn't consume server resources; hence being "serverless".

The thing to keep in mind is that PaaS and FaaS are both really examples of container-based computing. Your cloud vendor creates a container that includes an OS and various other platform-level dependencies such as the .NET Framework, nodejs, Python, the JDK, etc. You install your code into that pre-built environment and it runs. This is true whether you are using PaaS to host a web site, or FaaS to host a function written in C#, JavaScript, or Java.

I always think of this as a spectrum. On one end are virtual machines, on the other is PaaS/FaaS, and in the middle are Docker containers.

VMs give you total control at the cost of you needing to manage everything. You are forced to manage machines at all levels, from OS updates and patches, to installation and management of platform dependencies like .NET and the JDK. Worse, there's no guarantee of consistency between instances of your VMs because each one is managed separately.

PaaS/FaaS give you essentially zero control. The vendor manages everything - you are forced to live within their runtime (container) model, upgrade when they say upgrade, and only use versions of the platform they currently support. You can't get ahead or fall behind the vendor.

Containers such as Docker give you some abstraction and some control. You get to pick a consistent base image and add in the dependencies your code requires. So there's consistency and maintainability that's far superior to a VM, but not as restrictive as PaaS/FaaS.

Another key aspect to keep in mind, is that PaaS/FaaS models are vendor specific. Containers are universally supported by all major cloud vendors, meaning that the code you host in your containers is entirely separated from anything specific to a given cloud vendor.

Containers and DevOps

DevOps has become the dominant way organizations think about the development, security, QA, deployment, and runtime monitoring of apps. When it comes to deployment, containers allow the image file to be the output of the build process.

With a VM model, the build process produces assets that must be then deployed into a VM. But with containers, the build process produces the actual image that will be loaded at runtime. No need to deploy the app or its dependencies, because they are already in the image itself.

This allows the DevOps pipeline to directly output a file, and that file is the unit of deployment!

No longer are IT professionals needed to deploy apps and dependencies onto the OS. Or even to configure the OS, because the app, dependencies, and configuration are all part of the DevOps process. In fact, all those definitions are source code, and so are subject to change tracking where you can see the history of all changes.

Servers and Orchestration

I'm not saying IT professionals aren't needed anymore. At the end of the day containers do run on actual servers, and those servers have their own OS plus the software to manage container execution. There are also some complexities around networking at the host OS and container levels. And there's the need to support load distribution, geographic distribution, failover, fault tolerance, and all the other things IT pros need to provide in any data center scenario.

With containers the industry is settling on a technology called Kubernetes (K8S) as the primary way to host and manage containers on servers.

Installing and configuring K8S is not trivial. You may choose to do your own K8S deployment in your data center, but increasingly organizations are choosing to rely on managed K8S services. Google, Microsoft, and Amazon all have managed Kubernetes offerings in their public clouds. If you can't use a public cloud, then you might consider using on-premises clouds such as Azure Stack or OpenStack, where you can also gain access to K8S without the need for manual installation and configuration.

Regardless of whether you use a managed public or private K8S cloud solution, or set up your own, the result of having K8S is that you have the tools to manage running container instances across multiple physical servers, and possibly geographic data centers.

Managed public and private clouds provide not only K8S, but also the hardware and managed host operating systems, meaning that your IT professionals can focus purely on managing network traffic, security, and other critical aspects. If you host your own K8S then your IT pro staff also own the management of hardware and the host OS on each server.

In any case, containers and K8S radically reduce the workload for IT pros in terms of managing the myriad VMs needed to host modern microservice-based apps, because those VMs are replaced by container images, managed via source code and the DevOps process.

Containers and Microservices

Microservice architecture is primarily about creating and running individual services that work together to provide rich functionality as an overall system.

A primary attribute (in my view the primary attribute) of services is that they are loosely coupled, sharing no dependencies between services. Each service should be deployed separately as well, allowing for indendent versioning of each service without needing to deploy any other services in the system.

Because containers are a self-contained unit of deployment, they are a great match for a service-based architecture. If we consider that each service is a stand-alone, atomic application that must be independently deployed, then it is easy to see how each service belongs in its own container image.

This approach means that each service, along with its dependencies, become a deployable unit that can be orchestrated via K8S.

Services that change rapidly can be deployed frequently. Services that change rarely can be deployed only when necessary. So you can easily envision services that deploy hourly, daily, or weekly, while other services will deploy once and remain stable and unchanged for months or years.


Clearly I am very positive about the potential of containers to benefit software development and deployment. I think this technology provides a nice compromise between virtual machines and PaaS, while providing a vendor-neutral model for hosting apps and services.

Thursday, August 30, 2018 12:47:24 PM (Central Standard Time, UTC-06:00)  #    Disclaimer
 Thursday, August 23, 2018

Git can be confusing, or at least intimidating. In particular, if you end up working on a project that relies on a pull request (PR) model, and even more so if forks are involved.

This is pretty common when working on GitHub open source projects. Rarely is anyone allowed to directly update the master branch of the primary repository (repo). The way changes get into master is by submitting a PR.

In a GitHub scenario any developer is usually interacting with three repos:

Forks are created using the GitHub web interface, and they basically create a virtual "copy" of the primary repo in the developer's GitHub workspace. That fork is then cloned to the developer's workstation.

In many corporate environments everyone works in the same repo, but the only way to update master (or dev or a shared branch) is via a PR.

In a corporate scenario developers often interact with just two repos:

The developer clones the primary repo to their workstation.

Whether from a GitHub fork or a corporate repo, cloning looks something like this (at the command line):

$ git clone

This creates a copy of the repo in the cloud onto the dev workstation. It also creates a connection (called a remote) to the cloud repo. By default this remote is named "origin".

Whether originally from a GitHub fork or a corporate repo, the developer does their work against the clone, what I'm calling the Dev workstation repo in these diagrams.

First though, if you are using the GitHub model where you have the primary repo, a fork, and a clone, then you'll need to add an upstream repo to your dev workstation repo. Something like this:

$ git remote add MarimerLLC

This basically creates a (readonly) connection between your dev workstation repo and the primary repo, in addition to the existing connection to your fork. In my case I've named the upstream (primary) repo "MarimerLLC".

This is important, because you are very likely to need to refresh your dev workstation repo from the primary repo from time to time.

Again, developers do their work against the dev workstation repo. They should do their work in a branch other than master. Mostly work should be done in a feature branch, usually based on some work item in VSTS, GitHub, Jira, or whatever you are using for project and issue management.

Back to creating a branch in the dev workstation repo. Personally I name my branches with the issue number, a dash, and a word or two that reminds me what I'm working on in this branch.

$ git fetch MarimerLLC
$ git checkout -b 123-work MarimerLLC/master

This is where things get a little tricky.

First, the git fetch command makes sure my dev workstation repo has the latest changes from the primary repo. You might think I'd want the latest from my fork, but in most cases what I really want is the latest from the primary repo, because that's where changes from other developers might have been merged - and I want their changes!

The git checkout command creates a new branch named "123-work" based on MarimerLLC/master. So based on the real master branch from the primary repo; the one I just made sure was updated from the cloud to be current.

This means my working directory on my computer is now using the 123-work branch, and that branch is identical to master from the primary repo. What a great starting point for any new work.

Now the developer does any work necessary. Editing, adding, removing files, etc.

One note on moving or renaming files: if you want to keep the file's history intact as you move or rename a file it is best to use git to make the changes.

$ git mv OldFile.cs NewFile.cs

At any point while you are doing your work you can commit your changes to the dev workstation repo. This isn't a "backup", because it is on your computer. But it is a snapshot of your work, and you can always roll back to earlier snapshots. So it isn't a bad idea to commit after you've done some work, especially if you are about to take any risks with other changes!

Personally I often use a Windows shell add-in called TortoiseGit to do my local commits, because I like the GUI experience integrated into the Windows Explorer tool. Other people like different GUI tools, and some like the command line.

At the command line a "commit" is really a two part process.

$ git add .
$ git commit -m '#123 My comment here'

The git add command adds any changes you've made into the local git index. Though it says "add", this adds all move/rename/delete/edit/add operations you've done to any files.

The git commit command actually commits the changes you just added, so they become part of the permanent record within your dev workstation repo. Note my use of the -m switch to add a comment (including the issue number) about this commit. I think this is critical! Not only does it help you and your colleagues, but putting the issue number as a tag allows tools like GitHub and VSTS to hyperlink to the issue details.

OK, so now my changes are committed to my dev workstation repo, and I'm ready to push them up into the cloud.

If I'm using GitHub and a fork then I'll push to my personal fork. If I'm directly using a corporate repo I'll push to the corporate repo. Keep in mind though, that I'm pushing my feature branch, not master!

$ git push origin

This will push my current branch (123-work) to origin, which is the cloud-based repo I cloned to create my dev workstation repo.

GitHub with a fork:


The 123-work in the cloud is a copy of that branch in my dev workstation repo. There are a couple immediate benefits to having it in the cloud

  1. It is backed up to a server
  2. It is (typicaly) visible to other developers on my team

I'll often push even non-working code into the cloud to enable collaboration with other people. At least in GitHub and VSTS, my team members can view my branch and we can work together to solve problems I might be facing. Very powerful!

(even better, but more advanced than I want to get in this post, they can actually pull my branch down onto their workstation, make changes, and create a PR so I can merge their changes back into my working branch)

At this point my work is both on my workstation and in the cloud. Now I can create a pull request (PR) if I'm ready for my work to be merged into the primary master.

BUT FIRST, I need to make sure my 123-work branch is current with any changes that might have been made to the primary master while I've been working locally. Other developers (or even me) may have submitted a PR to master in the meantime, so master may have changed.

This is where terms like "rebase" come into play. But I'm going to skip the rebase concept for now and show a simple merge approach:

$ git pull MarimerLLC master

The git pull command fetches any changes in the MarimerLLC primary repo, and then merges the master branch into my local working branch (123-work). If the merge can be done automatically it'll just happen. If not, I'll get a list of files that I need to edit to resolve conflicts. The files will contain both my changes and any changes from the cloud, and I'll need to edit them in Visual Studio or some other editor to resolve the conflicts.

Once any conflicts are resolved I can move forward. Even if there weren't conflicts I'll need to commit the merged changes from the cloud into my local repo.

$ git add .
$ git commit -m 'Merge upstream changes from MarimerLLC/master'

It is critical at this point that you make sure the code compiles and that your unit tests run locally! If so, proceed. If not, fix any issues, then proceed.

Push your latest changes into the cloud.

$ git push origin

With the latest code in the cloud you can create a PR. A PR is created using the web UI of GitHub, VSTS, or whatever cloud tool you are using. The PR simply requests that the code from your branch be merged into the primary master branch.

In GitHub with a fork the PR sort of looks like this:

In a corporate setting it looks like this:

In many cases submitting a PR will trigger a continuous integration (CI) build. In the case of CSLA I use AppVeyor, and of course VSTS has great build tooling. I can't imagine working on a project where a PR doesn't trigger a CI build and automatic run of unit tests.

The great thing about a CI build at this point is that you can tell that your PR builds and your unit tests pass before merging it into master. This isn't 100% proof of no issues, but it sure helps!

It is really important to understand that there is an ongoing link from the 123-work branch in the cloud to the PR. If I change anything in the 123-work branch in the cloud that changes the PR.

The upside to this is that GitHub and VSTS have really good web UI tools for code reviews and commenting on code in a PR. And the developer can just go change their 123-work branch on the dev workstation to respond to any comments, then

  1. git add
  2. git commit
  3. git push origin

as shown above to get those changes into the cloud-based 123-work branch, thus updating the PR.

Assuming any changes requested to the PR have been made and the CI build and unit tests pass, the PR can be accepted. This is done through the web UI of GitHub or VSTS. The result is that the 123-work branch is merged into master in the primary repo.

At this point the 123-work branch can (and should) be deleted from the cloud and the dev workstation repo. This branch no longer has value because it has been merged into master. Don't worry about losing history or anything, that won't happen. Getting rid of feature branches once merged is necessary to keep the cloud and local repos all tidy.

The web UI can be used to delete a branch in the cloud. To delete the branch from your dev workstation repo you need to move out of that branch, then delete it.

$ git checkout master
$ git branch -D 123-work

Now you are ready to repeat this process from the top based on the next work item in the backlog.

Thursday, August 23, 2018 4:00:41 PM (Central Standard Time, UTC-06:00)  #    Disclaimer

Does anyone understand how System.Data.SqlClient assemblies get pulled into projects?

I have a netstandard 2.0 project where I reference System.Data.SqlClient. I then reference/use that assembly in a Xamarin project. And this seems to work, but creates a compile-time warning in the Xamarin project

The assembly 'System.Data.SqlClient.dll' was loaded from a different 
  path than the provided path

provided path: /Users/user135287/Library/Caches/Xamarin/mtbs/builds/

actual path: /Library/Frameworks/Xamarin.iOS.framework/Versions/

I don't think the warning actually causes any issues - but (like a lot of people) I dislike warnings during my builds. Sadly, I don't know how to get rid of this particular warning.

I guess I also don't know if it has anything to do with my Class Library project using System.Data.SqlClient, or maybe this is just a weird thing with Xamarin iOS?

Thursday, August 23, 2018 3:15:31 PM (Central Standard Time, UTC-06:00)  #    Disclaimer