BPMN question

Recently I learned how to model business processes with BPMN 2.0, the Business Process Modeling Notation. Following, a very, very brief introduction.

Using the BPM standard allows technical and business engineers (and everybody else) to better understand one another by using a process-oriented approach to modeling, while keeping the advantage of having executable (automatable) process models. It is highly recommendend to pre-define certain patterns and guidelines for usage, especially when working cross-departmental and cross-functional or rolling out company wide.

The great thing about BPMN is that it is really simple. This process obviously doesn’t do anything useful, it’s only to show some of the tasks, events etc.:

Now, a problem. Let’s say asynchronous service calls are modeled like like this:

How do you model synchronous service calls? You could do it like this:

But is that really 100% unambiguous? Sure, there would also be notes about the details (and objects used) of the process. But wouldn’t this be better?

There’s been quite some discussion about this; if anybody here has an opinion or best practise advice, let me know.

Advertisements
Tagged with:
Posted in Tools

MongoDB: when documents aren’t perfect, cope with it

MongoDB is a NoSQL database that organizes data as JSON documents in collections instead as rows in tables. One major difference between a document in a collection and a row in a table is that while all rows in a table share the same schema, each document in a collection can be completely different. With all the huge advantages this approach brings, there is a friction when strongly typed domain objects are deserialized from a collection. A document needs to match the shape of the object that it should be serialized to.

The “official” MongoDB driver for .NET – that is the one offered and supported by 10Gen, the company behind MongoDB – offers a serializer that can be told quite exactly how tolerant it needs to be with the deserialization of JSON documents in to objects:

  • When a JSON document defines extra elements that are not present in the class that it is deserialized into, the serializer can either ignore them, stash them into a “catch-all” property, or throw an exception.
  • When properties on the class are missing in the JSON document, the serializer can set the property to null, it can set a default value, or it can throw an exception.

These options are well described in the official documentation.

Handling deserialization errors in collections

The Mongo C# driver supports operations to retrieve single elements from the database as well as collections. As we’ve seen, there is a lot of flexibility in defining a level of tolerance that matches your application’s requirements for the deserialization of single objects, the policy for collections is very simple: if any element fails, the whole collection is not loaded. While this is the right behaviour for scenarios that rely on data integrity, applications that focus on availability are threatened by complete loss of functionality when just a single document in a collection does not match the deserialization requirements.

  • A scenario that relies on data integrity is a form in a line of business application that edits a list of closely related data like line items of an invoice. The line items all need to provide the same set of information, otherwise the invoice cannot reliably be calculated.
  • A scenario that focuses on availability is a search in a product catalogue in an online shop. If a single product is not correctly formatted in the database, it’s better to just not show that single product than letting the whole search page become unavailable and basically close the shop until the ill-formatted product is identified and corrected.

In the case of the search page, a better behaviour would be to exclude any document that cannot be serialized, and provide a callback that identifies the problematic documents. The approach I’ve taken is to copy the default implementation of the collection serializer, and augment the bit that iterates over the elements of the collection with an event handler that keeps the iteration running:

while (bsonReader.ReadBsonType() != BsonType.EndOfDocument)
{
	var elementType = discriminatorConvention.GetActualType(bsonReader, typeof(T));
	var serializer = BsonSerializer.LookupSerializer(elementType);

	T element;
	try
	{
		element = (T)serializer.Deserialize(bsonReader, typeof(T), elementType, null);
		list.Add(element);
	}
	catch (FileFormatException exc)
	{
		// Pass the exception to the provided callback
		if (HandleDeserializationError != null)
		{                           
			HandleDeserializationError(exc);
		}

		// Move the cursor to the next element after the faulted one
		while (bsonReader.State != BsonReaderState.EndOfDocument)
		{
			if (bsonReader.State == BsonReaderState.Value) bsonReader.SkipValue();
			if (bsonReader.State == BsonReaderState.Type) bsonReader.ReadBsonType();
		}
		bsonReader.ReadEndDocument();
	}                    
}
bsonReader.ReadEndArray();

You can download the full source from Bitbucket. Please note that it’s not production quality – it has not been used in production, but that could change over the next few weeks, and then it’s going to get an update with the real thing.

Tagged with:
Posted in Coding

Fixing non-localizable validation messages with JavaScript

There is a nasty globalization bug in ASP.NET MVC: the client-side validation message for numeric is hard coded to a string that is not localizable.

Take the following view model class:

public class SomeViewModel
{
    public int SomeNumber { get; set; }
}

… and this view:

@model SomeViewModel
@{
    Layout = "~/Views/Shared/_Layout.cshtml";
    Html.EnableClientValidation();
    Html.EnableUnobtrusiveJavaScript();
}
@using (Html.BeginForm())
{    
    @Html.EditorFor(vm => vm.SomeNumber)
    @Html.ValidationMessageFor(vm => vm.SomeNumber)
}

If the user enters a non-numeric value into the text box form SomeNumber, client side validation kicks in and displays a message. This is excellent, you can’t have validation for less effort. This message is “The field SomeNumber must be a number.”. Umm. That might need some rewording, especially when you wish to translate the user interface into a language other than English. The trouble is: you can’t.

The string “The field {0} must be a number.” is baked into ASP.NET MVC in a resource file. There is no way to define a different resource file, or overwrite the message in any other way (like there is for server-side default messages by setting DefaultModelBinder.ResourceClassKey). For other messages, like the message for required fields, this is not a big deal because a custom message can be applied by setting a DataAnnotation attribute – however, this is not possible for data type based validation.

The options at hand are:

  • Do without the integration of data types and client side validation, and not use EditorFor. Instead, declare the text box directly, and set the data-val-number attribute that controls the message directly. Yuck – that feels like gearing the progeamming model around a platform bug. I’m not desperate for that yet.
  • Fix ASP.NET MVC. It’s open sourced, you can run your own fixed branch, or at least replace the part that causes the problem. Yes, I might do this, but it causes some maintenance problem – each time a new version of the framework is released without a fix, I’d need to recompile. Sounds like a lot of work.
  • Use AOP to basically do the same. That solutions yields a lot less code, but still has a very tight dependency on the current version of the framework; in addition, I’d need to throw another framework into the project. Maybe.
  • The problem lies within ASP.NET MVC, but in the outcome, it’s just a client-side message. Hence, replacing the non-localized message with a localized one does not have to happen on the server. Let’s see where these messages are applied!

The validation rules and messages for client side validation are defined in HTML5-style “data-” attributes that are attached to the form’s input elements. As an example, this is what ASP.NET emits for the integer value from the example above:

 <input 
	class="text-box single-line" 
	data-val="true" 
	data-val-number="The field SomeNumber must be a number." 
	data-val-required="The SomeNumber field is required." 
	id="SomeNumber" 
	name="SomeNumber"
	type="text"
	value="0" />

There are two JavaScript modules that construct the client side validation:

  • jquery.validate provides all the heavy lifting for evaluating and comparing values, and for displaying the messages.
  • jquery.validate.unobstrusive hooks the validation functionality provided by jquery.validate into the page in an “unobstrusive” way – that is, without any JavaScript code that clutters the page. Instead, attributes on the validated controls declare what should be validated.

When jquery.validate.unobstrusive is loaded, it reads all validation attributes on the current page. This is the first chance to change the content of the attribute. The script is cleverly structured; it attaches a host of adapters that match attributes and then initialize the appropriate jquery.validate function. A set of options is passed through the adapter, containing (among other things) the message. Directly after jquery.validate.unobstrusive is loaded, we can hook into its adapters and change the one for numbers:

(function ($) {
    // Walk through the adapters that connect unobstrusive validation to jQuery.validate.
    // Look for all adapters that perform number validation
    $.each($.validator.unobtrusive.adapters, function () {
        if (this.name === "number") {
            // Get the method called by the adapter, and replace it with one 
            // that changes the message to the jQuery.validate default message
            // that can be globalized. If that string contains a {0} placeholder, 
            // it is replaced by the field name.
            var baseAdapt = this.adapt;
            this.adapt = function (options) {
                var fieldName = new RegExp("The field (.+) must be a number").exec(options.message)[1];
                options.message = $.validator.format($.validator.messages.number, fieldName);
                baseAdapt(options);
            };
        }
    });
} (jQuery));

Now, instead of the message defined by ASP.NET MVC, the default message provided by jQuery.validate is used. This is much better because this one _can_ be localized:

$.validate.messages.number="Eine Nummer für {0}, bitteschön";

Instead of inventing your own messages you should maybe use the jquery-glob plugin that comes with a variety of langauges.

Using this solution in your own project

  1. Copy the JavaScript code above into a new JavaScript file named jquery.validate.unobstrusive.fixMvcNumberMessage.js in the Scripts directory of your ASP.NET MVC project.
  2. Add a reference to that file directly beneath jquery.validate.unobstrusive. The references to scripts are normally held in the _Layout shared view.

Feel free to use the code in this article as it serves you.

Tagged with: , ,
Posted in Coding

On merging project files

Merging Visual Studio Project files: there’s not much nice to say about that. It’s just a pain, and I’ve ssen it become the number one or two reason of errors when merging between code branches.

Even when a project consists of a single team, it is not unusual that one developer adds a bunch of files while another one adds some more, or deletes and moves things around in the project. The source control system and its diff/merge tooling easily copes with the added and deleted code files, and it reliably notices and auto-resolves changes within code. However, to integrate into Visual Studio and the build process, code files also need to be referenced correctly in the *.csproj (or *.vbproj) project files. Here, developers often find strange conflicts, and the diff/merge tools are often not much of a help.

Project files consist or XML that is read by Visual Studio and Msbuild. Their root element is a Project. Project has child elements that indicate properties of the project like its build configuration (and some more), but most important in this context are the ItemGroup elements that contains the content the project:

  • Compile elements that define files compiled into the assembly
  • Content elements that define files not compiled, but otherwise included in the project: MVC views, script files, images contained in a Silverlight XAP…
  • Reference elements that define the projects binary or project dependencies.

Here’s a simplified project file for an ASP.NET MVC project file:

<Project>
<!-- Project properties and extensions omitted for brevity -->
<ItemGroup>
    <Reference Include="System.Web.Mvc />
    <Reference Include="System.Web.WebPages" />
  </ItemGroup>
  <ItemGroup>
    <Compile Include="Controllers\SquirrelController.cs" />
    <Compile Include="DataAccess\SquirrelRepository.cs" />
    <Compile Include="Global.asax.cs">
      <DependentUpon>Global.asax</DependentUpon>
    </Compile>
    <Content Include="Global.asax" />
    <Content Include="Content\Site.css" />
    <Content Include="Web.config" />
    <Content Include="Web.Debug.config">
      <DependentUpon>Web.config</DependentUpon>
    </Content>
    <Content Include="Web.Release.config">
      <DependentUpon>Web.config</DependentUpon>
    </Content>
    <Content Include="Scripts\jquery-1.4.1.js" />
    <Content Include="Scripts\jquery-1.4.1.min.js" />
  </ItemGroup>
  </Project>

Now why is there such a problem? First, most diff/merge tools just do not understand XML. These tools are designed to identify blocks of code. They know things about code: that outside strings, different kinds of whitespace can be considered as equivalent; that changes within a comment are less significant than elsewhere; where casing is significant or not. What most tools do not understand is that all of the following statements are equivalent:

<Squirrel name="Alex" fur="red"/>

<Squirrel fur="red" name="Alex" />

<Squirrel 
    fur="red" 
    name="Alex" />

<Squirrel fur="red" name="Alex"></Squirrel>

Secondly, the order of ItemGroup elements in the project, and of elements within them, does not matter – in programming languages, the sequence of statements is essential. Now that for project files the sequence is not significant, Visual Studio feels free to arrange items as it feels fit – and the diff/merge tool report hard to resolve conflicts even if two versions of a project file are absolutely equivalent.

Avoiding the pain

One feasible way of avoiding merging between versions of project files is to apply pessimistic locking. Only one developer can have a writable copy of the project, and there are no more conflicting version (at least within the same code branch). The downside is that developers may have to wait for their colleagues to finish their revision. The most frequent thing that a developer does with a project file is probably adding content. To get the shortest possible pessimistic lock on the project file, it should be committed as soon as the files are added – the added files should not contain any code at that point in time that could break the build until it is complete.

However, this approach does not remedy merges between code branches, and does not work with source control systems that just do not allow any pessimistic locking. And this includes each distributed source control system like Git or Mercurial.

Tooling

As we’ve seen, the root of the problem is not that project files are resistant to merging by their nature; it’s just the standard tooling that do not fit. However, there are specific tools that can help. The best candidate I’ve seen so far is a commercial yet relatively cheap (19€) tool called ProjectMerge (http://www.projectmerge.com). ProjectMerge understands XML: it recognizes equivalent XML elements and attributes despite or a possibly different serialization into text. But what is really sweet is that ProjectMerge can be told if the order of elements matters, and by which attributes equivalent elements are determined. In the case of Visual Studio project files, this attribute is the Include attribute of Compile, Content, None, and Reference elements. In effect, ProjectMerge will be able to understand these instructions are equivalent:

<Compile Include="1.cs" />
<Compile Include="2.cs" />
<Compile Include="3.cs" />

<!-- and -->

<Compile Include="3.cs" />
<Compile Include="2.cs" />
<Compile Include="1.cs" />

The necessary settings to understand a specific project file format are stored in a distributable file format. Here’s my (probably not very complete) shot at C# projects – download here (from my Dropbox) and store under %PROGRAMDATA%\Oso\ProjectMerge\Formats.

I haven’t used the tool in practice yet, but I think we’re going to try it in a >10 person team over the next week, and see how it performs.

Tagged with:
Posted in Tools

NUnit for Silverlight: Runs on Windows Phone 7, too

Yesterday I’ve hauled the “big” laptop home from work and tried plugging the unofficial port of the NUnit unit testing framework for Silverlight into Windows Phone 7. Jeff Wilcox had added support for Windows Phone 7 into his Silverlight unit test framework bis from last May already, so it’s little surprise that there was not much of a problem.

I did not compile the whole of the NUnit port to the Windows Phone 7 runtime, but use the Silverlight 3 version that references the May 2010 unit testing framework. When these assemblies are referenced, Visual Studio complains that using a Silverlight assembly could lead to unpredictable behavior, but this does not apply to this project. Firstly, the major differences are really close to the user interface, and NUnit does not render anything anywhere, it does not even reference anything Silverlight-specific types. Secondly, the whole test suite runs without errors. Compiling the NUnit framework to the WP7 runtime would be totally feasible, but any test project on WP7 needs to reference the Silverlight unit testing framework, which is a Silverlight 3 assembly. You’ll get the warnings about unpredictable behavior anyway.

The documentation on ho to run NUnit tests on Windows Phone 7 is on the Coogle Code Wiki. The project’s binaries have not changed, but I’ve added a version of the NUnit test suite for Wp7 that you can find in the code.

Tagged with: , ,
Posted in Coding, Testing

Link whole directories into Visual Studio Projects

You can include a file into a Visual Studio project in two ways: copy it into the project directory, or link it. Copying the file into the project directory is Visual Studio’s default option: it’s what happens when you drag a file from Explorer into VS’s project explorer, or when you use the dialog Project -> Add existing item. This is fine unless you wish to share files between multiple projects. An example where I needed to share a whole lot of files between different projects is NUnit for Silverlight: the same code is used by three projects for different versions of Silverlight and the Silverlight unit testing framework.

To link a file into Visual Studio, you can use the Project -> Add existing item dialog: the “Add” button has a dropdown menu, and the second option is “Add as link”. This will leave the file where it is and add a relative link to the Visual Studio project file. In the project file (which is BTW a MSBuild file), this looks like that:

    <Compile Include="..\..\..\src\VersionInfo.cs">
      <Link>VersionInfo.cs</Link>
    </Compile>

If you add multiple files, Visual Studio will create one item like this for each file. To add all files in a directory, mark all files in the the Project -> Add existing item dialog, and link them. This is fine as long as you don’t delete any files or add new ones – Visual Studio will not pick up these changes. New files in the directory won’t be in the project, and deleting/renaming files will cause the build to fail.

A solution is to modify the project file manually. The Include directive supports wildcards. So to link all C# code files in a directory, you can do this:

   <Compile Include="..\..\..\src\NUnit.Silverlight.Framework\*.cs" />

Nice, but in the case of NUnit for Silverlight, there were C# code files in subdirectories. Using the Compile directive as above, the files are included, but all appear in the root folder of the project – and if a file with the same name exists in the parent directory and the subdirectory, it won’t work. However, there is support for metadata keywords in MSBuild. For example, the %(Filename) keyword is a placeholder for each match in a wildcard. See this article for reference: MSBuild: By Example—Introducing Well-Known Metadata. Now we can include subdirectories:

 <Compile Include="..\..\..\src\NUnit.Silverlight.Framework\*.cs" />
    <Compile Include="..\..\..\src\NUnit.Silverlight.Framework\Attributes\*.cs">
      <Link>Attributes\%(FileName)</Link>
    </Compile> 
	<Compile Include="..\..\..\src\NUnit.Silverlight.Framework\Constraints\*.cs">
      <Link>Constraints\%(FileName)</Link>
    </Compile> 
    <Compile Include="..\..\..\src\NUnit.Silverlight.Framework\Exceptions\*.cs">
      <Link>Exceptions\%(FileName)</Link>
    </Compile> 

As you see, I need to link three subdirectories. Using a * placeholder in the directory name and the %(RecursiveDir) keyword should enable adding these with a single statement like this:

 
    <Compile Include="..\..\..\src\NUnit.Silverlight.Framework\*\*.cs">
      <Link>%(RecursiveDir)%(FileName)</Link>
    </Compile> 

… but that does not work. Never mind, including subdirectories one by one is enough for me right now.

Tagged with:
Posted in Coding

NUnit for Silverlight 2.5.8

Finally, another drop of NUnit test framework for Silvelight has been finished:

  • It is based on NUnit version 2.5.8, a step up from 2.5.5.
  • The complied assemblies are strong-named. I’ve included the key in the source code so that if you want to compile yourself, you’ve got the exact same version.
  • The project is now hosted on Google Code. I’ve removed the links to download the sources and binaries from Dropbox.
  • I reviewed the licenses to see what the license for the complete oevre could be. The answer is that there are in fact two licenses: NUnit has its own license; the source code contains the terms. This license applies to the Nunit.Silverlight.Framework and Nunit.Silverlight.Framework.Tests assemblies. The metadata provider for the Silverlight unit testing framework goes back to Jeff Wilcox, who has put this piece under MS-PL. So the MS-PL applies to the NUnit.Silverlight.Metadata assembly.
  • In the test runner, data tests now have names that reflect the parameter values, and are available for the original version of the unit testing framework for Silverlight 3.

The next thing to come will be support for Windows Phone 7.

Download

Please download from nunit-silverlight on Google Code.

Tagged with: ,
Posted in Coding, Testing
%d bloggers like this: