Month: June 2007

Dynamic Strongly-Typed Configuration in C#

    I’ve written at great length in the past about the perils of configuration, and I thought I’d written as much as I was willing on the topic. But I thought it was worth describing this solution, since it was so neat, and easy, and had most of the benefits of text based configuration and strongly typed inline configuration. I was recently messing about with some WCF P2P code, and the setup code had some configuration that looked like a likely candidate for a strongly typed configuration object that wouldn’t change frequently. I think this solution neatly addresses one of the main objections to hard coded configuration, which is that we do sometimes need to change configuration data at runtime without having to take down the servers or recompile them.

    The idea behind this solution stems from the use of a plug-in architecture such as the forthcoming System.AddIn namespace to arrive in VS2008. In that you get the options to load a namespace from a designated directory and make use of types found inside of the assembly. Why not use the same approach with configuration? We can dynamically load configuration assemblies and then use a single configuration setting to specify which type from those assemblies would be used as the new configuration. This has all the benefits normally reserved for text based dynamic configuration such as System.Configuration.ConfigurationManager, but with the added benefits of strong typing, inheritance, calculated configuration settings and added performance of POCOs.

    My WCF program was a simple chat client that I hope to be able to use between members of my family. Typical configurations were MeshAddress, and CredentialType that are unlikely to ever change frequently. Each of these configuration settings was defined on an interface called IChatClientConfig. Implementing that full interface was my default configuration class called DefaultChatConfig. That provided all of my defaults, and is perfectly usable. I then specialized that class with some others, for example with a different mesh address for chatting with people at work. A class diagram for the configuration objects are shown below.

    clip_image001

    Each class just provides a new implementation for the field that it provides a different value for.

    Loading the configuration is extremely simple. First you have to say which one of those classes you want to use for your configuration.

    <appSettings>
        <add key="P2PConfigSettings" value="ChatClient.Configuration.TechChatConfig, ChatClient.Configuration, Version=1.0.0.0"/>
    </appSettings>
    

    This simple app setting is the fully qualified type name of the TechChatConfig class on the bottom right of the diagram above. Which will be a default chat configuration with whatever tech chat configuration added. That’s all the prerequisites for loading configuration. Not all I need to do to load the configuration is this:

    private static IChatClientConfig GetConfigObject()
    {
        string configType = ConfigurationManager.AppSettings["P2PConfigSettings"];
        Type t = Type.GetType(configType);
        return Activator.CreateInstance(t) as IChatClientConfig;
    }
    

    Get whatever type I specified as a string from the configuration file, get the type specified by that string create and instance and return it. Simple. That configuration could be then stored as a singleton or whatever you need to do.

    [ServiceBehavior(InstanceContextMode = InstanceContextMode.Single)]
    public partial class Window1 : IPeerChat
    {
        private IChatClientConfig configuration;
    

    In my case I just stored it in the window object I was using it for – my chat client only has one window! Now I can just use it, whenever I do any comms.

    private NetPeerTcpBinding CreateBindingForMesh()
    {
        NetPeerTcpBinding binding = new NetPeerTcpBinding();
        binding.Resolver.Mode = config.PeerResolverMode;
        binding.Security.Transport.CredentialType = config.CredentialType;
        binding.MaxReceivedMessageSize = config.MaxReceivedMessageSize;
        return binding;
    }
    

    So you see that the process is very simple. With the addition of an AddIn model we could use a file system monitor to watch the configuration file, detect changes and reload the configuration object singleton using the mechanism described above. That fulfils most of the requirements that we have for type safety, performance, dynamism, intelligence, and object orientation. Very few configuration scenarios that fall outside of the bounds of this solution should be solved using local configuration settings anyway – in those cases you really ought to be looking at an administration console and database.

Using Mock Objects When Testing LINQ Code

I was wondering the other day whether LINQ could be used with NMock easily. One problem with testing code that has not been written to work with unit tests is that if you test business logic, you often end up making multiple round-trips to the database for each test run. With a very large test suite that can turn a few minute’s work into hours for a test suite. the best approach to this is to use mock data access components to dispense canned results, rather than going all the way through to the database.

After a little thought it became clear that all you have to do is override the IOrderedQueryable<T>.GetEnumerator() method to return an enumerator to a set of canned results and you could pretty much impersonate a LINQ to SQL Table (which is the IOrderedQueryable implementation for LINQ to SQL). I had a spare few minutes the other day while the kids were going to sleep and I decided to give it a go, to see what was involved.

I’m a great believer in the medicinal uses of mock objects. Making your classes testable using mocking enforces a level of encapsulation that adds good structure to your code. I find that the end results are often much cleaner if you design your systems with mocking in mind.

Lets start with a class that you were querying over in your code. This is the type that you are expecting to get back from your query.

public class MyEntity
{
    public string Name
    {
        get { return name; }
        set { name = value; }
    }

    public int Age
    {
        get { return age; }
        set { age = value; }
    }

    public string Desc
    {
        get { return desc; }
        set { desc = value; }
    }

    private string name;
    private int age;
    private string desc;
}

Now you need to create a new context object derived from the DLINQ DataContext class, but providing a new constructor function. You can create other ways to insert the data you want your query to return, but the constructor is all that is necessary for this simple example.

public class MockContext : DataContext
{
    #region constructors

    public MockContext(IEnumerable col):base("")
    {
        User = new MockQuery<MyEntity>(col);
    }
    // other constructors removed for readability
    #endregion
    public MockQuery<MyEntity> User;
}

Note that you are passing in an untyped IEnumerable rather than an IEnumerable<T> or a concrete collection class. The reason is that when you make use of projections in LINQ, the type gets transformed along the way. Consider the following:

var q = from u in db.User
        where u.Name.Contains("Andrew") && u.Age < 40
        select new {u.Age};

The result of db.User is an IOrderedQueryable<User> query class which is derived from IEnumerable<User>. But the result that goes into q is an IEnumerable of some anonymous type created specially for the occasion. there is a step along the way when the IQueryable<User> gets replaced with an IQueryable<AnonType>. If I set the type on the enumerator of the canned results, I would have to keep track of them with each call to CreateQuery in my Mock Query class. By using IEnumerable, I can just pass it around till I need it, then just enumerate the collection with a custom iterator, casting the types to what I ultimately need as I go.

The query object also has a constructor that takes an IEnumerable, and it keeps that till GetEnumerator() gets called later on. CreateQuery and CloneQueryForNewType just pass the IEnumerable around till the time is right. GetEnumerator just iterates the collection in the cannedResponse iterator casting them to the return type expected for the resulting query.

public class MockQuery<T> : IOrderedQueryable<T>
{
    private readonly IEnumerable cannedResponse;

    public MockQuery(IEnumerable cannedResponse)
    {
        this.cannedResponse = cannedResponse;
    }

    private Expression expression;
    private Type elementType;

    #region IQueryable<T> Members

    IQueryable<S> IQueryable<T>.CreateQuery<S>(Expression expression)
    {
        MockQuery<S> newQuery = CloneQueryForNewType<S>();
        newQuery.expression = expression;
        return newQuery;
    }

    private MockQuery<S> CloneQueryForNewType<S>()
    {
        return new MockQuery<S>(cannedResponse);
    }
    #endregion

    #region IEnumerable<T> Members
    IEnumerator<T> IEnumerable<T>.GetEnumerator()
    {
        foreach (T t in cannedResponse)
        {
            yield return t;
        }
    }
    #endregion

    #region IQueryable Members
    Expression IQueryable.Expression
    {
        get { return System.Expressions.Expression.Constant(this); }
    }

    Type IQueryable.ElementType
    {
        get { return elementType; }
    }
    #endregion
}

For the sake of readability I have left out the required interface methods that were not implemented, since they play no part in this solution. Now lets look at a little test harness:

class Program
{
    static void Main(string[] args)
    {
        MockContext db = new MockContext(GetMockResults());

        var q = from u in db.User
                where u.Name.Contains("Andrew") && u.Age < 40
                select u;
        foreach (MyEntity u in q)
        {
            Debug.WriteLine(string.Format("entity {0}, {1}, {2}", u.Name, u.Age, u.Desc));
        }
    }

    private static IEnumerable GetMockResults()
    {
        for (int i = 0; i < 20; i++)
        {
            MyEntity r = new MyEntity();
            r.Name = "name " + i;
            r.Age = 30 + i;
            r.Desc = "desc " + i;
            yield return r;
        }
    }
}

The only intrusion here is the explicit use of MockContext. In the production code that is to be tested, you can’t just go inserting MockContext where you would have used the SqlMetal generated context. You need to use a class factory that will allow you to provide the MockContext on demand in a unit test, but dispense a true LINQ to SQL context when in production. That way, all client code will just use mock data without knowing it.

Here’s the pattern that I generally follow. I got it from the Java community, but I can’t remember where:

class DbContextClassFactory
{
    class Environment
    {
        private static bool inUnitTest = false;

        public static bool InUnitTest
        {
            get { return Environment.inUnitTest; }
            set { Environment.inUnitTest = value; }
        }
        private static DataContext objectToDispense = null;

        public static DataContext ObjectToDispense
        {
            get { return Environment.objectToDispense; }
            set { Environment.objectToDispense = value; }
        }
    }

    public object GetDB()
    {
        if (Environment.InUnitTest)
            return Environment.ObjectToDispense;
        return new TheRealContext() as DataContext;
    }
}

Now you can create your query like this:

DbContextClassFactory.Environment.ObjectToDispense = new MockContext(GetMockResults());
var q = from u in DbContextClassFactory.GetDB() where ...

And your client code will use the MockContext if there is one, otherwise it will use a LINQ to SQL context to talk to the real database. Perhaps we should call this Mockeries rather than Mock Queries. What do you think?

Ubuntist deviant atones for sins

Some time back I wrote  a quote of the day: There is no greater joy than soaring high on the wings of your dreams, except maybe the joy of watching a dreamer who has nowhere to land but in the ocean of reality. For a moment I though the quote was apropos for Alec Clew’s announcement of his apostasy, but after a moment’s reflection I realize that it’s not.

Alec wrote the other day that, due to driver problems, he was thinking that he may have to depart the sunlit pastures of Gnu/Linux to return to the dreary halls of Windows XP. If you’ve noticed any of the censure I’ve directed towards Alec’s perverse habits (disgusting admissions of Ubuntism and flagrantly exhibitionist open-sourcism) over the last year, then you might think that there would be a spiteful little gleam in my eye. But I have admit I’m saddened. We are nearly at a point where we can perform painless .NET development entirely within a Gnu/Linux environment. It would be a sad thing if that were not possible because of hardware support shortcomings, rather than the quality of the toolset (which should be the only deciding factor).

GroupJoins in LINQ

OWL defines two types of property: DatatypeProperty and ObjectProperty. An object property links instances from two Classes, just like a reference in .NET between two objects. In OWL you define it like this:

<owl:ObjectProperty rdf:ID=”isOnAlbum”>
  <rdfs:domain rdf:resource=”#Track”/>
  <rdfs:range rdf:resource=”#Album”/>
</owl:ObjectProperty>

A DatatypeProperty is similar to a .NET property that stores some kind of primitive type like a string or an int. In OWL it looks like this:

<owl:DatatypeProperty rdf:ID=”fileLocation”>
  <rdfs:domain rdf:resource=”#Track” />   
  <rdfs:range  rdf:resource=”&xsd;string”/>
</owl:DatatypeProperty>

The format is very much the same, but the task of querying for primitive types in LINQ and SPARQL is easy compared to performing a one to many query like a SQL Join. So far, I have confined my efforts to DatatypeProperties, and tried not to think about ObjectProperties too much. But the time of reckoning has come – I’ve not got much else left to do on LinqToRdf except ObjectProperties.

Here’s the kind of LINQ join I plan to implement:

[TestMethod]
public void TestJoin()
{
    TestContext db = new TestContext(CreateSparqlTripleStore());
    var q = from a in db.Album 
            join t in db.Track on a.Name equals t.AlbumName into tracks
            select new Album{Name = a.Name, Tracks = tracks};
    foreach(var album in q){
        Console.WriteLine(album.Name);
        foreach (Track track in album.Tracks)
        {
            Console.WriteLine(track.Title);
        }
    }
}

This uses a GroupJoin to let me collect matching tracks and store them in a temporary variable called tracks. I then insert the tracks into the Tracks property on the album I’m newing up in the projection. I need to come up with a SPARQL equivalent syntax, and convert the expression passed for the join into that. SPARQL is a graph based query language, so I am going to be converting my requests into the usual SPARQL triple format, and then using the details from the NewExpression on the query to work out where to put the data when I get it back.

With the non-join queries I have been testing my query provider on, I have observed that for each syntactical component of the query I was passed an Expression tree, representing its contents. With a GroupJoin, you get one, and it contains everything you need to perform the query. My first quandary is over the process of converting this new expression structure into a format that my existing framework can understand. Below is a snapshot of the expression tree created for the join I showed above.

GroupJoin Expression contents

There are five parameters in the expression:

  1. The query object on the Album. That’s the “a in db.Album” part.
  2. The query object on the Track. The “t in db.Track” part.
  3. A lambda function from an album to its Name.
  4. A lambda function from a track to its AlbumName.
  5. A projection creating a new Album, and assigning the tracks collected to the Tracks collection on the newly created Album.

Parameters 1 & 2 are LinqToRdf queries that don’t need to be parsed and converted. I can’t just ask them to render a query for me, since they don’t have any information of value to offer me other than the OriginalType that they were created with. They have received no expressions filtering the kind of data that they return, and they’ll never have their results enumerated. These query objects are just a kind of clue for the GroupJoin about how to compose the query. They can tell it where the data that it’s looking for is to be found.

Here’s how I would guess the SPARQL query would look:

SELECT ?Name ?Title ?GenreName <snip> 
WHERE {
    _:a a a:Album .
    _:t a a:Track .
    _:a a:name ?Name.
    _:t a:albumName ?Name .
    OPTIONAL {_:t a: ?Title}
    OPTIONAL {_:t a: ?GenreName}
    <snip>
}

We can get the names for blank nodes _:a and _:t from the parameter collections of the GroupJoins parameters 3 and 4 respectively. We know that we will be equating ?Name on _:a and ?Name on _:t since those are the lambda functions provided and that’s the format of the join. The rest of the properties are included in optional sections so that if they are not present it won’t stop the details of the OWL instance coming back. By using

    _:a a:name ?Name.
    _:t a:albumName ?Name .

We achieve the same as equality, since two things that are equal to the same are equal to each other. That restricts the tracks to those that are part of an album at the same time.

I’m not sure yet what I will do about the projection, since there is an intermediate task that needs to be performed: to insert the temporary variable ‘tracks’ into the Album object after it has been instantiated. More on that once I’ve found out more.

Wireless Power Transmission

I spent years bemoaning the fact that the laws of physics didn’t allow the wireless transmission of power. Now it seems that I bemoaned prematurely. Researchers at MIT have a found a way to use inductance to power a 60W (think similar power rating to a laptop) lightbulb from a distance of up to 2 metres.

I  know it doesn’t sound like much, but as I look at my desk I can’t help thinking that a short range power transmitter would help. I’m sure it beats Quartz crystals as a source of magic moonbeams as well.

cables

Perils of Agile development – the outsourcee’s perspective

Alex Deva has recently written a very persuasive article on the frustrations of Agile development as seen from the far end of an outsourcing agreement. He runs a company in Romania – Indigenous Development – and has past experience of many Agile projects.

I couldn’t agree with his observations more. The problems he observes are problems that arise in on-shored and near-shored development models as well. They are inherent weaknesses of Agile. I think he stated them very clearly though.

Unlike Alex, I think what is required is not a return to the classic waterfall model. After all, what Agile sought to overturn really was just as bad. Instead, I think that what we need is a more formal version of Agile  with a period of up-front intense requirements analysis and high-level architecture. In other words we need to merge the benefits of both methodologies, throwing out any practices that will upset the balance of power within the project team.

Easily Bored?

Darren Neimke posted some interesting thoughts today about the way developers lose their drive on a project, and how it’s reflected in SCRUM meetings. He thought that it might be due to the SCRUM meetings themselves. Daniel Crowley-Wilson has another idea – the developers are just bored.

Developers relish challenges and opportunities to do new things, and solve novel problems. As Daniel says, about midway through a project, there is little novelty in the problems left to be solved. At the end there is just the soul destroying finishing touches, which we all know have to be done, but which we hate doing.

I think that ‘good’ developers (Daniel’s phrase) are a particular breed. They are stimulus hungry people. They tend to quickly become immune to the initial piquancy of stimuli, entertainments or whatever interests them this week. They are not likely to remain interested in a domain or technology for long.

Evidence of this trait can be seen by the amount of staff turnover that most software vendors suffer, or the amount of technological churn that the developers tend to create. Those are the negatives – there’s also positives, like the rampant pace of forward progress. Developers can (given practice & solitude) sustain a high level of attention on a topic for long periods of time. I think this just exacerbates the problem of their easy boredom in the long run.

Because of the two characteristics of easy boredom and manic singlemindedness, Darren’s solution will probably not solve the problem either – the problem is that they require fresh inputs. Both in the job and in the SCRUM meetings. Perhaps your best bet, Darren, is to periodically change the format of the SCRUM meetings, and mix up the teams if you can.

Here’s a simple test to see whether a team member falls into this category – ask them some of the following:

  • how many hobbies have they had
  • how often do they change their desktop backgrounds
  • how frequently do they change jobs
  • how many projects do they have on the go, or up their sleeves
  • how many ideas for killer apps have they had (and not followed up)