Announcing LinqToRdf v0.6

I’ve just uploaded LinqToRdf v0.6 with improved designer support for Visual Studio .NET 2008.

The release includes the following high-points:

  • LinqToRdf Designer and VS.NET 2008 extension completely rewritten
  • LinqToRdf Installer now includes the installer of LinqToRdf Designer (at no extra cost)
  • Project and Item templates now installed as part of LinqToRdf Designer
  • Generated object and data properties now get their own EntitySet or EntityRef.
  • Generates LINQ to SQL-style DataContext objects to hide query creation. Much Cleaner.

The user experience for LinqToRdf should be greatly improved in this release.  I focussed on getting project and item templates set up that would allow you to either create a dedicated LinqToRdf project that would have all the assembly references set up for you, or to create a new LinqToRdf designer file, that would generate C# code based on the new Attribute model introduced a few versions back.

The VS.NET extensions are not installed by default, instead they are created in the LinqToRdf directory. If you do install them, then you will find that visual studio will now have a LinqToRdf will have a new project type.


You also have the LinqToRdf designer file type, which has been around for a version or two:


The Solution view is like this:


The designer view is the same as ever:


Things are coming along, and the download stats for version 0.4 were actually quite healthy (at least i think they were) so I expect this version to be the most popular yet.

Expect to see the lazy-loading relationship representation process fully documented in the coming days.


Announcing LinqToRdf v0.5

The fifth release of LinqToRdf has just been uploaded to Google Code.

Go to for links to the latest MSI

LinqToRdf Changes:
– Member access projections like this:
  from a in c select a.Name
– initial support for Distinct results
– latest versions of SemWeb & SPARQL Engine, incorporating the new
SqlServer store provider
– updated the DSL tools to compile on RTM version of VS.NET 2008

Please let me know how you get on.

Functional Programming in C# – Higher-Order Functions

  1. Functional Programming – Is it worth your time?
  2. Functional Programming in C# – Higher-Order Functions

This is the second in a series on the basics of functional programming using C#. My topic today is one I touched on last time, when I described the rights and privileges of a function as a first class citizen. I’m going to explore Higher-Order Functions this time. Higher-Order Functions are functions that themselves take or return functions. Meta-functions, if you like.

As I explained last time, my programming heritage is firmly in the object-oriented camp. For me, the construction, composition and manipulation of composite data structures is second nature. A higher-order function is the equivalent from the functional paradigm. You can compose, order and recurse a tree of functions in just the same way as you manipulate your data. I’m going to describe a few of the techniques for doing that using an example of pretty printing some source code for display on a web site.

I’ve just finished a little project at Readify allowing us to conduct code reviews whenever an interesting code change gets checked into our TFS servers. A key feature of that is pretty-printing the source before rendering it. Obviously, if you’re displaying XHTML on an XHTML page, your browser will get confused pretty quickly unless you take steps to HTML-escape all the XHTML entities that might corrupt the display. The examples I’ll show will highlight the difference between the procedural and functional approaches.

This example shows a fairly typical implementation that takes a file that’s been split into lines:

public static string[] RenderLinesProcedural(string[] lines)
    for (int i = 0; i < lines.Count(); i++)
      lines[i] = EscapeLine(lines[i]);
    return lines;

public static string EscapeLine(string line)
  Debug.WriteLine(“converting ” + line);
  return line.Replace(” “, ”  “)
      .Replace(“\t”, ”  “)
      .Replace(“<“, “<“)
      .Replace(“>”, “>”);

There’s a few things worth noticing here. In C#, strings are immutable. That means that whenever you think that you are changing a string, you’re not. In the background, the CLR is constructing a modified copy of the string for you. The Array of strings on the other hand is not immutable, therefore a legitimate procedural approach is to make an in-place modification of the original collection and pass that back.  The EscapeLine method repeatedly makes modified copies of the line string passing back the last copy.

Despite C# not being a pure functional programming language[1], it’s still doing a lot of copying in this little example. My early impression was that pure functional programming (where all values are immutable) would be inefficient because of all the copying goign on. Yet here is a common-or-garden object oriented language that uses exactly the same approach to managing data, and we all use it without a qualm. In case you didn’t know, StringBuilder is what you should be using if you need to make in-place modifications to strings.

Let’s run the procedural code and record what happens:

private static void TestProcedural()
   string[] originalLines = new string[] { “<head>”, “</head>” };
   Debug.WriteLine(“Converting the lines”);
   IEnumerable<string> convertedStrings = RenderLinesProcedural(originalLines);
   Debug.WriteLine(“Converted the lines?”);

   foreach (string s in convertedStrings)

Here’s the output:


As you can see, the lines all got converted before we even got to the “converted the lines?” statement. That’s called ‘Eager Evaluation’, and it certainly has its place in some applications. Now lets use Higher-Order Functions:

public static IEnumerable<string> RenderLinesFunctional(IEnumerable<string> lines)
    return lines.Map(s => EscapeString(s));

static IEnumerable<R> Map<T, R>(this IEnumerable<T> seq, Func<T, R> f)
   foreach (var t in seq)
     yield return f(t);

static string EscapeString(string s)
   Debug.WriteLine(“converting ” + s);
   return s.Replace(”  “, “&nbsp;&nbsp;”)
     .Replace(“\t”, “&nbsp;&nbsp;”)
     .Replace(“<“, “&lt;”)
     .Replace(“>”, “&gt;”);

private static void TestFunctional()
   string[] originalLines = new string[] { “<head>”, “</head>” };
   Debug.WriteLine(“Converting the lines”);
   IEnumerable<string> convertedStrings = RenderLinesFunctional(originalLines);
   Debug.WriteLine(“Converted the lines?”);

   foreach (string s in convertedStrings)

This time the output looks different:


At the time that the “Converted the Lines?” statement gets run, the lines have not yet been converted. This is called ‘Lazy Evaluation[2]’, and it’s a powerful weapon in the functional armamentarium. For the simple array that I’m showing here, the technique looks like overkill but imagine that you were using a paged control on a big TFS installation like Readify’s TFSNow. You might have countless code reviews going on. If you rendered every line of code in all the files being viewed, you would waste both processor and bandwidth resources needlessly.

So what did I do to change the way this program worked so fundamentally? Well the main thing was to opt to use the IEnumerable interface, which then gave me the scope to provide an alternative implementation to representing the collection. in the procedural example, the return type was a string array, so I was bound to create and populate the array before returning from the function. That’s a point worth highlighting: Use iterators as return types where possible – they allow you to mix paradigms. Converting to IEnumerables is not enough. I could change the signature of TestProcedural to use iterators, but it would still have used Eager Evaluation.

The next thing I did was use the Map function to return a functional iterator rather than a concrete object graph as was done in the procedural example. I created Map here to demonstrate that there was no funny LINQ business going on in the background. In most cases I would use the Enumerable.Select() extension method from LINQ to do the same thing. Map is a function that is common in functional programming, it allows the lazy transformation of a stream or collection into something more useful. Map is the crux of the transformation – it allows you to insert a function into the simple process of iterating a collection.

Map is a Higher-Order Function, it accepts a function as a parameter and applies it to a collection on demand. Eventually you will need to deal with raw data – such as when you bind it to a GridView. Till that point you can hold off on committing resources that may not get used. Map is not the only HOF that we can use in this scenario. We’re repeatedly calling String.Replace in our functions. Perhaps we can generalise the idea of repeatedly calling a function with different parameters.

Func<T, T> On<T>(this Func<T, T> f, Func<T, T> g)
    return t => g(f(t));

This method encapsulates the idea of composing functions. I’m creating a function that returns the result of applying the inner function to an input value of type T, and then applying the outer function to the result. In normal mathematical notation this would be represented by the notation “g o f”, meaning g applied to f. Composition is a key way of building up more complex functions. It’s the linked list of the functional world – well it would be if the functional world were denied normal data structures, which it isn’t. 😛

Notice that I’m using an extension method here, to make it nicer to deal with functions in your code. The next example is just a test method to introduce the new technique.

private static void TestComposition()
    var seq1 = new int[] { 1, 3, 5, 7, 11, 13, 19 };
    var g = ((Func<int, int>)(a => a + 2)).On(b => b * b).On(c => c + 1);
    foreach (var i in seq1.Map(g))

TestComposition uses the ‘On’ extension to compose functions into more complex functions. The actual function is not really that important, the point is that I packaged up a group of functions to be applied in order to an input value and then stored that function for later use. You might think that that’s no big deal, since the function could be achieved by even the most trivial procedure. But this is dynamically composing functions – think about what you could do with dynamically composable functions that don’t require complex control logic to make them work properly. Our next example shows how this can be applied to escaping strings for display on a web page.

void TestComposition2()
   var strTest = @”<html><body>hello world</body></html>”;
   string[][] replacements = new[]
       new[]{“&”, “&amp;”},
       new[]{”  “, “&nbsp;&nbsp;”},
       new[]{“\t”, “&nbsp;&nbsp;”},
       new[]{“<“, “&lt;”},
       new[]{“>”, “&gt;”}

  Func<string, string> f = x => x;
  foreach (string[] strings in replacements)
     var s0 = strings[0];
     var s1 = strings[1];
     f = f.On(s => s.Replace(s0, s1));


This procedure is again doing something quite significant – it’s taking a data structure and using that to guide the construction of a function that performs some data-driven processing on other data. Imagine that you took this from config data or a database somewhere. The function that gets composed is a fast, directly executable, encapsulated, interface free, type safe, dynamically generated unit of functionality. It has many of the benefits of the Gang Of Four Strategy Pattern[3].

The techniques I’ve shown in this post demonstrate some of the power of the functional paradigm. I described how you can combine higher order functions with iterators to give a form of lazy evaluation. I also showed how you can compose functions to build up fast customised functions that can be data-driven. I’ve also shown a simple implementation of the common Map method that allows a function to be applied to each of the elements of a collection. Lastly I provided a generic implementation of a function composition mechanism that allows you to build up complex functions within a domain.

Next time I’ll introduce the concept of closure, which we’ve seen here at work in the ‘On’ composition function.

Some references:

1. Wikipedia: Pure Functions

2. Wikipedia: Lazy Evaluation

3. Wikipedia: Strategy Pattern

Functional programming – Is it worth your time?

Short Answer: Yes!

Regular readers of the The Wandering Glitch know I focused lots of attention on LINQ and the new wave of language innovation in C# 3.0. I’m intrigued by functional programming in C#. At university, I focused on languages like C, C++, Eiffel and Ada. I’ve never since needed to learn functional programming techniques – who uses them, after all? Functional programming had always seemed like a distant offshoot of some  Bourbakiste school of mathematical programming unconcerned with practical issues of software development. Don’t get me wrong – I find that attractive, but it was always hard to justify the time, when there was so much else of practical worth that I needed to study. So the years passed, and I never came near. Functional programming was suffering from bad PR. But times change.

A fundamental change is under way in how we develop software. Declarative, Functional, Model-driven, Aspect-oriented and Logic Programming are all examples where new ways of representing and solving problems can pay huge dividends in programmer productivity and system maintainability.  Suddenly, it no longer seems that functional programming is a means to try out obscure new forms of lambda calculus. Now it seems that there are fast, powerful, easy to understand techniques to be learnt that will make my systems more robust and smaller.


I regretted not learning functional programming – I felt that there were ideas I was missing out on. And that made me envious. So, now is as good a time as any to address that deficiency. Another deficiency I want to address is the dearth of posts on the Glitch. I got tied up in producing a SPARQL tutorial for IBM which swallowed up my evenings. After that I had in mind to pursue an idea for a blog post on the relationships between LINQ, and Meta-mathematical structures like Groups and Categories. I got a major dose of intellectual indigestion, which stopped me from producing anything. The only way I’ll get productive again is to break the topics I want to cover into bite-sized chunks. that’s enough apologia – here’s the post.

Functional Programming is probably simpler than you think. It’s based on the idea that there is often very little distinction between programs an data. Consider this function ‘f’: 

f(x): x + 5

This function ‘f’ adds five to whatever you pass into f. What do I mean when I say ‘f’. I’m talking about the function, not using it. It came completely naturally for you to go along with me and describe the function ‘f’ as a thing. Here’s what I mean:

  g(f, x): f(x) + 7

This function ‘g’ adds 7 to the result of calling ‘f’ on x. So the final result would be ‘(x + 5 ) + 7’. You see, that wasn’t really a complex concept at all. Yet that’s the essence of functional programming. To put it another way:

Functions are first class citizens.

Which means that:

  • They can be named by variables.
  • They can be passed as arguments to procedures.
  • They can be returned as values of procedures.
  • They can be incorporated into data structures. [1]

It should also mean that you can compose your own functions as I did with ‘f’ and ‘g’ earlier. Another possibly less vital feature to empower this charter for the rights and privileges of functions is the ‘lambda’ (or λ) function. A lambda function is simply a way to create function on the fly, without having to give it a name. Compare this C# function:

int f(int x){return x + 5;}

With this one:

int f(int x)
  int c = 5;
  return x + c;

They both perform the same function, but the second one pointlessly created a name for the value ‘5’. The first example got by perfectly well without having to give a name to the value it was working with. Well, the same principle applies to lambda functions. Here’s a C# example that does what ‘g’ did above:

int g(Func<int , int> f, int x){return f(x) + 7;}

The ‘Func<int, int> f’ syntax is a new piece of C#, used to represent that f is a function that takes a single int and returns an int. you can probably already see that this function ‘g’ could be used with many different functions, but sometimes we don’t want to exercise our right to be able to name those functions with variables. To just create a function, without naming it (to use an ‘anonymous function’ in .NET parlance) you use the new lambda function syntax in C# 3.0:

int x = 3;
int z = g(y => y + 5, x);

‘g’ gets an anonymous function and an integer as parameter, runs the function with the parameter, adds 7 to what comes out of the function and then returns the result. Pretty cool. We’ve exercised our second right – to be able to pass functions into procedures. What about the first right? Well we sort of already had that with parameter ‘f’ in the function ‘g’ earlier. Lets look at another example:

int Foo()
  Func<int , int> bar = y => y + 5;
  // …
  return bar(56);

We’ve kept our function around in a format that is very flexible. It hovers in a middle ground between program and data. If, like me, you have a procedural and imperative heritage – you regard anything that you can store, return and pass around as data. But when you can run that data as code, then the lines begin to get a little blurred.

The next right that we need to claim is the ability to return functions as values. We have all the machinery needed to do that now. If we can pass something into a function, then we could pass it straight out again. If we can create lambdas we can return them rather than use them or pass them into other functions. Here’s an example based on the function ‘g’ earlier:

Func<int , int> H()
  return (int a) => a + 7;

This is powerful – rather than give you the result of adding a number to some value you pass in, this function gives you a function that you can use to perform the function. you don’t need to know what the function is, just how to run it. Sounds like a perfect recipe for business rules. Obviously, adding numbers like that is trivial, but the principle can be applied to functions of great complexity. This can be lazy too – you can provide a function to calculate the result when you need it and not before. Think LINQ to SQL queries, that don’t incur the expense of hitting the DB until necessary.

The last right needed to be a first class functional citizen is also achieved through the capabilities that have been explained already (in the case of C# at least). If we can create a function and assign it to a variable, then we can do the same to a compound data structure. Here’s a slightly more elaborate example (thanks to Paul Stovell for the idea):

public class MySwitcher<T , R>
Func<T , bool> Pred{get;set;}
Func<T , R> Iffer{get;set;}
Func<T , R> Elser{get;set;}

public MySwitcher(Func<T , bool> pred,
  Func<T , R> iffer,
  Func<T , R> elser)
  Pred = pred;
  Iffer = iffer;
  Elser = elser;
R Run(T input)
  return Iffer(input);
  return Elser(input);

This class keeps two functions around for later use. It also keeps a predicate function (a function that returns a yes/no answer) to decide which of them to use for a given piece of data. This could be used, for example, in a UI to decide between different ways to filter or render data based on some criteria.

I hope this very simple introduction shows you that not only does C# (and .NET 3.5 generally) now support functional programming, but that the arsenal of the functional programmer is very small and easy to learn. Next time around I hope to show you just how powerful these simple techniques can be.

[1] Abelson & Sussman: the structure and interpretation of computer programs. 2ed. MIT Press. 1998.

Setting up SDB on SQL Server 2005

imageFor the non-Java developer, like me, setting up and configuring SDB can be tricky. While it’s fresh in my mind I’ll commit it to blog so that those who come after will not tear their hair out like I just did.

SDB sits below Jena providing it with database support for a triple store rather than the more restrictive RDF XML file. The benefits of using a database are obvious, but the difficulties are not. the following instructions assume that you are running JRE 1.6, Vista and Cygwin. You need Cygwin to be able to run the batch files that come with SDB. You could probably adapt them to work on DOS, but Cygwin allows you to run the scripts as is.


  1. Get a copy of SDB & put it somewhere handy (I chose C:\etc\downloads\dev\semantic-web\sdb-1.0)
  2. Get a JDBC driver for SQL server.
    1. copy it to the lib dir under SDB. I’m not sure why, but I had to do this  despite the fact that there is an environment var that ought to be enough.
  3. Create a database for your triple store (I called mine TestTripleStore)
    1. Remember that you must use binary collation (I chose Latin1_General_BIN).
    2. Create a user and login for the new database (I chose sdbuser).
      1. Login as that user using SQL Management Studio – in case you need to change the user credentials on first login.
    3. Assign the new user ownership of the new triple store database.
    4. Setup TCP/IP access on your database server.
  4. setup the following environment variables using your .profile or .bashrc file (remember to source the file before you try to do anything)
    Note the use of DOS/Windows style path declarations despite the use of Cygwin – this one always catches me out.
  5. write the SDB configuration file (sdb.ttl) and place it in the $SDBROOT directory. This is straightforward – just modify the server and database name as appropriate:
    @prefix sdb:     <; .
    @prefix rdfs:     <; .
    @prefix rdf:     <; .
    @prefix ja:      <; .

    # Microsoft SQL Server

    _:c rdf:type sdb:SDBConnection ;
        sdb:sdbType        “MSSQLServer” ;
        sdb:sdbHost        “localhost” ;
        sdb:sdbName        “TestTripleStore” ;
        sdb:driver         “”

    [] rdf:type sdb:Store ;
        sdb:layout         “layout2” ;
        sdb:connection     _:c ;

  6. Create the triple store for SDB:
    sdbconfig –sdb=sdb.ttl create

If all goes according to plan, you should get no errors and the following tables in SQL server.


The Ambient Context Design Pattern in .NET

For a piece of agent related work I’m doing at the moment I am making heavy use of multi-threaded development. I’m developing a little special purpose Agent Framework to manage some data that I maintain. As part of that work, I need to have an ambient context object to hold details about the currently active agent and the tasks that it is performing. This is a common pattern that we see used throughout the .NET framework. They’re a powerful mechanism to keep useful data around, to define scopes and to provide cross-cutting capabilities. They provide functionality and a non-intrusive management mechanism without having to clutter the components that need them with additional parameters or static variables. In effect they are a form of controlled global variable that exists to maintain scoped information.

Since I haven’t seen this pattern documented in any detail elsewhere, I thought I might make a first attempt to describe it in pattern language terms. in what follows, I’ll try to stick to the Gang of Four (GoF94) format wherever possible, but I may make a few digressions for the sake of drawing parallels with comparable facilities in the framework (.NET 3.5). I’ll also highlight when one of the characteristics I describe is not a universal feature of a context class, but is commonly enough used to be worth a mention.

Continue reading

An Introduction to LINQ at Victoria.NET Dev SIG Tonight!

I’ll be doing a short introduction on LINQ tonight at the Dev SIG of Victoria.NET in Collins Street. I only have 20 minutes to talk so I won’t be able to go into too much depth. What I do hope to do is persuade the audience that LINQ is worthy of their attention. If I can persuade a few people to go out and develop their own query providers, then even better.

It’s an over capacity crowd, and they’re not taking any more RSVPs. If you haven’t booked yet, don’t despair – I’m sure that there’ll be more good stuff coming down the line.

Wish me luck – it’s my first presentation for Victoria.NET.