Kristian Kristensen’s Blog

I’ve been looking into IronRuby and how to retrieve the Abstract Syntax Tree (AST) that’s used. In this post I’ll show a simple example of how you can do the same.

First let’s look at what a compiler is and how it works. A compiler basically works in three phases: Lexical Analysis, Syntactic/Semantic Analysis and Code Generation. Given a piece of code the compiler first tokenizes the input. This means splitting the code into its simplest parts and it’s the Lexical Analysis phase. Examples of these parts could be a variable name, a string, an assignment operator (such as = in C#). The output from the Tokenizer is feed into a Parser in the Syntactic/Semantic Analysis phase. The Parser takes the individual parts – the tokens – and creates a generic representation of the code. This representation is the Abstract Syntax Tree (AST). In the last phase (Code Generation) the compiler uses the AST to generate code that can run on the target platform.
Usually each of these phases has multiple sub phases that further helps to validate, optimize, etc. the code.

If we take a modern language such as C# the input will be C# code in a .cs file and the output will be IL code that can run on the Common Language Runtime (CLR).
Now let’s take a look at an example of how the AST might look like for the simple assignment shown in the following code segment:

a = 2 + 3

After being parsed this code might have an AST representation as shown in the following:

  =
 / \
a   +
   / \
   2  3

The parts shown in the tree are called nodes, and there are 5 in total. A variable name, two integers (or literals as they’re called), a plus sign (or more specifically a Binary Expression) and an Assignment (the “=”).

Getting the AST is a first step in trying to analyze the code yourself. Since creating a tokenizer and parser for a language such as Ruby is a non trivial effort, it’s great if you can piggyback on others work. And for looking at Ruby code on .NET IronRuby is obviously perfect.

In the following code I’m using IronRuby 1.0 RC2, which is the latest release.
First we need to add a couple of assembly references, which you can find where you installed IronRuby (usually “c:\ironruby\bin\”). We need IronRuby.dll, IronRuby.Libraries.dll, Microsoft.Dynamic.dll, Microsoft.Scripting.dll, Microsoft.Scripting.Core.dll, Microsoft.Scripting.Core.dll, Microsoft.Scripting.Helpers.dll

Then import the following namespaces:

using IronRuby.Builtins;
using IronRuby.Compiler;
using IronRuby.Compiler.Ast;
using Microsoft.Scripting;
using Microsoft.Scripting.Hosting.Providers;

Next we’ll setup an IronRuby runtime and engine, this is the core infrastructure we need to work with Ruby code. Last we’ll create an instance of the ScriptSource class with a simple piece of Ruby code that calls “puts” with “hello” as a parameter.

var runtime = IronRuby.Ruby.CreateRuntime();
var engine = runtime.GetEngine('rb');
var src = engine.CreateScriptSourceFromString(@"puts 'hello'");

The ScriptSource is actually executeable, so if you called the method Execute() the simple Ruby expression would be executed. This is the basic pattern you’d follow if you wanted to add scripting capabilities to your application.

Using a helper class in the Dynamic Language Runtime (DLR) API’s we can access the SourceUnit instance. We create a new IronRuby Parser and ask it to parse the code found in the SourceUnit instance.

var srcUnit = HostingHelpers.GetSourceUnit(src);
var parser = new Parser();
var srcTreeUnit = parser.Parse(srcUnit, new RubyCompilerOptions(), ErrorSink.Default);

To access the AST we need to implement a Walker. This class implements the Visitor Design Pattern, and using it we can look at each of the nodes in the AST. A simple walker is shown below.

public class MyWalker : Walker
    {
        protected override void Walk(MethodCall node)
        {
            Console.WriteLine("Method call: " + node.MethodName);
            base.Walk(node);
        }

        protected override void Walk(StringLiteral node)
        {
            Console.WriteLine("String Literal: " + node.GetMutableString(RubyEncoding.Default).ToString());
            base.Walk(node);
        }
}

We kickstart the process by calling the Walk() method on our Walker implementation and gives it the SourceUnit as input.

var walker = new MyWalker();
walker.Walk(srcTreeUnit);

For each node type that the AST can contain a method is called on our Walker, and we can then do what we want. In this example I print the current node to the console.

The output of running this code is:

Method call: puts
String Literal: hello

The next step is to implement the rest of the methods on the walker, thereby supporting the full set of AST nodes. Also you’d probably want to build up your own data structure for analysis.
This example showed how you can access the parsed Ruby code using IronRuby. The next step is to actually do something interesting with this knowledge.

As part of my current project we’ve upgraded from Microsoft Team Foundation Server (TFS) 2008 to TFS 2010. Part of the project gungho is to illustrate the status of the current build. Hence we need some sort of programmatic way of accessing the build server history of TFS. This part has obviously changed  from 2008 to 2010, but also from 2010 Beta 2 to the Release Candidate. So I thought I’d detail how you can connect to the build portion of TFS 2010 RC.

First you want to add references to the new TFS SDK dll’s. They’re located in “C:\Program Files\Microsoft Visual Studio 10.0\Common7\IDE\ReferenceAssemblies\v2.0″. You’ll need:

• Microsoft.TeamFoundation.Build.Client
• Microsoft.TeamFoundation.Build.Common
• Microsoft.TeamFoundation.Common
• Microsoft.TeamFoundation.Client

We’ll need the following namespaces included:

using Microsoft.TeamFoundation.Build.Client;
using Microsoft.TeamFoundation.Client;
using Microsoft.TeamFoundation.Framework.Client; 

Then setup the connection to your TFS server and make sure we authenticate:

Uri tfsUri = new Uri("http://tfsserver:8080/tfs/");
var tfssvr = new TfsConfigurationServer(tfsUri, new UICredentialsProvider());
tfssvr.EnsureAuthenticated();

A new feature in TFS 2010 is project collections, that group team projects. Therefore we need to locate the project collection where our team project is located. I know that our collection is called “Default” so I just look for that.

TeamProjectCollection tpc = null;
var collections = tfssvr.GetService<ITeamProjectCollectionService>().GetCollections();
foreach (var teamProjectCollection in collections)
{
    if (teamProjectCollection.Name.Equals("Default"))
    {
        tpc = teamProjectCollection;
    }
}

Next we retrieve a reference to the build server service running inside our project collection. Using this we create reference to the build queue for our specific team project. We can filter this build queue so we only get information about builds in progress and builds that are succeeded. We can also hook up an event handler so we’ll be called whenever there are changes to the build queue; we’ll look at this event handler in a minute. Finally we connect to the build queue, thereby setting up our callback.

IBuildServer buildServer = tfssvr.GetTeamProjectCollection(tpc.Id).GetService<IBuildServer>();

var queue = buildServer.CreateQueuedBuildsView("MyProject");
queue.StatusFilter = QueueStatus.InProgress | QueueStatus.Completed;

queue.StatusChanged += new StatusChangedEventHandler(queue_StatusChanged);

queue.Connect();

The event handler that’ll get called whenever there’s a change in the build queue is located below. We cast the sender to the right interface so we can access the queue. We verify that there’s indeed a change in the build queue, and that we’re not just being pinged. We extract the latest build (the list is sorted by TFS), and then we simply print out some information about the build.

static void queue_StatusChanged(object sender, StatusChangedEventArgs e)
{
  var queue = (IQueuedBuildsView) sender;
  if (e.Changed && queue.QueuedBuilds.Length > 0)
  {
    var curBuild = queue.QueuedBuilds[queue.QueuedBuilds.Length - 1];
    Console.WriteLine("New build detected: " + curBuild.Build.BuildNumber + " is " + curBuild.Build.Status.ToString());
}

The interesting use case for this is of course to hook in your own customer build notification system in the event handler. In my current project we have a Nabaztag bunny, that’ll inform the team when a new build is in progress and when it’s finished. You can see a video of it on Youtube – IronBunny.

Ewald Hofman pointed me in the right direction with his posts on how the SDK works in TFS 2010 Beta 2.

Disclaimer: This code is just an example, and you really shouldn’t put it blindly into production. Basically the usual “Works on my machine”.

I held a TechTalk at the Microsoft office on Wednesday, and have promised to post the slides. So here goes.

Slides @ slideshare.net

The video I showed of the Nabaztag bunny being connected to a TFS server is embedded in that deck. However, you can also view it directly on YouTube.

IronBunny @ YouTube

If there’s interest, I’ll post my demos up here as well.

I’ve had a couple of links lying around in my blog reader. I’d planned to blog about each of them, perhaps coupling them. Since they’ve been there for quite a few weeks and I still haven’t made a post, I guess I should just list and jot a few comments down for each.

Greenspun’s Tenth rule Any sufficiently complicated C or Fortran program contains an ad hoc, informally-specified, bug-ridden, slow implementation of half of Common Lisp.
Cite: wikipedia

I believe I’ve picked this up via Reg Braithwaite and his regular writings on programming languages and all things software dev. Dunning-Kruger effect The Dunning-Kruger effect is the phenomenon wherein people who have little knowledge (or skill) tend to think that they know more (or have more skill) than they do, while others who have much more knowledge tend to think that they know less.
Cite: wikipedia

Probably via Reg Braithwaite as well. It’s a pretty interesting hypothesis, and fits well with the sayings “The more you learn, the more you realize what you don’t know” and “Ignorance is bliss”. It also fits with a previous pots of mine The Mythical 5%, where I reference Bruce Eckel and Joel Spolsky. Sapir–Whorf hypothesis postulates a systematic relationship between the grammatical categories of the language a person speaks and how that person both understands the world and behaves in it.
Cite: wikipedia

I heard this a RubyFools where Matz referenced the theory during his key note. I’ve actually been thinking about this before, and wondered if there was any science behind it. My initial thoughts on this were also related to programming languages, but since PL’s and natural languages share many similarities it’s interesting to extend it. In speaking with my father about it he also noted whether or not it is a coincidence that so many philosophers are German. Does the German language with its strict strucutre and grammar promote a special way of thinking that tailors well to philosophy. An example of a German philospher could be Immanuel Kant.

Also I remember from my first year of university they wanted to teach us SML instead of a more mainstream language such as C and Pascal. One of their motivations was that by teaching students such a language first they would develop a different style of thinking compared to their peers who might have pre-university experience in C or another language. So does the first language you pick influence the way you think about programming for the rest of your life? And does self learning it impose a certain “dirty” way of doing things compared to being taught academically?

Either way, there you go a few thoughts to think about on a Sunday. Please feel free to comment below.

Min sidste post sluttede da jeg var på vej i centret for at blive klippet. Det viste sig at være et rimelig smart sted. Man kommer ind og bliver straks vist hen til nogle prøverumsbåse. Her skal man så tage overtøjet og skjorte og lignende af og iføre sig en slags kimono. Og også hvis man er mand (for jeg spurgte nemlig). Herefter bliver man så tilbudt drikkevarer og kommer ellers over ti lden stylist dre skal klippe en. Hun var en vældig nydelig udseende ung dame. Charlie hed hun. Det gik vældig fin med at blive klippet omend det var i en lidt anden stil end jeg var vant til. Til gengæld kan man få gratis nakketrimning, så det liver jeg nok nødt tilat få. Primært for at jeg så kan få et billede af mig selv i den kimonolignende ting…
Bagefter var der velkomstreception. Blev hentet af shuttlebusser for at blive kørt 5 blocks op til Hyatt. Stile og roligt. Der var mam og fri bar. Derefter lidt velkomsthallj og nogle lege.

Mit team til en af legene. Gæt selv hvilken.

Vores holdt til maks belastning. Yeah!
Om aftenen spiste vi på Joey’s Grill som også var rimelig fancy. Der var en super hyper dame der rendte og åbnede døre for folk og spurgte om de havde haft en god middag. Derudvoer delte hun appetizere ud. Snedigt. Det var go mad, steak og friturestegt kartoffelmos. Ret godt faktisk! Derefter var det bare hjem og i seng.

Onsdag morgen var første kurusdag. Vi blev hentet i busser kl 7 om morgenen og kørt til Campus bygning 122. Der var morgenmad, klassisk amerikansk med æg, og bacon, etc. Jeg fik noget mega mærkeligt bacon, men kunstig bacon lavet af soya. Det kan ikke anbefales. Ellers var der foredrag hele dagen. Vi blev også præsenteret for de her nåle som man kan gøre sig fortjent til. Dme der var vandt konkurrencen fra dagen før ville allerede får en Teamwork nål, så så var jeg 1 oppe.

En af de interessante talere om onsdagen var Bob McDowell som startede Services i Microsoft. Han var en gammel knark som var rimelig sej til at underholde i en time. Godt indlæg! Om aftenen var der mad på en japansk restaurant hvor der bl.a. var sushi. så¨fik jeg prøvet det, og jeg må nok erkende at det faktisk er meget godt. Bagefter var jeg ude med nogle og gøgle lidt øl. Hyggelig aften.

Jeg har taget et par billeder af udsigten fra terrasen i lejligheden. Se dem i galleriet. Et pudsigt billede:

En hel butik kun med forlovelsesringe. Imponerende!

Så er der tid til Microsoft Services University (MSSU) som alle der bliver ansat i Services i Microsoft skal på. Det er et 2-3 ugers kursus hvor mna lærer grundlæggende ting om hvad det vil sige at være i Services. Hvordan Microsoft leverer, etc. Vi skal afsted alle 5 NG’ere (der er nemlig blevet ansat en ny til april). Kurset foregår i Redmond, Washington på Microsoft’s campus. Vi bor i Bellevue som er en forstad til Seattle og ligger tæt på Redmond. Hver morgen bliver vi afhentet i shuttlebusser kl 7, og bliver returneret sidst på eftermiddagen. Vi har fri om søndagen og har en friweekend i løbet af perioden.
Efter kurset har jeg et par mellemliggende dage hvor jeg besøger min onkel i Delaware, og så har jeg et kursus i Irvine, Californien omkring Biztalk Server. Så det bliver nogle spændende kommende uger!

Igår ud på eftermiddagen tog vi en taxa fra Hellerup til lufthavnen. Havde tjekket ind fra kontoret og fået hvad der så ud til at være nogle okay pladser. Vi skulle allesammen sidde på samme række. Stille og roligt med at tjekke ind og komme om bord på flytet bortset fra at jeg skulle have en itinerary med for at kommer ombord. Det kræver de åenbart ved indrejse. Har aldrig oplevet det før, men så ved man da det. Det viste sig at vores pladser var super. Række 23 er bare guld på sådan en Airbus 340. Der var masser af benplads. Flyveturen gik som den gør, relativt kedeligt at sidde og koge i knap 10 timer. Me ndet gik nu meget godt. I Seattle skulle vi hente nøgler i “Ken’s Baggage and Frozen Food Storage”. Lidt mystisk sammenblanding måske, men vi fik vores nøgler. Mødte også to andre MSSU deltagere fra Finland og Sverige, så vi delte allesammen en taxa til Bellevue, WA hvor vi alle sammen skulle bo. Jeg skulle bo/bor med Christian, og vi blev smidt af foran Avalon Meydenbauer apartments. Og det viste sig at være nogle jævnt fede lejligheder!

Fra indgangen ind mod loungen hvor vi har en milliard tv-kanaler og internet.

Køkken med al gejlet

Mit værelse

Til hver af værelserne er der så badeværelse. Man kan jo ikke dele…

Egen vask og tumbel. Jow jow.

Christian og jeg crashede lidt og Rasmus kom ned. Så gik vi op i Bellevue Sq som er et stort shopping center i Bellevue. Jeg havde en plan om at skulle klippes, mne det viste sig at være sværere end som så. jeg fik dog bestilt en tid til tirsdag hos noget der hedder 7even, noget super fancy klippeting. Man betaler en pris afhængig af hvor fancy en stylist der skal klippe en. Jeg skal klippes af en der heddre Charlie. Først spurgte ekspedienten om jeg havde en favorit stylist, men det var vist ligemeget
Bagefter gik vi lidt rundt, gøglede i Apple butikken, og derefter fandt en restaurant hvor vi fik burgers og enchilades, plus noget lokalt ølbryg. Super. Derefter var det bare lidt almindelig fjernsynsafslapning inden en tidlig sengetid ved halv ti tiden.

Jeg sov rimelig godt og var “først” oppe halv syv, Christian derimod havde haft lidt sværere ved det og havde været oppe halv 5 og nede at træne. S åha nvar godt frisk da jeg stod op. Rasmus kom forbi og vi fandt en Denny’s et kvarters gåtur fra lejlighedne. Fik noget møgbeskidt morgenmad. Velkommen til. Da vi kom over til Denny’s bliver vi vist hen til en bås. Her sidder der en gut i båsen ved siden af med et Illustreret Videnskab, så jeg siger hej til ham. Vi snakker lidt sammen og det virker som om han kender os eller noget. Vi kan ikke rigtig finde uaf hvad der sker, men til sidste finder vi da ud af det. Det er kaptajnen fra vores SAS fly! Han tror vi var med som stewarder og at han bare ikke havde mødt os! Crazy tilfældigt. Vi snakker lidt og flytter så bås, så resten af besætningen kan sidde sammen. Totalt stenet.

Christian slapper af og ser ud ti lat kunne holde til det.

Rasmus og undertegnede.

Efter morgenmaden gik vi tilbage. Christian og Rasmus fik en dobbelt espresso på Starbucks. Christian og jeg gik også en tur i Bellevue parken. 

 så er det afslapning inden jeg skal klippes og lidt osen i centret inden vi bliver hentet af en bus halv 3 for at tage til en velkomstreception på Hyatt Regency i Bellevue.

Flere billeder i galleriet.

Last Friday (4th of April) I took the MCPD Web Developer exam. It went well with a very small margin. I scored 700 points, which is the exact required number of points needed to pass a Microsoft certification. This means I’m now an MCPD in Enterprise Application Development and Web Development. Very cool!

I also think it marks the need to ponder about my current certification status. Since I started at Microsoft in December I’ve taken 6 developer certifications. With respect to the multi-exam certifications this is probably as high as one can reasonably get. The MCPD is the largest or most inclusive developer certification in the MS Certification program if you disregard the Microsoft Certified Architect (MCA) program. The later requires a lot of things including meeting before a board and discussing a case. Suffice to say you’re not going to nail that one after 4 months on the job!

I think my next certification steps will be to take the Microsoft Solutions Framework (MSF) exam. After that it’s the infrastructure and server certifications I’ll focus on as part of the NG program.

In two weeks I’m going to Redmond, WA to attend Microsoft Services University (MSSU); a 2,5 weeks course that all new employees in Microsoft Services is attending. After that I’ll have a short 5 day break/vacation where I’ll visit family, and before I go home I have a Biztalk course in Irvine, CA. So a lot of traveling! I’ll be in the US for a month shy of a couple of days.

I’ll be attending Ruby Fools in Copenhagen this Tuesday and Wednesday. Add a comment if you’re going and want to meet up.

I’ve also created an event on Facebook for Ruby Fools. Feel free to join if you’re attending.

It’s been pretty quiet around here, but now I have something to write about. In my last post I wrote about passing the MCTS Distributed exam, and that this meant I was on track for the MCPD Enterprise Application Development exam. Well, this Friday I took the exam and passed with 860 points. So I’m now officially an MCPD. This page with statistics of the number of MCP’s worldwide states that there are 3424 persons that holds this certification. The numbers are from January 2008. Somehow that doesn’t seem like a lot of people.

Anyway, I followed the recipe that seems to work for me. Read the Self-Paced Training Kit book and do practice exams. I read MCPD Self-Paced Training Kit (Exam 70-549): Designing and Developing Enterprise Applications Using the Microsoft .NET Framework cover to cover, and did practice tests like MeasureUp.
For some strange reason I didn’t get a score report after my exam, so I can’t comment on how well I performed on different areas of the exam. However, I can comment a bit on the types of questions I was asked. Most of the questions involves a fair bit of text, stating requirements for a specific solution. The solution is then proposed and you need to answer if the solution fulfills the requirements and if not why not. It’s still multiple choice, but a tad more difficult than your standard techie questions. Mainly because they’re looking for a specific solution. And the answer is not always how you would go about doing it.
In summary after taking this exam I don’t find it particularly difficult if you’ve studied some kind of computer science or software engineering. A lot of this stuff is pretty basic, and easily approachable.

With the MCPD out of the way I don’t know what my next certification goals should be. Maybe I should focus on SQL Server or some Biztalk stuff. Maybe focus a bit on the Infrastructure and Server exams, which I’ll have to go for later this year anyway. I’ll keep the blog posted.

Shortly after my last post (on January 24th, yikes) I left with the 3 other NG’s for Prague. We were going to particpiate in MACH Intro School for EMEA. I wrote about about the connection between NG and MACH in a previous post. The Intro School is the first chance to meet your peers in your closest region. Denmark belogs to Western Europe (WE) which is part of Europe, Middle East and Africa (EMEA). This meant that there were a lot of different cultures and languages convening in Prague. The purpose of Intro School is to give you guidance and skills to tackle your work life at Microsoft, and networking. Actually the latter should probably be written as NETWORKING! And so we did I had a blast talking and just goofing around with the other MACH hires. Plus 2 weeks in Prague doesn’t kill you either. They’ve got good beer and good food. Awesome!

After Prague I started prepping for the exam Microsoft .NET Framework 2.0 – Distributed Application Development (70-529), and deals with Web Services, Remoting, WSE 3.0, Enterprise Services, and MSMQ. My strategy was the same as for the other exams, read the book, do practice tests, iterate on my weak points. And sure thing it worked! I passed on the 15th with a score of 965. Pretty cool.
This means that I’m now a Microsoft Certified Technology Specialist (MCTS) in Web Apps, Windows Apps and Distributed Apps. But more importantly I’m on track to become a Microsoft Certificed Proffesional Developer in Enterprise Applications Developer. I need one more exam – the PRO: Designing and Developing Enterprise Applications by Using the Microsoft .NET Framework (70-549) - to achieve the title. So that’s definitely my goal to accomplish that within the coming couple of months.

Followers of my Delicious feed will notice that I do a bit of work on Forefront, which is Microsoft’s Anti Malware software suite.

Last week I attended internal training on Microsoft Solutions Framework (MSF), which is Microsoft’s model for delivering solutions be it software development or infrastructure setup. It was good because it conveyed the language used internally when talking about which phase a project is in, or what milestone has been reached. I’ll definitely benefit from it.