Category: Tech

A quick recap. So far we have made sure we have a good source control environment and have created a Visual Studio Solution with a well structured folder setup. But we haven’t checked those files into our Source Control server yet – we’d better fix that.

Your Source Control administrator will probably tell you where to make your initial check-in of your new project, but I suggest you think about simplicity for a moment:

• If you’re using Perforce, consider using 1 depot as the ‘server side’ equivalent of your meta root. You don’t lose any security options, and you gain in that developers may already have this depot mapped in their client so won’t need to change any source control configuration.
• If you’re using Subversion, just use one repository for all the projects in your department (see here for a good explanation why.) Use a new directory for your new project (and probably check it in to a ‘trunk’ sub-directory, but you can always move it later.)
• If you’re using CVS, its fairly standard to create a new CVSROOT for each project, and I would recommend it. Note that you’ll have to setup any extra permissions and triggers that you use as standard. I’ve seen organisations make good use of GForge to manage their CVS server.
• For other Source Control systems, follow similar guidelines

Once you’ve figured out the source control location of your new project, don’t be too hasty about checking in. Its worth taking a moment to decide what you actually want to check in. Files you don’t want to include are:

• Build output folders – don’t check in the bin or obj sub-folders of your VS project folders
• Any Solution .suo or VS Project .user files – these are user and environment specific and should not be checked in
• Any Resharper, or other third-party tool output. (Resharper generates a SolutionName.resharperoptions file and a _ReSharper.SolutionName folder, neither of which you need to save)

Not checking these files in is good, but making sure no-one else ever checks them in later by mistake is even better. CVS and Subversion both offer such functionality through .cvsignore files and svn:ignore properties respectively. With Perforce, you can use Triggers, but this is not as elegant a solution.

Moving back to our Sycamore example, I’m going to use a Subversion server to check in our work. First of all I delete all the temporary files we discussed above. Then I’m going to use the svn command line tool, but you could use TortoiseSVN or AnkhSVN instead. My command line looks like:

c:develsycamore>svn import -m "Initial Sycamore Import" . file:///c:/svn-repos/sycamore/trunk

Once the intial checkin is complete I’m going to delete my ‘sycamore’ folder and then checkout from Subversion the folder we just imported to get a local versioned folder. After that I reload the solution in Visual Studio and compile. This recreates the temporary files.

I then set the svn:ignore value for src to be *.suo, ReSharper.Sycamore and *.resharperoptions. The svn:ignore for VS Project dirs should be set to *.user, bin and obj. You should be able to test you’ve captured everything by doing a svn status in the root folder and only seeing output for merging the properties of the src and VS Project directories. Make sure to commit these property updates.

To see exactly the state of Sycamore as it currently stands, download a zip file from here.

To summarise this part:

• Pick a Source Control location that is simple for everyone to use.
• When checking in your project directory, make sure not to include build artfacts or temporary environment files.
• If possible, configure your Source Control to make sure no-one can check in such files in the future.

In the next part we’ll be adding an automated build for our project using NAnt.

In Part 1 we looked at making sure we had our Source Control story straight. With that sorted out, we can start creating some files to put in it.

First some terminology, I’m going to use the word ‘Project’ to define the thing that all the files in our development tree go to make up. It is more than just a Visual Studio Project. I’m going to use an example project called Sycamore.

Next, I’m going to assume you are using Visual Studio 2003. Pretty much everything we are going to look at will work without Visual Studio, but I’ll assume you have it anyway.

First, we want to make a new folder. We’ll put it in our meta root – a place where you check projects out from Source Control. On my machine, this folder is C:\devel but on your machine (and anyone else’s) it might be different. You should never assume the concrete location of meta roots.

Call the new folder pretty much the same thing as your project. I say ‘pretty much’ since I like to remove capitals and spaces, but its really up to you. Our folder will be called sycamore. It is the root of our development tree. All source code for this project will exist somewhere under this root. Any tool or library dependencies that exist outside the scope of this root will have to be managed carefully. There will be no source under this root that belongs to any other project.

To start with we are just going to create a Visual Studio-compilable ‘solution’. A solution contains source code, so we’re going to create a sub-folder of sycamore called src. We will have other sub-directories later, but they will contain other things, so its good to separate the source out into its own location.

In src we create our new Visual Studio solution. To make things easy, I’m going to call it Sycamore. Unfortunately, Visual Studio doesn’t make it easy – it wants to put it in a another sub-folder called Sycamore so once I’ve created an empty solution I’ll close it down and move the Solution.sln file into the src folder. We can delete the extra Sycamore folder that Visual Studio created for us.

Next to create some VS Projects. In this part, I’m going to keep it simple – just a single command line application in one project. We’ll be creating some more projects later. I like projects to have their own folder under src. We never merge VS projects into one folder and never put their ‘project roots’ anywhere other than src.

In Visual Studio I create a C# Console Application called SycamoreConsole. Its location on my machine is C:\devel\sycamore\src, but the location on your’s will depend on your meta root. VS creates the project for us and creates a class called Class1.

We’re also going to change some of the project properties. First the Assembly Name. In later parts we’ll talk about dll names, but for console applications, pick something short and obvious. We’re going to call our’s sycamore. For the Default Namespace, I like to use the convention OrganisationName.ProjectName.VSProjectName, so in our case I’m going to use the Namespace SherwoodForest.Sycamore.SycamoreConsole. Save these properties and go back to the Class1 window.

First, set the namespace to be as you just set in the Project properties. Now rename the class to something sensible (we’ll use HelloWorld for now), and don’t forget to rename the file to match. (I recommend you use ReSharper which will do the file rename for you.) I also like to delete all the unnecessary comments. Sticking with tradition we’ll add the statement Console.WriteLine("Hello World"); to the main method. Compile, run and make sure everything does as expected.

We’re done for now. We may be making baby steps, but we are already seeing some defintions and patterns emerge:

• The root is the upper most point of our development tree.
• All files belonging to a project exist under the root.
• No files belonging to any other project exist under the root.
• The root itself is resident in a meta root which can change from machine to machine.
• All source code resides under a src sub-folder.
• The project .sln file is saved in the src folder.
• All Visual Studio Projects exist in their own sub-folders under the src folder.
• Visual Studio Project folders are atomic, and should be named identically to the project they contain.
• The default namespace for a Visual Studio project should be OrganisationName.ProjectName.VSProjectName.

In the next part we’ll look at what we have to do to get this project into Source Control.

So, lets start building our development tree. Feel free to join in. 🙂

The first thing you need is a Source Control environment. This may sound simple, but even at this stage I have seen some strange things happen on projects.

Here are some ‘must haves’:

• Your source control server must be fast. Your developers are often going to be waiting for your source control to do things, so don’t scrimp on hardware. Specifically:
  • Do use decent, modern, hardware.
  • Don’t use network shares to store files – in my experience it will slow your source control by about 10x. Instead invest in some locally redundant disks (RAID 5 is OK, RAID 0+1 is better), and a backup strategy.
  • Don’t put your Source Control server on the other side of the world from your team. Keep it local, and make sure your network isn’t getting bogged down. Obviously with distributed teams this may not be possible, but if your team isn’t distributed, don’t distribute your hardware.
• Don’t be tight on hard disk space. Get about as much as you think you might need in 3-5 years. Disk space really is cheap and having lots of it means that people can worry about producing software, and not about whether they are going over quota.
• Give developers write access to the code they need to work on. If you trust them to write code, you should trust them to be able to edit their own work without having to go through slow processes. Other team’s code may be a different matter.
• Put each development tree in its own folder under source control – don’t try and ‘save’ work or space by merging them. It really will save you headaches and time. See the ‘hard disk space’ point.
• Make sure new source control clients can be set up fast and correctly. Document what needs to be done for each project on a Wiki. If your Source Control Client setup takes over 10 minutes, or is more than a page of manual work, change it. If necessary throw away your current Source Control software and start again.
• Make sure basic source control operations are quick, simple and well understood. All developers should be easily able to do all of the following operations – if they don’t know how or if these processes are cumbersome or slow to execute, then change them (again, if necessary consider changing your Source Control software)
  • Check out from nothing
  • Get updates
  • Find differences between server and local versions
  • Revert local versions
  • Commit changes
• Your Source Control system must be consistently trustworthy – if developers are losing changes or files are becoming corrupted, fix it.
• Your Source Control server should support the following more advanced operations, which developers should be able to perform if necessary:
  • Labelling (tagging)
  • Branching (parallel, independent, development of integratable code lines)
  • Automation (be driven by a process, not just a person)

The above points I believe are all necessary for an effective development project. For an ‘excellent’ project I recommend the following:

• A good source control server can happily accomodate 100 developers. I recommend the following kind of system:
  • UNIX/Linux based – Most good Source Control software is written primarily for a UNIX/Linux environment so don’t support edge cases.
  • At least dual-CPU (I like the idea of one CPU being able to do work, and one doing I/O, but I’m sure that’s rather a simplistic model these days)
  • At least 1GB RAM – if your often-accessed source is already cached you should get a speed up.
  • Don’t run anything else on the machine apart from a Source Control server. If you do (e.g. source control reporting), invest in extra processors and monitor what impact those extra applications are having.
  • Use 1 disk set for applications and checkpoints/journals, and a separate disk set for your actual data.
• If cash is fairly easily available in your organisation, use Perforce. I’ve been using it on an off for 4 years now and it never ceases to amaze me how fast and stable it is. It also requires almost zero maintenance.
• Otherwise use Subversion. It is free, and better than any other SCM system I’ve tried apart from Perforce.
• If you are using Visual SourceSafe, I strongly urge you to migrate away from it. It is renowned for not being scalable and is also prone to file corruption. If you are not experienced with UNIX, or any other SCM tool apart from VSS, I have heard good things about SourceGear Vault.
• Use clean and simple setups for your ‘meta’ trees. In Perforce, putting all projects in one ‘depot’ is perfectly reasonable, and use similar ideas for other tools.

If after reading all of this you are thinking ‘Nice ideas, but we don’t have the time or money to do any of this’, then think how much it would really cost you to (say) invest in a new Linux server and Subversion, and how much money you are losing through lack of productivity. Its also a lot simpler than you think. Why not try out Subversion for half a day with a good book?

In the next part we’ll start looking at some code – stay tuned!

In the last few weeks I’ve setup 2 brand new .Development Trees for .NET projects. What do I mean by development tree?

• It is a directory structure
• containing:
  • source files
  • tools and dependencies
  • references to external tools and dependencies
• checked into source control
• that is atomically integratable
• to produce a set of artifacts

A good development tree should:

• be easily integratable on new environments
• require little maintainance
• but be easily maintainable when it does require maintenance
• support, but not hamper, developer productivity
• have consistent behaviour

This is all a bit wooly, but will do for an intial stab. I might come back and refine these points later.

Anyway, I’ve setup quite a few development trees in my time, in Java and .NET. In this series of blog entries I hope to develop a good ‘boilerplate’ development tree structure for .NET projects that other people can use.

If you find it interesting, please email me with your comments.

I’ve recently started using NVelocity. Its a brilliantly simple and powerful templating engine that’s been ported from the Java world.

I was going to explain how it works, and how to use it in your app, but its easier just to show you a test and an implementation.

[TestFixture]

public class MyVelocityTransformerTest

{

[Test]

public void ShouldUseVelocityToMergeContextContentsWithTemplate()

{

Hashtable contextContents = new Hashtable();

contextContents["foo"] = "bar";

MyVelocityTransformer transformer = new MyVelocityTransformer();

Assert.AreEqual("foo is bar", transformer.Transform("testTransform.vm", contextContents));

}

}

public class MyVelocityTransformer

{

public string Transform(string transformerFileName, Hashtable transformable)

{

VelocityEngine engine = new VelocityEngine();

engine.SetProperty(RuntimeConstants_Fields.RUNTIME_LOG_LOGSYSTEM_CLASS, "NVelocity.Runtime.Log.NullLogSystem");

engine.SetProperty(RuntimeConstants_Fields.FILE_RESOURCE_LOADER_PATH, "templates");

engine.SetProperty(RuntimeConstants_Fields.RESOURCE_MANAGER_CLASS, "NVelocity.Runtime.Resource.ResourceManagerImpl");

engine.Init();

string output = "";

using(TextWriter writer = new StringWriter())

{

engine.MergeTemplate(transformerFileName, new VelocityContext(transformable), writer);

output = writer.ToString();

}

return output;

}

}

This assumes that a file called ‘templates\testTransform’ exists with the following contents:

foo is $foo

The benefit to NVelocity is it makes generating string content much cleaner than any other method I’ve seen. Times when you would want to use it include code generation, HTML generation, etc.

The velocity template language is rich enough to make it more powerful that using (say) string.Format() – the #if and #foreach directives are especially useful.

As with all tools and libraries, NVelocity should be used where appropriate. I’ve seen Java projects with hideously complex, un-unittested templates which are incredibally fragile and hard to debug. My advice is to keep your templates relatively small, and only use directives to just go below the surface of the objects in your context. If necessary, introduce presentation objects and create them in normal, unit testable, code.

Update

Please see the separate page in my articles section for this.

Its all Mike Mason‘s fault. He sent me an email a few months ago asking me to review his new Subversion Book. Little did I know it was actually a ploy to turn his former bash-savvy, now cheese-eating Windows GUI dependent, sparring partner back to the realm of the $ and # ….

So on a slightly more serious note… Mike’s book, like the rest in the Pragmatic Starter Kit, uses the ‘command line’ version of tools to explain concepts and usage. I was a bit worried about this at first. I’ve become used to my Tortoise SVN, my P4Win and my Eclipse CVS plugin, why did we need to go ‘back to basics’? Anyway, Mike’s a mate so I carried on through to chapters 2, 3 and onward.

I don’t know what happened in those hours of reading but when next I went to access a Subversion server the following week, I found TortoiseSVN clunky to use. Now, don’t get me wrong, Tortoise SVN is a great SVN UI client, but therin lies the problem – its a UI client. I’d got used to the speed of doing a ‘svn stat’ (tell me what’s changed), or an ‘svn stat -u’ (tell me what’s changed, including any updates to the server). In Tortoise I have to navigate explorer to the right folder, right click a folder and then select .. umm, what is it again?

I always used to use command prompt tools but in recent years I’ve got lazy, and its not a good type of lazy. Its the lazy of living with a broken window, of learning ‘just enough to solve the current problem and no more’. The command line takes that little bit more effort to get right initially, but what I’d forgotten was that with a decent command line application you are repaid your efforts later. Combine that with a little scripting knowledge and you can start plugging tools together and really start becoming truely lazy, the good lazy of being able to sit back and drink your coffee while a computer does all the repetitive work that you would normally do repeatedly yourself.

So am I forced to become another member of the Mac-owning Masses to re-embark on my journey to shell nirvana? Not at all. With Cygwin I have a fully featured BASH shell on my Windows machine (and the Windows command prompt (cmd.exe) isn’t all that bad for getting started). VBscripters or Javascripters can also use Windows ‘cscript’ scripting host. For build scripts, NAnt allows you to interpret C# or VB.NET at run time through the <script> task. Finally .NET applications have full abilities to launch processes and access the standard streams.

And have I actually done anything beyond using command-line svn? Yep, you’re reading it. I used to upload my blog content using WinSCP, but now I use rsync, running on Cygwin, to upload the differences in my locally-generated blog pages to a UNIX server. It requires about 10% the of the hand movement on my part and completes in about 10% of the time. My next plan is to put a command line wrapper on to my blog application, wrap the whole process up in a script and run it from a CruiseControl.NET project, running under Mono, monitoring the Subversion server where my blog source content is hosted. Then updating my blog will be as easy as adding a new file to a Subversion repository (and with Mozilla Composer + mod_svn, that’s a trivial task.)

Oh, and maybe I’ll buy a Mac Mini anyway. 🙂

I haven’t been blogging much recently, but here’s excuse to start off again – CruiseControl.NET 0.8 is released!

There’s a couple of breaking changes – make sure to read the release notes. The biggest thing for me about the release is we are now recommending the ‘Web Dashboard’ over the old ‘single project’ web app. I think this is great since people now only need 1 web app instance to monitor and report all of their CCNet projects across all their build servers!

There’s a whole bunch more things coming to CCNet too, but I’ll mention them once they’re released. 🙂 For now, go and get the 0.8 release and try it out!

Its time to say goodbye to my domain names.

5 years ago I bought the domain name ‘transmorphic.co.uk’. At the time I was into reconfigurable computing, the idea of re-defining the hardware in a computer. ‘Transmorphic’ was a made up word, meaning (in my mind) ‘changing’ or ‘beyond’ (trans) ‘shape’ (morphic).

About a year later I became more vain, and wanted my own .com . ‘transmorphic.com’ had gone, but ‘tmorph.com’ was available, so I grabbed it. Apart from anything else it was a good place to host my CV. 🙂

For over a year now though I haven’t been advertising my domain names. Instead I’ve been using mikeroberts.thoughtworks.net for my web idendity and since Gmail my Gmail address for all personal email. Because of this, I’ve allowed my domains to expire. I’ll probably get another one in the future, but for now please use these internet identies to find me.

I’m very happy to say that CruiseControl.NET 0.7 has been released. This was a fascinating release for me to work on for three reasons:

– I updated and added to most of the application, so I real feel I know it pretty well now

– I started playing with some new concepts, (like starting my own MVC implementation)

– … and most importantly, I finally understood how wonderful TDD, Interface-first, Mocking, Constructor Dependency Injection and the like all go together to make coding a whole new experience, with much better results.

The following is taken from the release notes

—

CruiseControl.NET 0.7 is one of our largest single releases so far. If you are upgrading from 0.6.1 or earlier, there are some big changes. Some updates are:

* Web Dashboard now has reporting options, allowing one web application instance to report multiple CruiseControl.NET projects across multiple servers. This feature is still in development, but see here for more details

* State Managers now automatically work for multi-project servers (so you should be able to take them out of your config file and forget about them!)

* The Xml log publisher now has default settings that work for multi-project servers (so you should just need an empty tag.) File merging has been removed from the Xml Logger, so you must use a File Merge Task for this behaviour.

* Some Source Control plugins can now automatically update your source tree for you (so no need for bootstrap builds). This feature is currently implemented by the CVS, Perforce and Visual Source Safe plugins.

* New section replaces old (See here)

* Introduction of ‘Working Directory’ and ‘Artifact’ as Project-level concepts to make relative directories easier to use. More support for these concepts will be coming in later releases.

* Considerable documentation updates