Wikis and Source Control (with Fitnesse, Subversion and CruiseControl.NET)

Wiki‘s are a great tool for having loosely structured documentation that a whole team can update. They offer a very low barrier to entry both in terms of learning how to use them, and in terms of infrastructure (no software to install, no shared document strategy to invent, that kind of thing.)

One problem with Wiki’s though are that they typically sit outside of a team’s source control environment. This is bad because teams like the versioning functionality source control offers and also because it means you have another repository of knowledge that needs to be backed up. Typically I’ve just made do with these problems but on my current project we are using a Wiki to host our automated acceptance test scripts. We’re using Fitnesse, but that doesn’t really matter for the sake of this article. What does matter though is that we definitely want our automated acceptance tests under source control since they are closely tied to the state of our source code, so we need to figure out how to get our Wiki working with source control.

Our approach to solving this is as follows:

  • We check our entire wiki (application and data) into a single folder in source control. Since our acceptance tests are tied to the source code, we check the wiki into the same branch as the source code.
  • Developers and QAs who need to edit the acceptance tests do so on a locally checked-out version of the wiki. They run the wiki app locally and edit it through the web-app as normal, just on their local version.
  • The application build script refers to the acceptance tests defined in the wiki. Once any acceptance test changes are complete, a local build is run and then the wiki updates are checked in to source control.
  • At that point a build is kicked off on the Continuous Integration server. It updates both its local copy of source code and wiki files, validating the current state of source and tests.

OK, so far so good: we define our Acceptance Tests in a wiki, and it fits into source control and our automated build in the normal way. As a nice side effect people can read and edit the Wiki offline. But we’ve lost something. We said at the top that there’s a low infrastructural barrier to entry in using a Wiki, and by introducing a source control environment that’s no longer the case. Is there a way we can have the ease of use of a ‘shared’ wiki but still keep it in source control? Yes, and here’s how we do it:

  • Checkout the wiki from source control to a machine, and run the wiki on this machine as a shared wiki instance.
  • Install a Continuous Integration (CI) tool on the same machine that is hosting your shared Wiki instance.
  • Setup a CI project that monitors for changes both the local file system where your wiki files are stored and the folder in source control where these changes are checked-in.
  • If changes occur in either of these locations, synchronise the wiki data by both updating from the source control server and committing any local changes (that themselves were made by people using the shared wiki instance.)

There are a few gotchas though to be aware of:

  • You need to use a stateless wiki. If your wiki caches edits and/or data read from the filesystem you’ll need to figure out a way of stopping it doing this, or a way of flushing its caches. Fitnesse thankfully is a stateless wiki so we didn’t have this problem.
  • Some wiki’s can be quite ‘chatty’ about what they save to the file system. The version of Fitnesse that we are using out of the box edits one file as soon as anyone even looks at a page, which isn’t great for use with source control. Many wiki’s also have their own in-built source control system of sorts, which we no longer need. In short, take time figuring out how to turn off various features of your wiki that aren’t required in a source controlled environment.
  • Conflicts between users are going to be an issue, and we are still figuring out on our team a complete set of practices. The biggest tool for this is to make sure people using locally checked-out copies of the wiki update and commit very often and that the time between updates of the shared wiki is short (we have our’s set to 15 seconds.) We also have a rule that any wiki refactorings (e.g. page renames) are only done on locally checked out versions, not on the shared instance.
  • The ‘Recent Changes’ file is always becoming conflicted but since it is just a generated page we tend to let ‘local version win’ in this case. We’ll probably write some small batch scripts that make things easier for people.
  • Changes to Acceptance Test Fit Tables are only made on local wikis since they are part of the project’s buildable source files and so people should be running these changes locally before checking in. Similarly, Fit tests can’t actually be executed on the shared wiki – we decided it was too unclear what version of the code should actually be executed when tests were run in such a way.

Right, less talk, more code! 🙂 How did we actually implement all of this? In terms of environments we are using Subversion for source control, Fitnesse as our Wiki and CruiseControl.NET as our Wiki Update tool. For Fitnesse we updated the launch script as follows:

"%JAVA_HOME%binjava" -cp fitnesse.jar fitnesse.FitNesse -o -e 0 -p 8888

The -o and -e 0 options suppress unnecessary file updates and in-built source control. We also deleted all the .zip files which already existed (these are just for source control).

For the CruiseControl.NET project we have the following configuration (this is for CCNet 1.0) :

<project name="Wiki Sync">

<workingDirectory>c:sourcecontrolwiki-trunkwiki</workingDirectory>

<triggers>

<intervalTrigger seconds="15" />

</triggers>

<sourcecontrol type="multi">

<sourceControls>

<svn>

<trunkUrl>svn://oursvnserver/ourproject/trunk/wiki</trunkUrl>

<workingDirectory>c:sourcecontrolwiki-trunk</workingDirectory>

<autoGetSource>true</autoGetSource>

</svn>

<filtered>

<sourceControlProvider type="filesystem">

<repositoryRoot>c:sourcecontrolwiki-trunkwiki</repositoryRoot>

</sourceControlProvider>

<exclusionFilters>

<pathFilter><pattern>**.svn*</pattern></pathFilter>

</exclusionFilters>

</filtered>

</sourceControls>

</sourcecontrol>

<tasks>

<exec executable="sync.cmd" />

</tasks>

</project>

This is a little complicated, but basically it means that ‘sync.cmd’ is called if changes occur in our subversion copy of the wiki, or on the local version, but ignoring any of subversion’s own local files. Fitnesse is actually run as a service from c:\sourcecontrol\wiki-trunk\wiki .

sync.cmd is as follows:

@echo off

rem mark as removed any files that have been deleted

for /f "usebackq tokens=2" %%i in (`"svn st | findstr !"`) do svn rm %%i

rem add new files

svn add --force *.*

rem commit - this won't do anything if nothing is to be committed

svn commit -m "This is an automated Wiki commit"

The ‘svn update’ is done before all of this by the source control provider in CCNet. We may well update this to automatically handle conflicts (at the moment we have to do it manually by logging into the wiki server.)

OK, lets wrap this up. Wikis are great, writing acceptance tests in a wiki is also great, but acceptance tests should be in source control so we’ve put our wiki into source control. Through a bit of CI hackery we still have a ‘shared’ wiki that anyone can edit without having to use source control. It really is a hack though – it would be far cleaner if the shared wiki was actually persisting and reading directly to and from source control.

Kudos to Jeffrey Palermo – he’s already done most of this, blogged it and so provided the start to the work we did.