Blog

Last month I finished working at DRW Trading after nearly 4 years there. DRW has a fantastic technical organization on both the Software Engineering (SE) and IT Operations side, from the leaders (CTO Lyle Hayhurst, COO Seth Thomson and Director of SE Derek Groothuis) down. In many ways I expect this will be one of the best jobs I ever have – my technical colleagues were fantastic (especially Jay Fields, my right hand man for the last 18 months), my team had complete management and implementation control of our projects, I didn’t have to deal with much in the way of politics at all, and yes, the pay was good!

So the obvious question is why leave? As usual in such cases it’s not a simple answer. I’m going to go on something of a tangent before I give a couple of reasons.

With software development jobs where the goal is producing a product, that product is not an end in itself for the customer [1]. The product is a tool that will be used by the customer to do something. Compare this with doctors or actors – in the first making people healthier is the sole end goal, in the second entertaining is the sole end goal [2].

(Good) software developers have some understanding about what the thing they’re making is going to be used for. Compare this time with structural engineers responsible for building a bridge – they know they’re building a bridge but they’ve no idea what the final destination is of the people traveling over that bridge.

I think most software development roles in some ways resemble being a lawyer (hear me out!) As a lawyer the principle aim, in the context of litigation at least, is to win the case, but you’re always doing so for a particular type of case. You might be a criminal lawyer, patent attorney, etc.

Most lawyers I know are interested not just in the practice of law itself, but also to some extent in their more specific field. A non-profit housing lawyer (to use an example very close to me, thank you Sara) might not just be interested in winning cases, but in helping less privileged tenants in having a fair voice against landlords with far more means.

And so we come back to software. There’s no doubt that as ‘software people’ we are interested in making stuff. We all have parts of ‘making’ we’re more interested in, whether it be the technical design, the project process, the user interface, etc. But I would describe these all as second-order goals.

The first-order goal is the actual thing that will be useful to the customer. Again, I’ll say what I did above – good developers have an understanding about the first-order goal (in other words they have what Martin Fowler calls Customer Affinity.) Excellent developers for the long term have not just an understanding, but an active interest, of the first-order goal (otherwise known in our field as the domain.)

For most of the time I was at DRW the second-order problems I was solving were fascinating to me. With a very small team we built and maintained a large, solid, well-appreciated application. Furthermore I was making good inroads into understanding the domain (commodities trading). Once we’d solved a lot of the second-order problems though what remained was understanding, and appreciating, the domain.

The problem was that I don’t find trading very interesting. To me it’s ‘just’ math and moving money around. Trading in some ways has a lot in common with gambling : assessing financial risk and making positions (‘bets’) based on what you currently see in the slice of the world around you. I’ve never been particularly interested in gambling and I think the 2 are linked. I know many excellent developers who are truly interested in trading (Jay being one of them) – I don’t have a problem with them, I just don’t share their enthusiasm for this particular domain.

Further though, I think to be an excellent developer in the long term not only must you appreciate / have passion for what the users of the software want to do with the software, you also need to have empathy for the users themselves. Martin says this at the end of the first paragraph in the link above : “[Customer Affinity] is the interest and closeness that the developers have in the business problem that the software is addressing, and in the people who live in that business world.” (emphasis mine).

This leads to a second reason I left DRW – I didn’t empathize with many of the traders I worked with. Note that I’m absolutely not saying I thought they were wrong, they’re certainly far more financially successful than I am at least, what I’m saying is that my approach to work and their’s didn’t meld in the general sense. I don’t think it’s useful to get too specific here, and probably this is something you wouldn’t know either way until you’ve been in trading for some time, but I think there’s a general lesson worth taking from this beyond just trading.

One thing I’m happy about is that I realized these things before they started to make an impact on my work. Being able to leave knowing you’ve done your best, but that you wouldn’t be doing your best if you stayed, is a very satisfying position to be in.

The reasons I give for my leaving sound negative, but I’m picking specifically the reasons I left, not the many, many reasons I stayed for nearly 4 years. Of the financial services companies I’ve worked with I enjoyed being at DRW by far the most, I don’t regret my time there at all, and I absolutely appreciate the opportunity I had (and am grateful to the leadership of DRW for giving it to me.) For developers wanting to join, or continue in, the trading industry I would recommend joining DRW wholeheartedly.

—

Of course, there’s an obvious postscript here. What am I doing now? I’m taking time off! I’ve never taken extended leave in my life, not ever since school, and I’m fortunate to be able to do so now. I have a few ideas of what I might do and I look forward to updating this blog as some of them (and others!) come to fruition.

[1] There are other types of software development jobs, e.g. programming language research, training, and ‘pure’ consultancy (as opposed to ‘body shopping’). The difference with consulting is that you’re not building a product – the end goal is to help other people build a product. I would even consider going back into finance as a ‘pure’ consultant.
[2] Maybe I’m over simplifying here, but I don’t think I’m too far off the mark

For listening to music at home I use an Apple TV plugged into my fancyish sound system, and so I use music stored in lossless format. Since I use an Apple TV this music is stored on a computer using iTunes. I also have an iPhone, and my music library is on there too, but I can’t fit my entire lossless library on there (it’s more than 100GB) so up until now I’ve also kept a totally separate iTunes library, on a different computer, with the same music in 128kbps AAC format that can fit on my iPhone.

iTunes for a while has had an option where when syncing to an iPod shuffle it will automatically convert songs to a low bitrate to fit more songs on. I realized a couple of weeks ago that this option now exists for iPods and iPhones too – it appears on the main iPhone screen when you look at the device in iTunes

I tried this out last week. It definitely works, but takes a long time, about 15 hours syncing from my ~4 year old iMac. I can live with that slowness though now I don’t have to look after 2 separate libraries and manually convert all my music to smaller formats myself.

 

Previously when I’ve pair programmed (2 people programming at the same computer at the same time) I’ve always used one keyboard, screen and mouse (KVM – V means Video). In the last couple of weeks I’ve been trying out ‘dual KVM’ pairing though – in this scenario each programmer has their own keyboard, mouse and monitor, where the screens are setup to mirror each other (each person sees exactly the same thing)

This style of pairing isn’t new, and certainly is common on other teams at DRW, I just hadn’t used it before. In fact I had concerns, the principal ones being:

1. Wouldn’t something be lost in communication with not having a shared physical screen? (I point at things on the screen fairly often when pairing)
2. Wouldn’t programmers be constantly aware that they might be fighting each other for control of the mouse pointer / cursor if they had their own keyboard and mouse?

It turns out that I really like this style of pairing. My concerns with communication about the screen are largely alleviated by turning on line numbers in the code editor, and the keyboard and mouse fighting isn’t nearly the problem I feared. The benefits are chiefly ergonomic, but they are significant. Being able to look straight forward, and not having to lean in towards the keyboard and mouse makes work a lot more comfortable. The only thing I slightly miss is being able to use 2 screens for a stretched desktop, but that’s a price worth paying – I can always switch back the screens to this mode when I’m not pairing.

6 months ago my team decided to migrate our functional tests to being coded in Scala rather than Java, the native language our application is written in. However we have now reverted back to writing them in plain Java. What follows is an experience report of this exercise and our reasons for bringing it to an end.

Background

The application under test is a message-driven server application. We define the functional tests of this application as those that run against the largest subset of our application we can define without requiring any out of process communication. The functional tests themselves run in process with the application under test.

Each functional test is written in a style that treats the whole system (mostly) as a black box. We stub out all external collaborators – those stubs simulate collaborators sending messages, and also collect any messages that they receive allowing the tests to make assertions about the application’s interactions with its environment.

Our application is not trivial; writing functional tests that are concise, understandable and maintainable is a tricky task. We’ve created a fair number of support classes that start the system and act as the collaborator stubs described above to help keep the tests themselves clean.

We use functional tests extensively, and typically write at least one functional test per work item on our backlog. Just in terms of numbers about 10% of all the automated tests we have are functional in style, the rest are per-class-level unit tests.

For our development environment we use IntelliJ as our IDE and Rake as our command-line build environment.

Switching to Scala

We were interested in trying Scala as our functional test language for 2 main reasons:

1. To improve clarity and maintainability of the tests
2. To assess Scala as a possible production-code language

First go and read this great post by Bill Caputo. Bill’s site doesn’t seem to allow comments right now so I’ll put my response here instead.

I think part of the problem is that the agile ‘movement’ is so top-heavy with consultants. For many of these consultancies its very hard to sell a story like agile, which isn’t for a specific technology, when it’s packaged as a technical solution. They have to make it a business process problem to get through the door at a high enough dollar rate, or (for bigger consulting firms) to get enough bums on seats to make it worth the effort.

Think of other cross company technology efforts like standards bodies. Are all of these racked to the gills with consultants? No, they have a bunch of CTOs, senior developers, or whatever who are having to live with this stuff every day in a consistent environment. They do have consultants too, but not drowning out everyone else.

I know that the agile movement started with a bunch of really smart people, most of which were consultants, and some of its current leads (tip of the hat to some of my previous colleagues still at ThoughtWorks) are brilliant and continue to add valuable insight to our industry.

However for agile to get to anywhere beyond where it’s become (mostly a big mix of fluffy ideas that are easily billable but which don’t really solve anything without the necessary discipline which most companies are incapable of) it needs a much better diversity of background of leaders. Unfortunately I don’t see that happening – it’s just too big. Take the Agile 20xx conferences – they’re now basically 3 things:

• 101-level training for newbies
• an expo for largely pseudo-agile consulting firms and mediocre tools
• a small amount of people who’ve known each other for ages catching up and complaining about the state of agile.

So I think you’re right, Bill, agile is dead. It served a good purpose, and did a pretty good job of giving our industry the kick up the behind it needed, but it is now pining for the fjords.

To end optimistically though, there’s still a lot of great stuff going on in our industry, its just these days I’m much more interested in technically based conferences and communities, and having conversations on the side of these around process. Its from these technical communities I’ve learned about things like Kanban, for example. And its a blessed relief not to have to justify whether the team I’m on ‘is agile or not’.

Retlang and Jetlang are open-source libraries for the .NET CLR and JVM that provide concurrency through in-process messaging. Mike Rettig, one of my colleagues at DRW, is the lead of both of these projects.

Today at Speakerconf I presented on these libraries, and the slides can be found here (in Keynote ’09) and here (in PDF).

I encourage you to use the project mailing lists (for Retlang and Jetlang) if you are interested in learning more.