Asking better questions

I didn’t know Aaron Swartz. I met him very briefly in December but that was all. Nevertheless it’s been a realization this last week and a half hearing from those that did know him what an amazing human he was, and how much of a loss there is for the world in his passing away too soon.

I watched online some of the memorial for Aaron that took place in New York last Saturday. I was most impressed and moved by the last speech, from his partner Taren Stinebrickner-Kauffman. There was much in what she said about the legal pressures surrounding Aaron’s last year, but what resonated most with me was this section:

Aaron didn’t believe he was smarter than anyone else, which is hard for — it was very hard for me to accept that he really believed that. He really, really believed that he was not smarter than anybody else. He just thought he asked better questions.

He believed that every single person in this room is capable of doing as much as he did, if you just ask the right questions.

Whatever you’re doing, are you confused? Is there anything that doesn’t quite make sense about what you’re doing? What is it? Never assume that someone else has noticed that niggling sense of doubt and already resolved the issue for themselves. They haven’t. The world does not make sense, and if you think it does it’s because you’re not asking hard enough questions.

If you’re in the tech sector, why are you there? What do you really believe in? If you believe that technology is making the world a better place, why do you believe that? Do you really understand what makes the world a bad place to begin with?

I’m serious. If you’re in this room and you work in the technology sector, I’m asking you that question. Do you understand what makes the world a bad place to begin with? Have you ever spent time with and listened to the people your technology is supposed to be helping? Or the people it might be hurting?

While I do believe that much needs to be done with regard to the unfairness of Aaron’s trial, there is little I can personally do about that. But the calling above is something that we all in the software development world can consider. If some of us act on this then Aaron’s passing will be a little less in vain.

The video of Aaron’s NY memorial is here. Taren’s speech is at about the 1:47 mark. Thanks to Chris Burkhardt for transcribing Taren’s speech – the full text is available here. My sympathies go out to all of Aaron’s family, friends and colleagues at this time.

Agile people over agile process

In June 2012 I gave a talk at QCon New York titled ‘Agile people over Agile process’. The full talk is here, and below are some of my thoughts on this topic.


What’s below is pretty long so if you don’t want to read it all, here’s the essence of my opinion.

In the ‘agile world’ these days I see a decent amount of pre-defined, unarguable, process and dogma – the very things that the agile movement initially tried to do away with. I think it’s worth stepping away from this and focussing first on individuals, how they communicate, and how as a team they best choose their techniques and tools

There are no such things as ‘best practices’, at least when it comes to being part of a software team or software project. There are practices that teams find useful within their context, but each context is different. Teams would do well to continually re-judge what process and tooling works best for them.

Agile teams can use values and principles to help drive their choice of process and tools.

So let’s begin…

There’s too much focus on process

When I got started in the agile world 10+ years ago we used to talk a lot about extreme programming (XP), Scrum, and the like. Obviously part of that was figuring out test driven development, pair programming, continuous integration, iterations, etc. A lot of it was also about how we needed to change as individuals though. Gone were the times where we could just sit in our cubicle and complete our individual tasks on the almighty gantt chart. No longer could we assume that we didn’t need to test code because that was someone else’s responsibility. We needed to embrace how we worked as a collaborative team, and not just argue over Emacs vs Vi. This was a revolution in how we identified as humans on a software project.

People back then accused XP of being a developer-focused methodology, and they were right, but this was with good reason. For developers to be most effective they needed to stop just being pawns in a bigger process and start talking to people, work with feedback, and take responsibility for delivery. XP helped them do this.

People in the agile world still talk a lot about Scrum, lean, kanban, etc., just like we used to 10 years ago. However I feel the tone of a lot of conversations has changed – now a lot of times it’s just about the process. Agile seems to no longer be about people changing their attitude to projects, to delivery or most importantly to people. Now often it’s just about introducing a new team management methodology in the hope that Lean or XP or whatever will be a process magic bullet that will solve all their problems.

But with process, as with many other things, there is no magic bullet.

Process is very important. It’s where the rubber of any methodology hits the road, but there are problems with an overly-zealous focus on process:

  1. Processes can become kings. Processes are at the end of the day just tools – they have no intrinsic value by themselves. They can only be judged valuable in the context of their application. When processes become kings then our discussions descend to hypothetical judgments of a supposed intrinsic value that the processes don’t have. Such discussions are a waste of time and don’t help anyone.
  2. If processes are considered axiomatic then we can no longer adapt how we work. If we believe the best way to do something is X, yet we do not understand the motivation for X, how can we decide if Y would be better?
  3. It misses the the point of what Agile was supposed to be about…

What I think is important about Agile

The Agile Manifesto starts as follows:

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
Individuals and interactions over processes and tools…

In other words the unique characteristics, personalities and abilities of members of a software development team, and the conversations that they have with each other and with their stakeholders (management, users, etc.) are worth considering more than the process and tools that they use.

This is not to say at all that processes and tools are unimportant, I am merely arguing that that they are not so important that they should drive discussion about the basic way we choose to deliver software.

Focussing on individuals sounds like a management technique. While that is important, I think it is more a call to individuals themselves to consider how they can most effectively be part of a software delivery effort.

There are many ways individuals can answer this call, but one way that may be useful is to look at the values and principles from ‘Extreme Programming Explained‘ – Kent Beck’s original book about XP. These values and principals are not specific descriptions of how to do something but guides to help people decide how they actually want to work. I’m not going to go into detail here since literature is already available for each point (and I also discuss further in my talk at around the 13 minute mark), but a list of ideas is as follows:


  • Simplicity
  • Communication
  • Feedback
  • Courage


  • Assume the values are present
  • Incremental change
  • Deliver at the earliest responsible time (an addition / variant of mine that I think is worth considering separately)
  • Quality (the Jim Highsmith article I refer to in the talk is here)
  • Accepted responsibility – everyone on the team should assume they have responsibility
  • Local adaptation – change everything according to context

How this applies to practice

These values and principles are all somewhat theoretical – the application of them is the actual choice of what set of  practices and processes a team choose to use. Not one pre-defined overall process, but an active, continuing choice of what techniques to use.

This is necessary since in the case of software development teams and projects, there are no such things as best practices. There are practices that teams find useful within their own context, but this is not an absolute categorization.

How I’ve recently embraced this

In the video I describe how I applied these ideas on my most recent project. It starts at about the 28 minute mark. It made sense to include this in the talk but I’m going to leave detail out of this post.

It is worth mentioning here though that there are certain ‘typical practices’ of agile that we did use but others that we chose not to. For example we didn’t use ‘iterations’ to structure our week-to-week work. However we did often deploy new software, we frequently re-prioritized our next work, etc. Since we already did these things formal iterations in our world would have been unnecessary baggage. For many other teams formal iterations would be very valuable.

Is this really for everyone?

In discussing this subject some people have challenged me that this way of thinking is only useful for ‘experts’, such as people who already have previous agile experience. I disagree. While I think that picking an off-the-shelf methodology might make for a ‘decent first set’ of practices I think a team needs to know from the beginning some amount of why that may be so. I think for someone with experience to provide a pre-canned process set as ‘the way things should be’ is disingenuous.

I wouldn’t expect everyone on a team new to agile to be able to immediately make their own choices about the entire implementation of principle to practice, but I would expect them to know that the introspection of their process (based on values and principles), and their subsequent refinement of the same, is a more important aspect of agile development than any of the individual techniques they may be using.

Concluding thoughts

None of this is new at all, and a lot of the good agile literature from the last decade describes these ideas. As a more recent example Ross Pettit does a good job talking about them here.

I think it’s worthwhile repeating it though for 2 reasons:

  • I see some amount of the agile community as a whole moving to a ‘process first’ mindset and I disagree with it.
  • I’ve seen myself at times throughout my career treating process, practice or technique as dogmatic. Invariably when I do this it’s because I’ve missed something important. Stepping back and thinking ‘why’ always leads to improvement. I think this is a valuable reminder to myself and hopefully others.

5 months banking with Simple

Five months ago I opened a bank account with Simple. Here are some thoughts of how things are going so far.

tl;dr . I like Simple and will keep using them for most of my banking, but they can’t (yet) fully replace all of my banking needs.

Updated – The friendly people at Simple got back to me about a few things I mention here and I’ve put a few notes at the end, with in-line markers.

Simple (originally named ‘Bank Simple’) is a new ‘bank’ that aims to bring the usability of modern web-commerce to retail banking. I quote ‘bank’ since they are not legally a bank themselves, they are effectively a customer proxy on top of a separate underlying partner bank (currently Bancorp.)

I’ve been living in the US nearly 7 years and the customer experience of using banks here is pretty poor, most notably their websites. I used to bank with Citi and I currently bank with Chase, and for each I’ve routinely cursed their online capabilities. Things are getting better, but very slowly. When I heard about Simple I decided to give them a go.

The main benefit of using Simple over a regular bank is that they aim to provide a much better online experience, both through the website and their mobile app. With this they absolutely succeed. Both apps are fast, have a decent amount of useful features, are clean and well thought through. As an example once you’ve signed in on your mobile phone once with your full passphrase you can subsequently just use a short PIN. Try to sign on somewhere else and you’ll still need the full passphrase.

The apps also offer a lot of features to help you keep track of where and how you’re spending your money. This is less useful for me since I tend to do most of my spending through my credit cards, but I could see this being interesting for some.

Simple are also pretty good on the support front. There’s been a couple of times where I’ve needed to communicate with a human being and they’ve been efficient and friendly. They are still a relatively small company though and I’m sure that helps.

I’ve moved most of my checking account (current account in UK parlance) activity to Simple – which is mostly my regular bill payments (credit cards, rent, etc.)

There are, however, a few things I haven’t moved to Simple and for this reason I still keep a checking account with Chase. The main reason is ATM (cash machine) usage. Simple do offer free access to a large network (Allpoint) of ATMs but there are a couple of problems for me:

  • In New York City at least there are far fewer Allpoint ATMs than Chase ATMs. I did once try to find the nearest Allpoint ATM to my apartment, which is about twice as far away as my nearest Chase branch, and I wasn’t even able to find it once I got to its supposed location.
  • Allpoint ATMs tend to only allow taking out $200 at a time [1] (they are the kind of ATM you find in small stores, Walgreens, etc.) While this is sufficient most of the time for me there are occasions when I want to take out more than that. Chase will let me take out up to $500 at a time should I need it.

Simple could fix this by offering to pay fees on any domestic ATM you may want to use. Other banks do this, often up to a limited amount (e.g. the first $15 of ATM fees per month). Of course my friends in the UK are probably laughing at this point since in the UK anyone can use any ATM for free – I do miss that.

Another reason to keep a bricks-and-mortar bank account is for the very occasional times you need something from a branch. For example when I first moved into my current apartment my landlord needed a cashier’s check [2]. Recently I also needed a special kind of financial notarization when changing my brokerage accounts and my local Chase account would do that – I’m not sure if they would have if I didn’t bank there. Simple also don’t (I believe) currently offer outgoing international wire transfers which could be a deal breaker for some people.

I could likely survive without these services, especially if my wife were to keep her ‘regular’ account.

Another area where Simple don’t quite do what I’d hoped is that I wanted to move my money from one of the ‘mega banks’ to a much smaller company. While Simple are still small their partner, Bancorp, is part of US Bank – still pretty huge [3]. I doubt I’ll ever have the situation I did in the UK where I was able to have a good banking experience with an ethical bank (the Co-operative bank) but I still hold out some hope.

So, in summary, my first 5 months with Simple have been encouraging and enjoyable. If they can fix up the ATM situation I may even close my regular bank account.

[1] Simple will allow you to take out up to $500 / day from an ATM, but if the ATM you’re at will only let you take $200 / time you’ll need to perform multiple transactions.

[2] Simple will mail you a cashier’s check if you request one. The once or twice I’ve ever needed one though I’ve needed it the same day to secure the apartment I’ve been looking to rent.

[3] Apparently ‘The Bancorp Bank” (who Simple partner with) is not part of “US Bancorp” (yes, I’m confused too.) “The Bancorp Bank” is its own entity and a small – medium size bank.

Hiking with Avenza PDF Maps

The hike on Sunday gave me the first opportunity to use a new app on my iPhone in the wild – Avenza PDF Maps.

Avenza ScreenshotAvenza PDF Maps is (unsurprisingly) a map application for iOS. The key though is that it doesn’t come with it’s own maps – you download apps using its inbuilt store. The New York-New Jersey Trail Conference have a great selection of hiking maps near me, and some of these are available through the Avenza app. I bought the ‘East Hudson (North)’ map for my hike at Breakneck Ridge.

Since this is a universal app I downloaded on both my iPhone and my iPad. Buying a map on one allows you to download it again on the other. Before leaving home therefore I used the iPad app to get a good idea of where we were headed. Once on the path I used my iPhone to figure out where we going. Even better there’s GPS support so you can see exactly where you are (the attached picture is a screen shot from when were on the hike.)

I’d definitely recommend this if you can get good quality maps for your hikes, even though it does feel like I’m ‘cheating’ a little!

Hiking Breakneck Ridge

With the weather finally cooling off a little in New York it was time last weekend to get out of the city and into the countryside. I’d been wanting to get back to the beautiful Hudson Valley for some time so on Sunday morning we got an early rise to head to Breakneck Ridge.

Breakneck Ridge is a hugely popular hiking spot about 50 miles north of New York City. Located in the Hudson Highlands it offers some breathtaking views, lush woodland and, even better, is easily accessible by train. There is a relatively frequent service to Cold Spring, 3 miles or so south of the Breakneck Ridge trail head, but Breakneck Ridge Station itself has trains stopping from Grand Central a couple of times on a Sunday morning. We got the later of these two, the 8:50am, which arrived a little after 10.

View from Breakneck Ridge, looking southThe popularity of this area was very apparent to us as a huge stream of hikers left the train at the tiny platform (just a couple of steps really, a platform is giving it too much standing). A quarter of a mile away the trail started and we were greeted almost immediately by the first of several steep climbs over boulders. This isn’t quite rock climbing territory, but you wouldn’t want to try doing this in flip flops or soon after a rain shower. I was certainly on all fours several times.

After each of the 5 or so climbs going up the ridge you are greeted with some fabulous views of the Hudson. Once at the top you’ve climbed 1200 feet or so in only 3/4 of a mile, so this is definitely up there in ‘strenuous’ territory, for me at least.

There are various options you have once at the top to continue your hike. Most people will take a circular route to get back to their car, but we knew we wanted to end up at Cold Spring where the return train service to Grand Central is much more frequent (about once an hour) than trying to get a train from the Breakneck Ridge stop. Knowing this we planned a route through the woods that took in another climb (part way up Mt Taurus, giving us great views of the ridge we’d just climbed) before putting us back in civilisation a quarter of a mile north of Cold Spring (click on the attached picture for the route.)Breakneck Ridge to Cold Spring

We were walking about 5 hours, with a couple of stops for food. Once back in Cold Spring we enjoyed a well earned beer at McGuires, and then an ice cream at Moo Moo’s before getting the train back to Grand Central. All in all a wonderful day out.

Leaving DRW, and my take on Customer Affinity

Last month I finished working at DRW Trading after nearly 4 years there. DRW has a fantastic technical organization on both the Software Engineering (SE) and IT Operations side, from the leaders (CTO Lyle Hayhurst, COO Seth Thomson and Director of SE Derek Groothuis) down. In many ways I expect this will be one of the best jobs I ever have – my technical colleagues were fantastic (especially Jay Fields, my right hand man for the last 18 months), my team had complete management and implementation control of our projects, I didn’t have to deal with much in the way of politics at all, and yes, the pay was good!

So the obvious question is why leave? As usual in such cases it’s not a simple answer. I’m going to go on something of a tangent before I give a couple of reasons.

With software development jobs where the goal is producing a product, that product is not an end in itself for the customer [1]. The product is a tool that will be used by the customer to do something. Compare this with doctors or actors – in the first making people healthier is the sole end goal, in the second entertaining is the sole end goal [2].

(Good) software developers have some understanding about what the thing they’re making is going to be used for. Compare this time with structural engineers responsible for building a bridge – they know they’re building a bridge but they’ve no idea what the final destination is of the people traveling over that bridge.

I think most software development roles in some ways resemble being a lawyer (hear me out!) As a lawyer the principle aim, in the context of litigation at least, is to win the case, but you’re always doing so for a particular type of case. You might be a criminal lawyer, patent attorney, etc.

Most lawyers I know are interested not just in the practice of law itself, but also to some extent in their more specific field. A non-profit housing lawyer (to use an example very close to me, thank you Sara) might not just be interested in winning cases, but in helping less privileged tenants in having a fair voice against landlords with far more means.

And so we come back to software. There’s no doubt that as ‘software people’ we are interested in making stuff. We all have parts of ‘making’ we’re more interested in, whether it be the technical design, the project process, the user interface, etc. But I would describe these all as second-order goals.

The first-order goal is the actual thing that will be useful to the customer. Again, I’ll say what I did above – good developers have an understanding about the first-order goal (in other words they have what Martin Fowler calls Customer Affinity.) Excellent developers for the long term have not just an understanding, but an active interest, of the first-order goal (otherwise known in our field as the domain.)

For most of the time I was at DRW the second-order problems I was solving were fascinating to me. With a very small team we built and maintained a large, solid, well-appreciated application. Furthermore I was making good inroads into understanding the domain (commodities trading). Once we’d solved a lot of the second-order problems though what remained was understanding, and appreciating, the domain.

The problem was that I don’t find trading very interesting. To me it’s ‘just’ math and moving money around. Trading in some ways has a lot in common with gambling : assessing financial risk and making positions (‘bets’) based on what you currently see in the slice of the world around you. I’ve never been particularly interested in gambling and I think the 2 are linked. I know many excellent developers who are truly interested in trading (Jay being one of them) – I don’t have a problem with them, I just don’t share their enthusiasm for this particular domain.

Further though, I think to be an excellent developer in the long term not only must you appreciate / have passion for what the users of the software want to do with the software, you also need to have empathy for the users themselves. Martin says this at the end of the first paragraph in the link above : “[Customer Affinity] is the interest and closeness that the developers have in the business problem that the software is addressing, and in the people who live in that business world.” (emphasis mine).

This leads to a second reason I left DRW – I didn’t empathize with many of the traders I worked with. Note that I’m absolutely not saying I thought they were wrong, they’re certainly far more financially successful than I am at least, what I’m saying is that my approach to work and their’s didn’t meld in the general sense. I don’t think it’s useful to get too specific here, and probably this is something you wouldn’t know either way until you’ve been in trading for some time, but I think there’s a general lesson worth taking from this beyond just trading.

One thing I’m happy about is that I realized these things before they started to make an impact on my work. Being able to leave knowing you’ve done your best, but that you wouldn’t be doing your best if you stayed, is a very satisfying position to be in.

The reasons I give for my leaving sound negative, but I’m picking specifically the reasons I left, not the many, many reasons I stayed for nearly 4 years. Of the financial services companies I’ve worked with I enjoyed being at DRW by far the most, I don’t regret my time there at all, and I absolutely appreciate the opportunity I had (and am grateful to the leadership of DRW for giving it to me.) For developers wanting to join, or continue in, the trading industry I would recommend joining DRW wholeheartedly.

Of course, there’s an obvious postscript here. What am I doing now? I’m taking time off! I’ve never taken extended leave in my life, not ever since school, and I’m fortunate to be able to do so now. I have a few ideas of what I might do and I look forward to updating this blog as some of them (and others!) come to fruition.

[1] There are other types of software development jobs, e.g. programming language research, training, and ‘pure’ consultancy (as opposed to ‘body shopping’). The difference with consulting is that you’re not building a product – the end goal is to help other people build a product. I would even consider going back into finance as a ‘pure’ consultant.
[2] Maybe I’m over simplifying here, but I don’t think I’m too far off the mark

Book Review – William Gibson’s ‘Virtual Light’

Virtual Light (Bridge, #1)Virtual Light by William Gibson
My rating: 4 of 5 stars

The last time I read this book was 10+ years ago. After I recently traveled to San Francisco, where a lot of this book is set, I wanted to re-read it. I remember it as being ok, but not his best, and certainly not as good as the last book in this ‘Bridge Trilogy’ – ‘All Tomorrow’s Parties’.

I was also a little trepidatious about doing so since a relatively recent re-read of Neuromancer did not hold up to my memories.

It turned out though that Virtual Light is much, much better than I remembered. The concept of the San Francisco-Oakland Bay Bridge becoming effectively a shanty town I remembered as being brilliantly described, and that was the same this time. Furthermore the plot seemed to whizz by and the character development was good. The ‘historical’ sub-plot of the ‘HIV martyr’ JD Shapely was well conceived and fantastically exposed during the rest of the book.

I only give 4 stars since I think the character development could have been a little better, and also I thought that one of the key plot points (why the glasses were so valuable) was a little weak.

Another thing that works about this book now is that it was set at the time in the ‘near future’ – the early 2000’s (it was first published in 1994.) We’re obviously now past that point so instead of science fiction this book effectively becomes more speculative fiction. I enjoyed this as such since it’s good to know things didn’t turn out as bad as they could, yet on the other hand some of the themes still offer a warning about how out society could become if left to be screwed up by corporations and the upper classes.

View all my reviews