6 months ago my team decided to migrate our functional tests to being coded in Scala rather than Java, the native language our application is written in. However we have now reverted back to writing them in plain Java. What follows is an experience report of this exercise and our reasons for bringing it to an end.
Background
The application under test is a message-driven server application. We define the functional tests of this application as those that run against the largest subset of our application we can define without requiring any out of process communication. The functional tests themselves run in process with the application under test.
Each functional test is written in a style that treats the whole system (mostly) as a black box. We stub out all external collaborators – those stubs simulate collaborators sending messages, and also collect any messages that they receive allowing the tests to make assertions about the application’s interactions with its environment.
Our application is not trivial; writing functional tests that are concise, understandable and maintainable is a tricky task. We’ve created a fair number of support classes that start the system and act as the collaborator stubs described above to help keep the tests themselves clean.
We use functional tests extensively, and typically write at least one functional test per work item on our backlog. Just in terms of numbers about 10% of all the automated tests we have are functional in style, the rest are per-class-level unit tests.
For our development environment we use IntelliJ as our IDE and Rake as our command-line build environment.
Switching to Scala
We were interested in trying Scala as our functional test language for 2 main reasons:
- To improve clarity and maintainability of the tests
- To assess Scala as a possible production-code language
We already had a good number of functional tests going into this exercise and so our first task was to rewrite these in Scala. We also rewrote most of our test-support classes in Scala.
Since this was our first time writing Scala the translation wasn’t a blisteringly fast process but the Intellij Scala plugin’s ‘copy Java, paste as Scala’ feature did help us get going. If nothing else it was a useful guide when translating generic code.
Another initial task was to setup our development environment to support Scala. Intellij’s Scala plugin, while having a number of deficiencies, does the basics well and we were very quickly compiling and testing Scala alongside Java in the same project. Even though Intellij will support Java and Scala code in the same source we kept all Scala code in a separate source tree to avoid complications with the command line build. With that setup updating our rake script to compile Scala and run the Scala tests was relatively easy.
What was good
The main thing that attracted us to Scala was the ability to write code in a semi-functional style much more concisely than can be done in plain Java. We’ve also been coding a good amount of C# recently and we sorely miss the basic functional support in C# 3 when switching to Java. We were not disappointed by Scala’s abilities in this regard: there were many occasions where we could write 1 line of concise, readable Scala where previously we’d had 8 lines of a Java method.
Why drop it?
There were several reasons we decided to roll back to Java, and to be fair to Scala most of them were not it’s fault.
The biggest reason was that despite 6 months of experience we still found we were slower to code and debug Scala than Java. We probably spend around 5 to 10% of our coding time working with the functional tests and that just isn’t enough to really ‘get’ the language. I think this would be similar for most languages – you’ve got to use them significantly to become fluent in them – but I think this is particularly true with Scala since it is a large language with an equivalently large library.
I don’t think its just the time aspect either. Tests are a very specific style of coding and mostly procedural. Where we got most of the benefit of Scala were in our test support classes, but even they aren’t hugely complex. We never got into any meaty problems in our Scala realm and so never really pushed our knowledge of it.
Scala is absolutely a more powerful language than Java and as I mentioned above we could write code more concisely in Scala than we could in Java. However IntelliJ is a great tool and it makes up for a surprising number Java’s deficiencies. You end up with more code on the screen with Java but I’m not convinced that it takes more time to write it. Furthermore once the code is written the rest of the IDE experience is far better in Java than Scala – compiling is faster, code browsing works much better and debugging Scala in Intellij is no fun at all. (yes, we use a debugger, I know that probably makes us awful programmers in the eyes of some readers!)
Again this isn’t necessarily Scala’s fault – if I didn’t have an IDE at all Java would be more painful than Scala – but I do have an IDE and even if I don’t write the most elegant solution that’s not what my goal is – my goal is to create functioning software as quickly as I can (for the next and following releases.)
Some reasons we ditched Scala though can’t be blamed on tools or the particular problem we were trying to solve with it. Scala is large, larger than I’m comfortable with. I want a language that’s more opinionated, at least when I’m getting started.
Furthermore the libraries, especially the collection libraries, are hard to get a handle on. As a particular example Scala has both mutable and immutable Set classes … but they are both called ‘Set’ . Relying on a namespace definition to specify something as fundamental as a collection type is just plain frustrating. The Interop between Java and Scala, specifically with Collection types, can also be painful (even with some of the updates in Scala 2.8 .)
In Conclusion
To me using Scala isn’t a great fit in a polyglot environment: it’s a large language to learn, the interop still needs work and and the tool chain adds friction in comparison with plain Java. That said I think with time Scala could become a very interesting monoglot language for developing an entire app on the JVM – in that scenario developers would naturally come to learn it more throughly and wouldn’t be brushing up against interop problems.
Using Scala was a good experience overall and has been yet another push for us to write more functional-style Java. We’ll be looking at other languages to enhance our productivity but will likely leave our functional tests in Java.
Mike Roberts