One year ago we in Smaato's data engineering team decided to use Scala programming language instead of Java for all new applications. This decision was driven by a number of reasons. First, the software we’ve been using in this space (Spark and Kafka) is predominantly developed in Scala. We need to have a good grasp on the language to work with these tools to contribute and extend them. Furthermore, our development team was pushing to use a modern language, and Scala's functional aspects lend themselves well to the kind of problems we have to solve.
Being able to seamlessly interface with Java code allows us to leverage our existing code base and work with other teams at Smaato. This is a good moment to reflect on the transition and talk about our learnings.
Scala is a huge language with a large number of concepts. Coming from Java, it's perfectly reasonable to start using it as a "better Java". Our team is made up of developers who already had experience with Scala, along with others who were new to the language.
In our experience, it was very easy for Java developers to pick up Scala and be productive in a short amount of time. Obviously there is a lot to learn, but you can always start with the simple language features. Coming from Java, the most important aspects to understand are:
- Case classes as a substitute for Java's POJOs
- Pattern matching
- Traits, Scala’s interfaces
- Everything is an expression, e.g. you can use
trystatements to generate values
Once you understand this and know how to construct Scala classes and objects, you're very close to being able to write the same code you could write in Java. The main hurdle from our point of view is Scala's complex type system and its collections library, which require a strong understanding in order to use them effectively.
Just because we have some working Scala code doesn’t necessarily mean it’s good, idiomatic code. You can write Scala that almost exactly looks like Java, and sometimes it’s not easy to loose the big old OO hammer. It’s worth reading Steve Yegge’s classic rant “Execution in the Kingdom of Nouns”. Java is driven by objects and nouns - Scala by functions and verbs. It tries to bridge OO and functional programming, and there are a number of things you can do to get onto the functional path:
- Try to construct your applications from small, pure functions instead of using objects for everything. Pure functions are side effect free, taking arguments and producing a value but not changing state in objects. It’s very easy to reason about such functions.
- Stick to immutability. This will make it easier to support parallelism and to understand your code.
Sometimes functional programming is described as a very academic and dry matter. Yet the main reason to learn about it should be to make your code simpler, easier to understand and free of errors. Scala is supporting this well.
We sat down and tried to come to a mutual understanding how to use Scala. We're not the first ones doing this, so it made sense building on e.g. Twitter's and Databrick's Scala guidelines. They are very similar and reflect the current understanding of the Scala community.
To enforce a common coding style, we decided to use ScalaStyle. We created a style definition which can be used in the developer's IDE, and we produce Checkstyle reports in Jenkins with the ScalaStyle Maven plugin. All code we release is peer-reviewed. So this is always a good chance to talk with each other about style and architecture choices and maybe point out non-idiomatic code. We also had a number of team meetings to walk through current projects, review our coding style and discuss Scala’s language concepts.
Maven vs. SBT
You can't pick a build tool in a vacuum. We already have a CI environment based on Maven and a lot of experience with this setup. For us, it made sense to stick with Maven, even though SBT seems to have a strong following. This may change in the future. It seems that Maven plugins for Scala are not being as actively developed as SBT plugins.
We decided to use ScalaTest with FunSuite style tests, which is very close to what we write in Java. Code coverage is measured with scoverage which is configured to produce Cobertura style reports which get displayed in Jenkins. For mocking we’re still evaluating if we should stay with Mockito or switch to ScalaMock.
So, after working with Scala the past months, what did we like, and where do we also still see room for improvement?
- We write a lot less code and also feel like the code is more expressive than what we would write in Java. This makes changes easy and improves productivity in general.
- Developers need to be challenged. The mental shift to more functional programming and a new language made us a stronger team.
- Compile times are a good bit longer than for Java
- Scala libraries abusing the power of the language and using symbolic method names for their API
As the team is growing and we gain more experience, some of the opinions above will change. We'll report back in a couple of months when we have even more Scala code in production. In the meantime, we’d love to hear your own experiences in our comments section.