Hi. In this post we will be exploring some unexpected gotchas we found while working with Scala. Hopefully it will help you to better understand Scala and fix some naughty bugs :)
Consider this code
It’s an obvious error -
bar isn’t defined here, so initialization will not succeed. What about now?
This code will compile just fine. Unfortunately asking for the calculated value will show an issue:
Test.foo produces 0. If you dig deeper the compiler will give you a hint what’s going on
Warning:(2, 23) Reference to uninitialized value bar
val foo: Int = 2 * bar
Essentially the same problem as above. Wrapping in an object reduced the compile error to a warning. Not good … but it get’s even more confusing when you consider traits.
When mixing a trait we would expect our overrides to get the code under control, but it’s not the case. The result for the second case won’t be “test!”, but “null!” instead. Although the first call gives us a correct result. What’s even worse compiler won’t issue any hints or warnings here.
What can we do here? An obvious solution here is to make
def in the base trait. Which is a good practice, as we gain two things:
Or you can make the computed field
lazy thus changing the execution order.
Why did it happen in the first place? In both cases the problem boils down to execution order.
name value is
null until we won’t override it in
Implementation. Unfortunately, before it happens Scala builds the dependent field using null. If you consider that
null.asInstanceOf[Int] produces 0, the first example becomes understandable. When evaluating fields in Test object, Scala knows what fields exist, so no error is generated. But as they weren’t evaluated yet, default is used.
Mixing concurrency models in one unit of code is (usually) a bad idea … I understood that when debugging an Akka application.
The code went like this:
In Akka we are used to stateful actors - very often it’s the simpler solution and, thanks to actor model, we have the guarantee that only one thread will be modifying our state. It works good in bounds Akka puts on us. The problems start when when we go beyond these bounds, for instance adding callback based
Futures into the mix.
onSuccess, get called when the result is ready. Not when the actor is ready to safely process another message. And it can get us into trouble.
Imagine that we sent many
Query objects to the actor. Since the request goes through the network we cannot say which one will get back first. As a result we can’t say how state mutation will be interleaved. In this code we expose our internal state to all the problems we had with mutable shared state.
This kind of bug can easily break your code, but can also easily be fixed. Just delegate handling the asynchronicity and synchronization to Akka. For instance like this:
With this approach we still cannot say for sure what will be the order in which the responses will come in, but we will get rid of threads fighting over a shared resource as all the mutation changes will have to go via actor’s mailbox.
Maps have significant advantage over their Java counterparts. They are easier to construct and work with. Standard Scala library comes with abundance of methods to manipulate maps and their contents. You can do what you want with one or two lines of code while in Java you would need for doing the same one or two screens. There are of course gotchas and you need to be aware of them.
One of the most convenient things you can do with map is to transform it using
map method. The signature of
Map.map is following:
and actually it comes from trait
TraversableOnce which means you can expect it to offer the same functionality as with all other
Sets and whatnot. Let’s see an example:
The gotcha here is actually valid also for
Sets, but with sets it’s more vivid and actually people tend to avoid it. But recently I had to fix more than one bug related to maps where they didn’t. Let’s complicate our example a little bit:
So we have a map of developer unique ids to their names and a map that maps developer id to a department code he’s working in. The goal is that we want to be able to tell developer names by the department code, maybe because we want to send customized emails depending on the department code, why not.
The thing incautious developer would do was shown in the example above. But the result wouldn’t be exactly as we expected:
Where is poor Mr. Mason? Got fired?
Problem here is that quietly map throws away those keys which happen to get overridden by subsequent result of mapping function. So if we fist mapped “frontend” -> “Mason” it soon after got replaced by “frontend” -> “McCarthy”.
Here it may look simple, but if we have maps of sizes of hundreds or even bigger, and the domain hasn’t that clear rules, the threat may get blurred out. Always map over maps, funny as it may sound, only if you are 100% certain that you will use distinct keys in resulting map. The function f: A => B we saw in the signature must be one-to-one with regard to keys.
How do we fix the problem? One way to go may be to transform the map to list of tuples and then use
It feels a bit more complicated, but just a bit, yet it provides the right result and it turns out Mr. Mason didn’t get fired after all.
Let’s depart from core scala and wander away to the world of actors. It would probably take a bit too long to provide sufficient introduction to Akka, so I will just assume you are familiar with it. To the point. Let’s imagine we have system built of actors that interoperate with each other by exchanging messages (that’s actually a truism, I know) and that we handle some more complex tasks with dedicated actors. We may have some actors that have one instances, state and serve for a pillars of the system, but there are occasional tasks, for which we may want to create a separate actor instance for. Or maybe that one actor sends a message to another and then expects a response - for example some route controller asks repository actor for collection of users:
Those who have worked with actors a bit know that these examples are quite ubiquitous in actor-based systems. We use two Akka patterns here. One is “ask”, which provides the
? operator for
dispatch-and-await-response-future thing. The other is “pipe”, which allows dispatching the results of the future as a message to another actor (not necessarily sender, as it was in our examples). We will concentrate on the former.
It is important to understand what is happening under the hood. But before we shed some light on it, let’s use an example that refers to the
case 1 from the snippet above.
Ok, looks good. Problem is the parent never gets the result. We set the endpoint, make sure the message is dispatched to the parent, but the future times out nonetheless. What’s wrong? After all, didn’t we create the
ServiceActor? Yes, we did! Didn’t we dispatch the message from there? Well, let’s take a look under the hood…
ask pattern creates an internal actor that is invisible from outside. It is only this actor that sends the message to the final recipient and it is this actor that should receive the response. That response is visible from the outside as the
response-future part of the pattern.
So what happens when we use
context.parent ! getResult(taskDef)? We might observe it in logs if only we had used
LoggingReceive instead of
Receive (and by the way I encourage you to do so). With properly configured logging, we would see, that
ServiceActor receives unhandled messages of type
DidTheMath. So instead of landing in the future, it landed right in receive! It’s perfectly correct, if you give it a bit of ponder.
The right thing to do is to send the result back using
sender ! getResult(taskDef). The lesson to learn is that when you use the ask pattern, remember that the response should always sent back to
sender(). Otherwise it will end up as unhandled and you will observe Timeout-related exceptions.
Examples above are Based On True Story®
We wrote these few corner cases down, so you can learn from our mistakes and improve your code. If you seek a better understanding of Scala’s edge cases visit Scala Puzzlers