Our thoughts, knowledge, insights and opinions

When old-school developer goes freestyle

Full source code is available on GitHub.

Introduction

This post is about creating relatively simple project with Freestyle, my adventures during this process and some afterthoughts.

Self-description from the project website: Freestyle is a library that enables the building of large-scale modular Scala applications and libraries on top of Free monads/applicatives. [1]

Freestyle logo

I’m limited to single blog post, so I’ll make non-trivial-scale application instead of large-scale one. Precisely it is a module to perform registration and log in with credentials or GitHub’s OAuth 2.0 and http4s server that uses this module in HttpServices (endpoints) for issuing JWT.

This article describes my journey and feelings during it, so it is written in the first person and past tense.

Author context

I guess most blog posts are written by someone either excited or angry about something. This post is not such a case. I got resistant to hypes and I won’t write a post that is a gospel of [put any buzzword here] saving our codebases. On the other hand my previous unsuccessful experiences with free algebras haven’t discouraged me.

Free Monads is a topic which I’ve tried to get into by writing non-trivial applications at least few times but all attempts ended when I thought: All this boilerplate and additional types in codebase are not worth the benefits. Accidental complexity grows too big.”. I’ve judged that I prefer to stick to “traditional” style, which perhaps is not so easily testable, and does not allow to run a program against many interpreters (I’d love to hear from someone who has more than one for tests and one for production). Here, please don’t get me wrong, when I began I was thinking that Interpreter pattern is a good thing, although too expensive, a trade off that doesn’t pay off. Have Freestyle changed my opinion about it? You can read the whole story or jump to conclusion.

How it unfolded

Before I dived deeper in solving somehow real problem that I face in my daily programmers life, the registration and log in, I had started with something really simple, namely arithmetic, just to get used to Freestyle, and ran over against first line of hurdles.

Arithmetic

The target was to write free programs for calculating mathematical formulas. Following quick start I’ve created such algebras:

Freestyle transforms BasicMath and HighMath traits to two separate free algebras. I’ve combined these algebras together in AllTheMath with help of Freestyles @module to obtain new free algebra with operations of both components.

Formulas

The next step was to write a program that uses given algebras or to write interpreters. I chose to write a simple program to evaluate math formulae first, here it is:

Macros

At glance: Freestyle uses macros to generate boilerplate code for you, it expands macros indicated by @free and @module annotations.

There were some negative consequences of using macros I faced. The first one was that my Intellij Idea indicated errors: cannot find symbol FS in @free algebras definition. In fact @free trait HighMath is expanded to trait HighMath[FF$15532[_]] extends _root_.freestyle.internal.EffectLike[FF$15532] and EffectLike contains type alias FS, thus sbt build was OK. I just had to live with “red margin” in the IDE.

The second problem seemed more severe, because it caused sbt build failure!

Fortunately that were missing imports of freestyle._ and freestyle.implicits._. Imports had been repeatedly removed by my beloved Idea, until I disabled organize imports in the IDE.

The last problem I encountered can be painful for ones who like to use sbt REPL. It turns out that console task failed because of org.scalameta % paradise % 3.0.0-M9, which is required by current version of Freestyle. Unfortunately this made working with examples and learning a bit harder.

Very, very positive thing I’ve learned when facing problems with Freestyle was helpfulness of developers on the 47deg/freestyle Gitter channel. They are so open and kind! Thank you guys again!

Id interpreter

After fixing compile-time errors I wanted to get the ultimate reward of a programmer: feeling when the whole code finally works. So I’ve rushed with interpreter implementation. I haven’t been fancy about it, selection between monix.eval.Coeval and Id has been won by the simpler one.

Interpreting program

The finale of my example in the arithmetic domain is running the computation:

In the last line I run program Formulas.`(a+b)^2` in the AllTheMath algebra, interpreted to Id monad. Imports, reported as unused by Idea, by the way, bring in scope my Id interpreters for BasicMath and HighMath and Freestyle machinery (including one that wires my handlers into AllTheMath).

One thing worth noting at this point is that I still can write programs that use only BasicMath and I don’t need anything related to HighMath to run them. Only lines of code that bound them together were in @module AllTheMath.

Type annotations in wrong places

I’ll come back to my formulae definition: def `(a+b)^2`[F[_]](a: Int, b: Int)(implicit A: AllTheMath[F]): FreeS[F, Int] Opinions may be different but I don’t really like, and find it redundant, to add type parameter and implicit parameter. Good news is that there are two ways of dealing with this.

The first one is placing methods inside module definition. I had to rename method, because some non alphanumeric characters in names cause macro expansion failure.

Ironically, it causes dual problem: I cannot define return type, because macro adds type parameter with semi-random name to the trait it operates on.

Second one requires some boilerplate code, but it can pay off in case of big number of methods. Trick is to define class to capture type, like here:

Irrelevant type annotations are gone, the relevant one is present.

Conclusion

That’s it for simple case of arithmetic. I’ve run into some problems with not found symbols, and had to find out how to place types annotations, but that just happens in learning period. I’ve become resistant to IntelliJ Idea false positives and imports management. In the end amount of boilerplate was acceptable for me. After this prelude I was ready to solve original problem.

Log in

Log in domain at glance:
  • allows to sign up - register an account
  • allows to log in - obtain JWT
  • sign up and log in are possible with email and password or GitHub

This tiny slice of real log in domain was already demanding enough for my goals. It turned out that I needed LoginDatabase and GitHubClient at least. I wanted to have logging effects under control to be able to test if the most important events were not left without a trace, that brought Log algebra in. Fourth and the last component of free algebras set was JwtService.

You don’t really need to contemplate my algebras deeply, I write them down mostly to show volume of what is required.

Further, I’ve joined (somehow following Freestyle getting started guide) GitHubClient with Log to obtain GitHub and put LoginDatabase together with Log to get Persistence.

Final programs required Deps, a module with all algebras.

At this point I could see huge benefit of Freestyle: it’s macros derived a type for each @free trait method and required types coproducts for each @free and @module. Thus, it really did reduce amount of boilerplate by great extent!

You can see that I’ve defined a method inside module GitHub leaving it without the return type annotation. For more advanced LoginPersistence programs I’ve really wanted to see return types. I’ve defined PersistencePrograms boilerplate class, but I find it fair trade off.

LoginPersistence.scala

Log in programs

Let me share implementation of two out of four programs: log in with password and register with password. I’ve used a helper (boilerplate) class to capture types again:

Programs.scala

For registering user I’ve chosen to use inner methods:

I nested for-comprehensions in login:

I’ve found out that when using Freestyle sometimes I need to put a lot of type annotations, you can observe this in case of both methods. To be honest I don’t know why, because in PersistencePrograms it works without them. Besides that (and that I use a monad for logging in Scala first time) these programs doesn’t look very different to ones I would write if I have committed to some particular monad or used tagless final style. In some time I’ve gained skill to quite quickly deal with errors caused by too few type annotations.

Handler

Writing handlers is nothing more than implementing interface in particular monad. Because I’ve used http4s in this project I’ve chosen to interpret in fs2.Task. There no are significant differences to imperative style. Nice thing was, that I could very easily obtain handlers for fs2.Task when I had cats.Id handlers. That holds true every time when there is natural transformation from one monad to another.

Links to implementations of interpreters for interested readers:

Wiring up

To run programs for compound free algebra in target monad, an interpreter for it is required. User has to provide interpreters for each component algebra in implicit scope only, because one of super-powers of Freestyle is that it constructs compound interpreter from implicitly available interpreters for you. I’m really glad that I didn’t have to make it manually.

Main.scala

Such code reminds me my preferred style of (static) dependency injection in Play Framework.

Next example presents using implicitly available interpreters, creating my Programs instance and interpreting one of programs to fs2.Task.

LogInService.scala

The code above concludes effort of using Free Monads with Freestyle. In the end there is some boilerplate in production - factitious classes (Programs and PersistencePrograms) for sake of having return type annotations in public methods and extra type annotations in functions bodies. Personally, I didn’t find that I needed huge amount of extra code to have free algebras based application. Of course, I didn’t reach this satisfying outcome easily, but I was still in early learning phase.

Testing

Free algebras way of writing programs yields two kinds of objects to test: interpreters and programs. I haven’t implemented tests for interpreters because I’ve assumed that it won’t differ from testing regular services implementations.

When it came to testing programs, I had a real problem in finding what is endorsed way of doing it. Ok, I appreciate you may now think: “this guy is really not-so-bright”, but I’ll dare to be honest: this lack of guidance, and problems with finding it, made me lose some time in unproductive doubts. Finally I’ve used an advice from Gitter channel and have used mocked interpreters. This way I could easily inject failures when testing error paths. I’ve written these mock interpreters manually, because Freestyle handlers methods are protected[this], thus one can’t mock them with scalamock.

Not finished specs for register/login programs can be found in ProgramsSpecs.scala.

There is other thing that worries me about testing when using Free Algebras: a problem with lack of hierarchy and scalability.

For example when testing Programs I cannot mock PersistencePrograms (class I’ve created to have return type annotations, not for dependency injection). It contains pure functions, so there is no reason to mock them, you can tell, but then I have to mock all their dependencies! I have to fit all the problem in my head at once, what turns out to be hard sometimes.

On the other hand, again, it’s a trade off with a benefit: I couldn’t make false assumption about how PersistencePrograms works and “mis-mock” it.

Short recap:
  • testing interpreters is not different
  • testing programs requires to mock all (leaf) interpreters
  • one won’t make mistake in the middle levels
  • using scalamock is hard because Freestyle protects handlers methods

Other problems

Doobie trap!

Freestyle provides DoobieM, with def transact(c: ConnectionIO) method, for support of Doobie database access. Helper for transforming Transactor to DoobieM.Handler is in place. Wiring Doobie in is easy. The trap is in testability. I’ve found ConnectionIO really untestable, I wasn’t able to match on queries or updates in my test handler and I don’t believe that H2 is a solution for tests when in production other database is used. I’ve decided to throw DoobieM code away and implement my own, higher level, algebra for interacting with DB. If I had written tests for its interpreter, then I would provision some DB and place these tests in integration suite.

IDE

I don’t want to speculate about reasons but that is just a fact: I have been restarting Idea, few times per day at least, when playing with Freestyle. I turned type-aware highlighting off but it didn’t help a lot. IDE still was very sluggish and I was really irritated sometimes. Whole system was affected. This really was a huge blow for my productivity, the biggest impediment I’d say.

Conclusion

I started evaluation of Freestyle with mixed feelings about Free Algebras programming, perhaps more sceptic than enthusiastic. Some people even doubt that it is feasible with current programming languages but macros based approach that Freestyle brings to the table looks promising. There are some areas to improve, two most important for me I’ve pointed already: problems with types annotations and suffering IDE. I’m happy with code that I’ve written but process was quite painful few times. Although Nice Guys at Gitter eased the pain! I think the ultimate question is: would I use Freestyle in my next commercial project (let’s assume I make it alone)? Today honest answer is: no, I would not. But I truly hope that it will change soon. Cost-benefit-analysis significantly changed widely for me, Freestyle reduced boilerplate and accidental code tremendously! For sure I’ll keep an eye on Freestyle.

Links

  1. Freestyle
  2. Doobie
  3. http4s
  4. Project repo

Post Scriptum

Please take a look at type signature for Deps free algebra after stripping all packages names:

You like this post? Want to stay updated? Follow us on Twitter or subscribe to our Feed.