Amsterdam loves Scala!
Last week Amsterdam was hosting ScalaDays 2015 conference. I’m really happy that a big Scala event has finally reached The Netherlands.
There is a huge Scala community with relatively long history in here. First Scala user groups existed here since 2009-2010. Nowadays the biggest one, Amsterdam.Scala meetup, has more than 500 members. With events happening every 2-3 weeks (hackathons, talks, workshops) and awesome speaker line-ups123. It’s far from the biggest Scala community in Europe, but definitely one of the most active ones (hard to compete with London Scala UG though :)
So I’d like to share my thoughts on some talks I found particularly interesting.
Martin Odersky and future of Scala
It all started with Martins keynote. As usual, he spent some time explaining how language was evolving and what exactly he understands by ‘Scalable language’ term.
I really admire some irony he usually adds to his talks:
But I guess the most interesting part was about Scala future.
Making Scala TASTY
Meet TASTY (Typed Abstract Syntax Trees) - an intermediate format for compiled Scala classes that aims to bring more backward compatibility for Scala libraries.
It’s not suprising at all that it’s super important for Typesafe to solve binary compatibility issue, in order to satisfy Enterprise players and defeat this long-time counter-Scala argument. Overall, it looks to me as a better way to address that rather than just slowing down innovation as was suggested in ‘Scala-should-be-boring’ Rod Jonhsons talk that was given at ScalaDays 2013.
Better type system with Dotty
Dotty - is a EPFL playground language platform for new generation of Scala with new and refined features. With its generalised union/intersection types, implicit function types and etc it surely attracts a lot of attention amongsts typelevel people.
It seems natural that two initiatives (improving binary compatibility and evolving the language) are coming together. And the former one should ease introducing the latter. Nevetheless it will probably take at least a couple of years till we get a chance to start playing with new Scala major version.
Expressing effects in types
Another interesting concept that Martin mentioned is a ‘better treatment of effects’ which is essentially about being able to express code side effects (throwing exceptions, IO, null-dereferencing) via type system.
So on the second slide
throws keyword from the first method signature is actually just a type
throws[R, Exception] that requires to have an implicit
CanThrow[Exception], so called capability, in scope. Looks cool, isn’t it? :)
However, only time will show if it’s feasible to make it ‘non-boilerplatty’ and useful.
Jonas Bonér: “Life Beyond the Illusion of Present”
The next keynote by Typesafe CTO Jonas Bonér was about essential concepts of distributed systems design. I always enjoy his talks on this topic. They are great at introducing people to new ways of thinking and reasoning about state and events in distributed systems.
“Tracking time is actually tracking causality.”
That’s what important: Causality and events Ordering.
- There is no such thing as reliable time or guaranteed message delivery.
- Think in facts.
- Never do in-place updates, just collect and apply new events/facts.
- CRDTs to the rescue!
Event sourcing is the key. Updating the locally observed state is simply merging new facts with old ones:
Akka Actors vs Types
I’m a true fan of Akka framework. It is a great piece of software and it really helps building scalable applications faster and more effectively. But lack of type safety when working with actors was always a bit … strange :).
Development with Akka nowadays largely depends on knowing common conventions and patterns. Could be cool if a type system, rather than just docs (which are awesome for Akka) and blogs, guides you in the right direction.
New typed Actor API proposal made me think: “Yes, that’s a proper way to do it in Scala, language with a rich and strong type system.”
Actoris characterised from now by a
Behavior[T]represented as a partial function
Message | Event => Behavior
- No more overridable methods like
postStop, etc: you can define reaction to these Signals in behavior function.
- No more direct reference to actor
- No more built-in
sender(): you have to encode a sender reference to your message API.
Latest Akka Typed docs and examples can be found here: http://doc.akka.io/docs/akka/snapshot/scala/typed.html
Lambda Architecture & Spark Streaming
Another notable talk was by Helena Edelson - “Lambda Architecture with Spark Streaming, Kafka, Cassandra, Akka and Scala”.
She demonstrated how you can build stream processing pipelines leveraging high composability of this stack.
Althought a bit too much Cassandra advertising (I like C*, but still), it was cool to see some snippets how you can glue these things together and what are potential use-cases. And of course Spark connector for Cassandra is pretty awesome. Now other NoSQL vendors start understanding4 necessity of Spark-support as well. :)
Few years ago Storm was almost a standard component of ‘Lambda architecture’. Event the term itself was coined by the creator of Storm. Now Spark Streaming together with Akka Streams, Kafka and others might get this place.
Monad Transformers for Mere Mortals
How often do you have to deal with nested containers? For example,
Composing them together in for-comprehensions is not than easy. Consider this code sample from Erik Bakkers presentation “Options in Futures, how to unsuck them”:
To solve this, people usually end up creating a set of utility functions, implicit transformers, like FutureOption[T], that allow you to flatten containers. Althought it works, you still have to steal^Wwrite these.
Consider another solution using Scalaz
OptionT, a so called Monad Transformer:
Sometimes it’s a valid point :), but I prefer to borrow just essential stuff.
ValidationNEL of Scalaz, or
HList of Shapeless, or these Monad Transformers
\/ that Erik demonstrated.
Slides from Eriks workshop: https://speakerdeck.com/eamelink/flatten-your-code.
I can also recommend another great presentation on Scalaz ‘gateway drugs’ by Brendan McAdams: https://speakerdeck.com/bwmcadams/scaladays-sf-2015-a-skeptics-guide-to-scalaz-gateway-drugs.
Followed by their work on Spores Heather Miller and Philipp Haller presented this year a new distributed programming model based on Spores: “Function-Passing Style, A New Model for Asynchronous and Distributed Programming”.
- In this model, unlike with traditional data passing, you actually send a serialized clojures that are executed on the remote node against target data.
- The data holder, or Silo, resides on the node that has target data. Location transparency in system is achieved by introducing a SiloRef which encapsulates lazy application of the clojure to target data in remote Silo.
- Serializability of clojures is guaranteed by Spores.
- There is a limitation of Spores: nodes running application code should have the same classpath (otherwise deserialization will fail).
It’s still unclear to me how to handle concurrent updates to the same data. Probably it’s out of the models scope and can be tackled by using CRDT-like data structures or just immutable data.
Overall I think it’s great and makes a lot of sense for data-crunching systems like Spark where data locality is the key and sending it over the wire is not an option (because of the volumes).
Rapture Result and Modes
Jon Pretty presentation with a quite complicated name (“Delimited dependently-typed monadic checked exceptions in Scala”) attracted my attention because of two things:
- Interesting concept of abstracting the actual return type of library API (using so called Modes).
- A new
Resultcontainer similar to
Rapture.io Modes allow users to choose the return type of the library API:
It’s looks like a cool magic, but I see a couple of different scenarios where this might be useful:
- Turning library API to blocking for tests and to async in your main code.
- Switching between, for example,
Try-based API without having to write your own wrappers.
Return type looks something between
I like the trick of simulating union types using ‘with’:
This is useful here as
Errata is actually a typed Multimap.
Kamon and Akka metrics
There is a huge demand for Akka monitoring nowadays. Typesafe console was looking great, but tracing performance was … awful.
Authors of Kamon are trying to meet that demand with their toolkit for collecting actor metrics and traces.
Ivan Topolnjak showed in his talk, “Kamon: Metrics and traces for your reactive application”, how they were able to minimise performance impact of metrics collection to the system. It was interesting to know that they are using HdrHistogram for this.
HdrHistogram (High Dynamic Range Histogram) is “A Histogram that supports recording and analyzing sampled data value counts across a configurable integer value range with configurable value precision within the range”5. And it is really-really fast. :)
Tracing actor message flows is another important problem which is not fully solved yet. There is a still experimental Akka Tracing extension. And Kamon also has some of these capabilities: http://kamon.io/core/tracing/core-concepts/. It would be cool to eventually have a visualization for flows like the one Twitter Zipkin provides.
Also would be nice to be able to trace composite
Other notable talks
- Amanda Laucher: “Types vs Tests : An Epic Battle?”. That was a great talk and its old version is even available on InfoQ: http://www.infoq.com/presentations/Types-Tests
- Richard Dallaway: “Towards Browser and Server Utopia with Scala.JS: an example using CRDTs”. Follow up blog post is here. Cool example of coding CRDTs in Scala.JS.
- Josh Suereth and Eugene Yokota: “The road to sbt 1.0 is paved with server”. That was an entertaining talk about new sbt features and how sbt-server is going to change our lives. =)
I will update this post with the links to videos as soon as they become available on Parleys.