MadMode

Fun and Frustration with Scala

In a September item, Martin Kleppmann says:

Scala in 2009 has the place which Python had in 2004.

I bookmarked Scala (the language; not the band ;-) back in June 2007, but I didn't find a good excuse to try it out until Alexandre Bertails, the new W3C webmaster, suggested adding scala to the php/perl/python/java mix that powers w3.org. He gave a great PreparedKata on scala. I have now built a couple little projects using Scala. The experience brings me back to a June 1996 Usenet posting, where I wrote:

Modula-3 was more fun to learn than I had had in years. The precision, simplicity, and discipline employed in the design of the language and libraries is refreshing and results in a system with amazing complexity management characteristics.

I have high hopes for Java. I will miss a few of Modula-3's really novel features. The way interfaces, generics, exceptions, partial revelations, structural typing + brands come together is fantastic. But Java has threads, exceptions, and garbage collection, combined with more hype than C++ ever had.

I'm afraid that the portion of the space of problems for which I might have looked to python and Modula-3 has been covered -- by perl for quick-and-dirty tasks, and by Java for more engineered stuff. And both perl and Java seem more economical than python and Modula-3.

I'm happy to say that I was wrong; python matured quickly enough that I use it for most of the spectrum. The libraries matured quickly enough to allow me to get away from perl. And I'm pretty happy that I avoided Java long enough for scala to come along and fill in the bits of Modula-3 that Java lacks.

The main reason I never did pick up Java is that the main part of my job was project management, i.e. on the manager's schedule, and an hour isn't enough to do any software engineering. It is enough time to write, test, and document some python code! I'm doing more software development these days; working on the UI part of a Science Commons project last summer finally gave me several days in a row to dig in and learn JavaScript development. And I had to interface to a Java API in JMOL, so I dipped my toe in the Java waters using Jython. I got it working, but since I largely depend on doctest mode for emacs and never got jython working there, it's only manually tested.

I can now write, test, and document scala code, though it's about equal parts fun and frustration at this point.

The first frustration was finding that there's nothing like the python tutorial on the scala web site. The tour of scala was very tasty, but didn't teach me enough to read scala code and be confident about what's going on. I tried reading the language spec, but got lost in abstractions (that's one thing Java has over scala; GJS's Java spec is a joy to read). Alexander eventually got me to read the ebook, which is quite good, though not freely available. Shortly after that I discovered the video of Martin Odersky's FODEM talk; I think that one pleasant hour could have substituted for several earlier frustrating hours on the scala web site. And I discovered the O'Reilly scala book; people say it's nowhere near as good, but I'm going to try to migrate to it for reference purposes, since I can more easily share what I find there.

The next frustration I feared was giving up emacs in favor of a modern Java IDE. But the friendly folks in the #scala channel assured me it wasn't necessary:

<DanC>    I'm an emacs addict, but I gather the way to do
scala is with Eclipse
<paulp>   DanC: don't know where you gathered that but I
would bet eclipse user a minority.
<DanC>    oh.

 <DanC>    what do you use?
<paulp>   textmate.
<dcsobral>    jEdit here.

I did give up make for simple build tool (sbt); I only miss it a little; sbt emacs integration is pretty raw and next-error gets out of sync about which line to go to (workaround: restart sbt-shell). Flymake looks cool, but I haven't managed to get it working.

Giving up doctest is much harder. I learned to use scalatest, but it's no it's tedious and using the 1.0 version requires using unreleased versions of sbt (which worked fine for me). ScalaCheck is even more bothersome, as it uses level 12 scala type inference magic while I'm only a level 4 apprentice, but at least it rewards you by generating zillions of test cases for you. None of the scala test frameworks are integrated with scaladoc, the documentation framework. Every time I had to fill in a test name or description I'd think "Why is this not integrated with docs? An interpreter and REPL are as much a part of the scala culture as the python culture; surely there's a doctest for scala out there" and go searching. No joy. I did find a couple starts at doctest for Java (they use JavaScript for the REPL; Java itself just doesn't work that way). I eventually got fed up enough to start my own doctest.scala, though it's not feature complete enough to use yet.

"Beautiful is better than ugly." says the Zen of Python, and scala feels pretty elegant. But the next aphorism is "Explicit is better than implicit." Java clearly takes this too far with

FileInputStream x = new FileInputStream(file);

Telling the compiler type type of x once should be enough, and with scala, it is. But scala has lots more magic that, all together, can make it hard to read. The complexity shows up in the compiler diagnostics, which I find misleading more often than not. Scala has parallel namespaces for types and values; it's kinda cute, but consider this diagnostic:

 [error]
/home/connolly/projects/rdfsem/src/test/scala/rdfstdtest.scala:137:
not found: value Graph
[error]     val manifest = Graph(WebData.loadRDFXML(args(0)))
[error]                    ^

I sit there pulling my hair out, saying "Graph is imported 10 lines up; are you blind?!?!?!" But what I imported was the type, not the value. The real problem in that line of code is that scala is like java in using a new keyword for instantiating (most) classes, but python habits die hard.

And that's just the beginning when it comes to mystifying compiler diagnostics. Be very afraid of "Missing closing brace `}' assumed here." The missing brace may be very, very far away. The ScalaCheck docs really need a special decoder ring due to its use of higher order magic; check this out:

[error]
/home/connolly/projects/rdfsem/src/test/scala/strquot1.scala:33:
missing parameter type for expanded function ((x0$1) =>
x0$1 match {
[error]   case (s @ (_: String)) =>
dequote(quote(quote(s))).$eq$eq(quote(s))
[error] })
[error]     Prop.forAll((genQuotEsc) {
[error]                              ^

That "case (s @ ..." stuff isn't in my code; the compiler magically conjured it up. I only know from monkey-see-monkey-do reading of the ScalaCheck docs that the right answer is:

    Prop.forAll(genQuotEsc) {

Here the compiler is being sadistically misleading:

[error]
/home/connolly/projects/rdfsem/src/test/scala/rdfstdtest.scala:103:
not enough arguments for method apply: (n:
Int)org.w3.swap.logic.Term in trait LinearSeqLike.
[error] Unspecified value parameter n.
[error]     println(manifest.each(u, rdf_type, what).mkString())
[error]                                                   ^

My sin in this case was to break the rules for methods without parentheses.

Many thanks to RSchulz and company in #scala for taking my side in several battles against the compiler's disinformation campaign.

Once that battle is over, life is much more fun. That is, after all, much of the value proposition of statically typed languages, though the global consistency guarantee in the language and build tools comes with a downside that when you change a type, you can't just test a few modules without getting everything in sync.

My debugging tool so far is the trusty println(). When my code hangs, I'm used to hitting ctrl-c and getting a python backtrace. The java runtime, and hence scala runtime, just quits with no backtrace when you hit ctrl-c. Ouch.

When I asked about debugging and profiling tools in #scala, the suggestions I got were about various GUI tools, many of them commercial. I managed to get IDEA with the scala plugin configured to navigate my code, but it took 20x longer than sbt to build, and before I managed to learn to use its debugger, I spotted the bug myself. For profiling, java -Xprof worked just fine for my needs, though jvisualvm is free and packaged by Ubuntu and I did get it to attach to my running code; I'm still stumped about how to get it to tell me which methods are taking the most time, though.

I like the idea that scala is now where python was a few years ago, i.e. that the frustrations that I'm running into are rough edges that will get smoothed out soonish. The cascade that started with scalatest 1.0 requiring using an unreleased version of sbt continued thru using version 2.8.0.Beta1-RC5 of the compiler and libraries. I still love python, but I'm happy to restore an elegant statically typed languge to my toolset after Modula-3 went fallow, especially one that interoperates with the java platform everywhere from android mobile devices to Google App Engine.

tags: programming