Scala at Mind Candy

Reading some recent negative commentary about Scala with interest I felt like it would be good to share our experiences with Scala.

The Good.

Scala is an expressive language – It often results in a lot less code getting in the way of what you want to do or achieve. The simple example for something like this would be a simple bit of code like this:

case class Person(id: Int, name: String)
def lookupPerson(id: Int): Person = {
  new Person(id, "Test" + id)
} // Token implementation for the example.
val ids = Seq(1,10,100)
ids.filter(id => id < 50).map(lookupPerson)

To write this in Java would require a whole load of boilerplate, which includes the case class generating the stock class methods like toString, multiple lines to create a collection and then transforming a collection.

More powerful ways to write tests – This can fall under the Spiderman grouping of power admittedly, but libraries like specs2 and scalacheck make it easier to create the kind of tests we wanted. Scalacheck is the stand-out example of this where a generator for creating Person objects from above is as easy as this:

object TestGenerators {
  val personGenerator = for {
    id <- arbitrary[Int]
    name <- alphaStr
  } yield new Person(id, name)
}

That’s all it takes and that object can be imported into all of your test code that needs to generate Person objects.

Less magic – A lot of libraries like Spring and Hibernate need to use byte code modification or reflection to achieve what they do, which means that when they fail it can be really hard to diagnose or fix the problems. We’ve even seen some of these do things non-deterministically, which has caused hours of bemusement. Contrary to this, Scala libraries just tend to use the type system to achieve these ends which means that in our case we catch problems at compile-time, not at run-time.

The REPL – The idea scratchpad and general utility will be your friend in times of need. Use this as an education tool to step through an idea with some else. Use it to test some code interactively if you want to confirm something but you’re not quite sure how to code it or what results you’ll get. Use it to solve those gnarly Project Euler problems without having to create a whole new build for each one.

SBT – Controversial one this may be, but it manages to give you the sensible build model and plugin system that Maven has while allowing you to easily create custom tasks. If nothing else being able to run a command, for example the ‘test’ task, on each save is the most useful thing I’ve seen in a while.

POWAH! – There’s an elegance that comes with time when using Scala, in much the same way that it does with a lot of languages, that means code slots together so cleanly and with little friction. For me personally the Option class was the beginning of this change in thinking, where I realised that representing the possible lack of something without using a null made a lot more sense.

The Bad.

SBT – It’s a double edged sword in you’ll need to understand a bit of Scala to be able to do non-trivial configuration in it. Documentation for this has improved massively in recent times, it can still be somewhat impenetrable, especially to someone new to Scala.

Somewhat idiomatic libraries – Databinder Dispatch is a good example of this, writing a custom handler to parse a HTTP response is just unnecessarily puzzling. As with all libraries how easy they are to use and extend should be evaluated, so don’t be blinded by those libraries just because they’re written in Scala. It’s better to pimp a stock Java library that already works well than to use one that is badly written in Scala.

Binary compatibility – This is the stock issue that is often complained about, fortunately SBT does notice when two versions of the same library that relate to two different Scala versions are pulled into the dependencies. The way others have presented this is as a major pain point, it’s generally only so much of an issue as it is with Maven dependencies with a little more granularity. Also if you’re using SBT it’s possible to create dependencies that tie to the Scala version used automatically.

Knowledge – There’s a couple of aspects to this. The first is that Scala is a “new” language and as such there is one learning curve which relates to the language, SBT, the libraries and how to use them all effectively. Beyond this is that some functional programming concepts are foreign to a lot of programmers and this can be a wall that isn’t scalable in a short period of time for a lot of people. Hopefully with time this will become less of an issue but at the moment there aren’t a lot of Scala developers that can hit the ground running.

The Ugly?

As with all new things, there is a learning curve with Scala, which can be problematic, but the benefit of the design is that it’s possible to do something the “wrong way” as the language is very flexible. People with a history in languages like Java can start out writing code that looks not that much different but still get benefits like better collections. Then with time progress onto using more the powerful features in the language like pattern matching and implicits. For the foreseeable future Scala is a tool we intend to keep using, as it’s been of great benefit to us (this week I parsed a 37GB log file with a couple of lines of code in the REPL), maybe you should too…

99 Bottles of JMeter on the wall

I’ve recently had to do some performance testing on a couple of our new web services. I know of a few handy open source tools available for this. Tsung, Grinder and JMeter spring to mind. I find that I can get up and running in JMeter quicker than I can with the other tools and it tends to be the one I use most. One of the web services I wanted to test required some dynamic content to be generated in the body of the HTTP POST. Normally I’ve been able to get away with using the standard counters and config variables provided by JMeter, but this time I needed a bit more control. Luckily, JMeter allows you to do some scripted preprocessing of HTTP requests and I was able to use this to generate dynamic content within JMeter. I found the JMeter documentation to be a bit lacking in this area, so I created this blog post to give a quick explanation of how I managed to do this.

I’ve created a demo project that you can follow along with to see how it all works. Download the source code here: https://github.com/groodt/99bottles-jmeter Follow the README on GitHub to get everything setup and running. All you need is git, Python and JMeter. Open up the file “Test Plan.jmx” in JMeter to follow along.

The demo
The demo project is a simple web service that parses JSON payloads and prints a modified version of the “99 Bottles of beer on the wall” song onto the console. The JSON payload looks something like this:

{“drink”:”beer”, “bottles”:”99″, “date”:”1321024778956″, “thread”:”4″}

The server then parses these payloads and prints them out to the console:

JMeter then aggregates the response times in the summary report:

The HTTP POST
If you navigate to the “HTTP Request” node in the example you can see the JSON POST body being constructed:

The variables ${drink}, ${bottles}, ${date} and ${thread} are generated dynamically by a script that JMeter executes for each request.

The BSF PreProcessor
The BSF PreProcessor is run before each HTTP request to generate the dynamic content mentioned earlier. The BSF PreProcessor allows you to run Javascript, Python, Tcl and a few other languages inside JMeter. I decided to write my script in Javascript.

If you navigate to the “BSF PreProcessor” node in the example you can see the script that is used:

The Javascript
The simple Javascript basically places 4 variables in scope that are then available for JMeter.

// Calculate number
var count=vars.get("count");
var bottles=99-count;
vars.put("bottles",bottles);

// Calculate drink
var random=new Packages.java.util.Random();
var number = random.nextInt(4);
var drink = vars.get("drink"+number);
vars.put("drink", drink);

// Calculate date
var date=new Packages.java.util.Date().getTime();
vars.put("date",date);

// Calculate thread
var thread=ctx.getThreadNum();
vars.put("thread",thread);
  • In lines 1 to 4, the counter is read from JMeter then 99 is subtracted and the value is placed into scope under the name “bottles”.
  • In lines 6 to 10, a random number from 0 upto 4, it then uses this number as a lookup into the names of drinks (beer, wine, mead, cider) defined in the JMeter General Variables. It then stores this value in a variable named “drink”. It makes uses of java.util.Random to generate the random integer.
  • In lines 12 to 14, java.util.Date is used to generate a timestamp in milliseconds. This value is stored in a variable named “date”.
  • In lines 16 to 18, the JMeter thread number is read from the JMeter context and then stores this value into a variable named “thread”.

Executing Java libraries within the scripts
If you noticed in the scripts above, the Java libraries are exposed in JMeter under Packages.*. This allows you to execute any of the Java standard libraries or Java code in the classpath of JMeter. I think you can also write your own Java code and place it in the JMeter classpath and expose it in the same way.

Putting it all together
Putting all of that together gives you a handy way of doing reasonably complex performance testing in JMeter and I hope you find it useful.