Using giter8 to build Scala REST projects with Unfiltered, Netty and Gatling load testing

At Mind Candy we have a number of different Scala REST services to provide common aspects to our games, for example authentication and moderation.

A standard we have developed is to use an Unfiltered Netty standalone server wrapped with the apache-commons daemon library. Also, we want our services to be well tested, easily deployable, and stable under heavy load. For ease of deployment we use Fabric scripts, and for load testing we use Gatling to hammer the services under different stress-tests to check they are up to standard.

We have quite a few services that have a similar setup, and it was getting tedious to create and configure a new project. So, we created a giter8 script to do it all for us!

Giter8

Giter8 is an awesome command line tool which can create a template project structure for you with example files and sbt configuration.

There are quite a few templates contributed by the community already but none did exactly what we want, so we created our own which we’d like to share.

The template will create an example sbt project that:

  • Uses unfiltered and netty to give you a very simple starting endpoint service which will output ‘Hello, <project>’ when hit
  • Gives you some basic test stubs using Specs2
  • Is setup with gatling load testing with an example scenario
  • Is configured with the sbt-idea plugin (IntelliJ IDEA is our preferred IDE)
  • Is configured with the sbt-assemblysbt-release, and sbt-dependency-graph plugins
  • Has a simple fabric deploy file, unix start/stop scripts, and an example init script

The generated project is structured with 3 sub projects:

  • project-example
    • example-core // Business logic
    • example-resources // Containing the Unfiltered Netty request plan
    • example-server // To control the server hosting the plan

Usage:

First make sure you have setup giter8 using the instructions here. Then fire open a terminal and run:

g8 mindcandy/unfiltered-rest-gatling

There are some options to enter, but most can be left as default (See here for more info). The important ones are:

name => The main projects name
project => The name prepended to each of the sub projects
organization => Used as the basis for packages

Once you’ve entered the properties, giter8 will do its magic and whip you up a full sbt project. After it completes, you can cd to the directory, and run sbt.

Test it:

sbt test

Run it:

sbt run

Now open up your favourite browser and go to http://localhost:8889/<project> where <project> is whatever you specified above, and you should get a “hello, project” message back.

Load test it:

Start the server running in one terminal. In another terminal do:

cd yourproject
sbt
project <project>-server
test-load:run

You should get presented with a choice of different classes to run:

  • Engine → Executes the gatling Engine in interactive mode prompting you which scenario to run
  • BatchEngine → Runs all simulations available in a batch with no user prompts
  • Recorder → Starts the pretty cool Gatling recorder which you can use to create scenarios. Basically, you setup your web browser to use the recorder’s proxy, and then just browse as normal on your webpage. Gatling records all of your actions as a scenario that can be replayed and customised.
Hit the option for BatchEngine, then load the results in your browser to see something like:

Results from Gatling are put in the gatling/results directory. In the example scenario in the giter8 skeleton we simulate 10 users hitting the simple endpoint, ramping up over a 3s duration. The results are rendered in a nice html page with graphs showing exactly what happened during the simulation. The full gatling feature set is quite extensive and worth checking out.

Create IntelliJ IDEA project files:

sbt gen-idea

Run from start script:

First setup jsvc. Then :

sbt assembly
cd bin
chmod +x start.sh
./start.sh

This has the same affect as ‘sbt run’, but using the apache-commons daemon wrapper. Try opening the service in your browser again and it should work as before. Don’t forget to stop the server when you’re done with the ../stop.sh script :)

Other pieces:

There is also a sample init script that is configured for the skeleton in the bin folder, which is a good starting point for creating a complete init script.

Last to mention is there is a small fabric file that provides a (very) basic setup for copying over the built assembly onto a test/integration server. You’d need to configure your host and change the paths as appropriate to use it, but it’s useful as a starting point.

Let us know if you find this useful, and thanks to n8han for providing a great tool!

Scala at Mind Candy

Reading some recent negative commentary about Scala with interest I felt like it would be good to share our experiences with Scala.

The Good.

Scala is an expressive language – It often results in a lot less code getting in the way of what you want to do or achieve. The simple example for something like this would be a simple bit of code like this:

case class Person(id: Int, name: String)
def lookupPerson(id: Int): Person = {
  new Person(id, "Test" + id)
} // Token implementation for the example.
val ids = Seq(1,10,100)
ids.filter(id => id < 50).map(lookupPerson)

To write this in Java would require a whole load of boilerplate, which includes the case class generating the stock class methods like toString, multiple lines to create a collection and then transforming a collection.

More powerful ways to write tests – This can fall under the Spiderman grouping of power admittedly, but libraries like specs2 and scalacheck make it easier to create the kind of tests we wanted. Scalacheck is the stand-out example of this where a generator for creating Person objects from above is as easy as this:

object TestGenerators {
  val personGenerator = for {
    id <- arbitrary[Int]
    name <- alphaStr
  } yield new Person(id, name)
}

That’s all it takes and that object can be imported into all of your test code that needs to generate Person objects.

Less magic – A lot of libraries like Spring and Hibernate need to use byte code modification or reflection to achieve what they do, which means that when they fail it can be really hard to diagnose or fix the problems. We’ve even seen some of these do things non-deterministically, which has caused hours of bemusement. Contrary to this, Scala libraries just tend to use the type system to achieve these ends which means that in our case we catch problems at compile-time, not at run-time.

The REPL – The idea scratchpad and general utility will be your friend in times of need. Use this as an education tool to step through an idea with some else. Use it to test some code interactively if you want to confirm something but you’re not quite sure how to code it or what results you’ll get. Use it to solve those gnarly Project Euler problems without having to create a whole new build for each one.

SBT – Controversial one this may be, but it manages to give you the sensible build model and plugin system that Maven has while allowing you to easily create custom tasks. If nothing else being able to run a command, for example the ‘test’ task, on each save is the most useful thing I’ve seen in a while.

POWAH! – There’s an elegance that comes with time when using Scala, in much the same way that it does with a lot of languages, that means code slots together so cleanly and with little friction. For me personally the Option class was the beginning of this change in thinking, where I realised that representing the possible lack of something without using a null made a lot more sense.

The Bad.

SBT – It’s a double edged sword in you’ll need to understand a bit of Scala to be able to do non-trivial configuration in it. Documentation for this has improved massively in recent times, it can still be somewhat impenetrable, especially to someone new to Scala.

Somewhat idiomatic libraries – Databinder Dispatch is a good example of this, writing a custom handler to parse a HTTP response is just unnecessarily puzzling. As with all libraries how easy they are to use and extend should be evaluated, so don’t be blinded by those libraries just because they’re written in Scala. It’s better to pimp a stock Java library that already works well than to use one that is badly written in Scala.

Binary compatibility – This is the stock issue that is often complained about, fortunately SBT does notice when two versions of the same library that relate to two different Scala versions are pulled into the dependencies. The way others have presented this is as a major pain point, it’s generally only so much of an issue as it is with Maven dependencies with a little more granularity. Also if you’re using SBT it’s possible to create dependencies that tie to the Scala version used automatically.

Knowledge – There’s a couple of aspects to this. The first is that Scala is a “new” language and as such there is one learning curve which relates to the language, SBT, the libraries and how to use them all effectively. Beyond this is that some functional programming concepts are foreign to a lot of programmers and this can be a wall that isn’t scalable in a short period of time for a lot of people. Hopefully with time this will become less of an issue but at the moment there aren’t a lot of Scala developers that can hit the ground running.

The Ugly?

As with all new things, there is a learning curve with Scala, which can be problematic, but the benefit of the design is that it’s possible to do something the “wrong way” as the language is very flexible. People with a history in languages like Java can start out writing code that looks not that much different but still get benefits like better collections. Then with time progress onto using more the powerful features in the language like pattern matching and implicits. For the foreseeable future Scala is a tool we intend to keep using, as it’s been of great benefit to us (this week I parsed a 37GB log file with a couple of lines of code in the REPL), maybe you should too…