Learning Erlang #1

after attending dotScale 2016 I finally got enough motivation to properly start learning Erlang.

From past experience I know that I need a project to play around with while learning a new language. So I decided to work on an outstanding task for traq, my CLI time-tracking tool: a JSON API to use as a remote storage backend.
While rsync (or dropbox, fwiw) would work great as a solid synchronization mechanism I want an API which allows me to write a mobile app for traq someday.

I’m using rebar3 for the project, and I really like that it’s enforcing OTP conventions and best practices. This way I can focus on my project; the only thing I’m really missing is a standalone formatter. erl_tidy produces code I can not (yet) make much sense of, so I stick to my own source formatting; ideally I’d like to not think about this.

If you happen to write something like rebar3 fmt please reach out, would love to test & use a tool which unifies source formatting.

The API is using cowboy as HTTP server and jsx for json en- & decoding. The combination of both modules works good; I’m only missing some minor things:

  • a comprehensive tests-first guide to cowboy. Information is scarce on how to write acceptance/ integration/ unit tests for http handlers. I’ll probably use etest_http for acceptance tests; the official cowboy code base seems to use eunit heavily, but there seems much boilerplate involved; I’ll probably find a minimal example sooner or later.
  • live-reloading of routes in cowboy. sync works great for everything but routes. Probably requires some additional setup.
  • jsx seems to be unable to encode an array of objects:

    jsx:encode([
        {<<"an">>, <<"example">>}
    ]),
    

    will produce

    {"an": "example"}
    

    but not

    [
      {"an": "example"}
    ]
    

    But maybe I’m just overlooking something.

Since I’m very much a beginner at Erlang I will probably learn how to fix all of the above points and get much more fluent with the language in general.

Until then I’ll use the official documentation as much as possible, and Learn you some erlang for specific topics.

All other things aside the current setup works quite well for learning Erlang, which I didn’t expect at first.
Also I’m looking forward to sharing more about my journey into Erlang :)


The five important abilities of systems

Reading The Architecture of Open Source Applications, especially Chapter 4, struck a cord with me:

The authors, Margo Seltzer and Keith Bostic, state, that building systems based on simple components as building blocks lead to systems which are superior to monoliths in the five important -abilities: understandability, extensibility, maintainability, testability, and flexibility.

This made me wonder how to clearly differentiate between these abilities, since many discussions about software architecture evolve around one or more of these abilities.

Also the abilities are rather generic, making them applicable to system design in general, so I consider a strong understanding of them essential to be able to perform well.

Here are some definitions & thoughts:

understandability

describes if the code clearly communicates its purpose to readers other then the author.

This is important because code is read more often than written and if your code is hard to understand it will take up more time to read, or worse, it won’t be understood at all; Problem is this is a subjective metric - it depends on the knowledge & background of the reader.

To me, this effectively comes down to:

  • don’t be clever.
  • make implicit assumptions explicit.
  • optimize for readability.

extensibility

speaks to a system being able to accommodate new functionality or changes over time, without requiring bigger rewrites of existing code.

I think every software engineer has experienced changes cascading through their code base. Yet I think it’s hard to avoid sometimes. When building a system, chances are you don’t know every thing up front, meaning you can always run into this situation.

While I think extensibility can be achieved in parts by building loosely coupled, eventually consistent systems, it’s better to perform technical spikes regularly, to ensure the problem at hand is understood properly, as well as possible integrations into the existing code base.

maintainability

describes the afford required to keep a system in good shape, e.g. the amount of work required to regularly upgrade your dependencies, fix defects; but also incorporate non functional requirements after a system has launched.

Since I don’t know of a better way to describe this I’d say “how often do you upset your product owner?” - mostly because product owners always want new features, and you can’t deliver them if you’re busy keeping the system up.

testability

measures how easy it is to write test for your system.

Good testability often speaks to an easy understand code base; It’s also extremely important as a safety net when thinking about maintenance. I use tests as an indicator on how easy it is to understand a system. If you’re tests don’t immediately point you to the source of a bug, or require so much setup that it’s hard to even find the testing code, you don’t have this ability.

flexibility

describes how your system adapts to external changes.

External changes for a system range from slow responses from 3rd parties over occasional timeouts to unavailability.

To me, this mostly concerns operations but is extremely important to get right. If done wrong, your software will be hard to keep up and running.
I’d always argue: if it is worth building a system to begin with, it should always be kept available.

Fortunately, many architectural patterns exist to enable this.

The challenge

I think that these abilities build upon each other; and you always have to find trade offs since you can’t fulfill all of them at once. And trade offs can only be discussed in a specific context, meaning there is no one size fits all solution.

For example, rather generic trade offs:

  • when building a technical spike, it might be okay to write less understandable & maintainable code, since your goal is a proof something is actually doable. When you have to go live with one, however, you should invest time in rebuilding critical systems/ part of a system, which are hard to understand/ maintain.
  • when taking over an existing project on should check if the system reaches the desired level of flexibility & maintainability. Nobody wants to be stuck with a system that fails all the time.
  • when starting a new project, the project expectations should be stated explicitly. E.g. when the project team is expected to change often, writing understandable and testable code should have higher priority, to reduce onboarding time.

But also more specific ones:

  • hard to understand code is hard to test. You are probably missing good abstractions.
  • introducing abstractions just for tests makes your code harder to understand.
  • decoupling two systems, e.g. with a circuit breaker, also make it less easy to understand; simply because there’s more code to read.

If you think it’s simple, then you have misunderstood the problem.
- Bjarne Stroustrup

My key take away is this: software engineering is hard, and it helps to have some guidelines in mind when trying to make the best decision for the moment.
I think these abilities help with that.


docker-compose and load balancing

We’re using docker-compose in production @work and I needed an ad-hoc, lightweight solution to load-balance requests across multiple instances of a specific service, all running on a single host. While docker-compose allows you to scale your services you need to take care of load balancing yourself.

Before sharing my approach, which uses golang, let’s start with the problem description first.

I’ll be using a dummy service, httpd echo, for demonstration purposes.

Given a docker-compose.yml like this:

echo:
  image: nicolai86/http-echo
  ports:
    - 9090:9090

Assuming you need to scale echo, the above docker-compose.yml won’t allow you to scale echo because you defined a port mapping:

$ docker-compose scale echo=2
WARNING: The "echo" service specifies a port on the host. If multiple containers for this service are created on a single host, the port will clash.
Creating and starting 2 ... error

ERROR: for 2  failed to create endpoint dockercomposelbexample_echo_2 on network bridge: Bind for 0.0.0.0:9090 failed: port is already allocated

Removing the port mapping allows you to scale your container but now you need to route the traffic using a reverse proxy.

Available solutions include using a HAProxy / Nginx, with dynamic configuration reloads when your instance pool change. However, these solutions require at least two new, huge-ish docker containers, because of the dependencies required to make this work (did I mention fast access to the internet is a problem?).

Since golang already comes with a ReverseProxy implementation I decided to roll my own reverse proxy with docker-compose integration.

It works like this:

echo:
  image: nicolai86/http-echo

proxy:
  image: nicolai86/docker-compose-reverse-proxy
  ports:
    - 80:8080
  volumes:
    - "${DOCKER_CERT_PATH}:${DOCKER_CERT_PATH}"
  environment:
    DOCKER_COMPOSE_SERVICE_NAME: echo
    DOCKER_HOST: "${DOCKER_HOST}"
    DOCKER_TLS_VERIFY: "${DOCKER_TLS_VERIFY}"
    DOCKER_CERT_PATH: "${DOCKER_CERT_PATH}"
# docker-compose up -d
Starting dockercomposelbexample_proxy_1
Starting dockercomposelbexample_echo_1

# docker-compose scale echo=10
Creating and starting 2 ... done
Creating and starting 3 ... done
Creating and starting 4 ... done
Creating and starting 5 ... done
Creating and starting 6 ... done
Creating and starting 7 ... done
Creating and starting 8 ... done
Creating and starting 9 ... done
Creating and starting 10 ... done

Now verify that it’s working by checking the X-Internal-Service header:

 # for i in {0..5}; do curl 192.168.99.100 -v 2>&1 | grep X-Int; done
< X-Internal-Service: 172.17.0.15:9090
< X-Internal-Service: 172.17.0.8:9090
< X-Internal-Service: 172.17.0.7:9090
< X-Internal-Service: 172.17.0.8:9090
< X-Internal-Service: 172.17.0.10:9090
< X-Internal-Service: 172.17.0.13:9090

As you can see the requests are handled by different containers! Quick and easy.
The final docker image is ~10mb big:

# docker images | grep docker-compose-reverse-proxy
nicolai86/docker-compose-reverse-proxy   latest              dc50f87279f3        8 hours ago         9.832 MB

While I propably wouldn’t use this for a high traffic service this setup works reasonable well since its initial deployment, and since it’s < 100LOC it’s easy to replace and extend.

However, I’d still recommend using Kubernetes or Docker-Swarm in multi-host setups.

That’s it for now. Happy Hacking!


Prototyping with XCode Playgrounds

With the introduction of the Swift programming language Apple also introduced Playgrounds, interactive environments for easy sharing & prototyping.

Today I want to share a tiny example on how to use them to prototype custom UIViews. Specifically a PolarClock. You can download the complete Playground here.

I love playgrounds because they allow you to quickly sketch ideas, and visualize steps, before heading for a fully fledged implementation. Let’s try it out:

First, open a recent XCode (I’m using 7.2) and create a new Playground.

We’ll start by constructing a circle using polar coordinates:

let radius = 50.0
var points = [CGPoint]()
for degree in 0.stride(to: 360.0, by: 10) {
  let theta = (90 + degree) * M_PI / 180.0
  let x = radius * cos(theta)
  let y = radius * sin(theta)
  let p = CGPoint(x: -x, y: y)
  points.append(p)
}

While you can not visualize [CGPoint] with Playgrounds, you can visualize a UIBezierPath:

let bezierPath = UIBezierPath()
bezierPath.moveToPoint(points[0])
for var i = 1; i < points.endIndex; i++ {
  bezierPath.addLineToPoint(points[i])
}
bezierPath

When you click the visualization button you should see a circle:

UIBezierPath visualized

Let’s wrap this in a tiny UIView, called Polar:

import UIKit

public class Polar: UIView {
  public var radius: CGFloat = 20
  public var completeness: CGFloat = 0.1 {
    didSet {
      self.setNeedsDisplay()
    }
  }
  public var strokeWidth: CGFloat = 10
  public var stepSize: CGFloat = 0.25
  public var color: UIColor = UIColor.redColor()

  func path(ps: [CGPoint]) -> UIBezierPath {
    let bezierPath = UIBezierPath()
    bezierPath.lineJoinStyle = .Round
    bezierPath.fillWithBlendMode(.Normal, alpha: 0.0)
    bezierPath.moveToPoint(ps[0])
    for var i = 1; i < ps.endIndex; i++ {
      bezierPath.addLineToPoint(ps[i])
    }
    return bezierPath
  }

  func circle(radius: Double) -> [CGPoint] {
    var points = [CGPoint]()
    let center = CGPoint(x: self.frame.width/2, y: self.frame.height/2)

    for degree in 0.0.stride(to: 360.0*Double(self.completeness), by: Double(self.stepSize)) {
      let theta = (270 + degree) * M_PI / 180.0
      let x = radius * cos(theta)
      let y = radius * sin(theta)
      let p = CGPoint(x: Double(center.x)+x, y: Double(center.y)+y)
      points.append(p)
    }

    return points
  }

  public override func drawRect(rect: CGRect) {
    let ctx = UIGraphicsGetCurrentContext()
    CGContextSetLineWidth(ctx, self.strokeWidth)
    CGContextAddPath(ctx, path(circle(Double(self.radius))).CGPath)
    self.color.set()
    CGContextStrokePath(ctx)
  }
}

The Polar view just placed the above code in separate functions and added some properties, as well as a drawRect method to display the view.

The observing reader might spot the changed theta calculation; this is necessary because the coordinates system for UIViews is different than the preview in the playground. To support a clockwise rotation at the center we start at 270°.

Finally, let’s use the class to build a real PolarClock:

let frame = CGRect(x: 0, y: 0, width: 375, height: 667)
let view = UIView(frame: frame)
XCPShowView("PolarClock", view: view)

let colors = [
  UIColor(red: 23.0/255.0, green: 170.0/255.0, blue: 178.0/255.0, alpha: 1.0),
  UIColor(red: 255.0/255.0, green: 32.0/255.0, blue: 172.0/255.0, alpha: 1.0),
  UIColor(red: 7.0/255.0, green: 241.0/255.0, blue: 255.0/255.0, alpha: 1.0)
]

let clockFrame = CGRect(x: 100, y: 100, width: 100, height: 100)
let clock = (
  Polar(frame: clockFrame),
  Polar(frame: clockFrame),
  Polar(frame: clockFrame)
)
clock.0.backgroundColor = UIColor(red: 0, green: 0, blue: 0, alpha: 0)
clock.0.color = colors[0]
clock.0.radius = 45.0
view.addSubview(clock.0)

clock.1.backgroundColor = UIColor(red: 0, green: 0, blue: 0, alpha: 0)
clock.1.color = colors[1]
clock.1.radius = 35.0
view.addSubview(clock.1)

clock.2.backgroundColor = UIColor(red: 0, green: 0, blue: 0, alpha: 0)
clock.2.color = colors[2]
clock.2.radius = 25.0
view.addSubview(clock.2)

let timer = dispatch_source_create(DISPATCH_SOURCE_TYPE_TIMER, 0, 0, dispatch_get_main_queue())
dispatch_source_set_timer(timer, DISPATCH_TIME_NOW, 1 * NSEC_PER_SEC, 0)
dispatch_source_set_event_handler(timer) {
  let now = NSDate()
  let calendar = NSCalendar.currentCalendar()
  let components = calendar.components([NSCalendarUnit.Hour, NSCalendarUnit.Minute, NSCalendarUnit.Second], fromDate: now)
  let hour = components.hour
  let minutes = components.minute
  let seconds = components.second

  clock.0.completeness = CGFloat(hour) / 24.0
  clock.1.completeness = CGFloat(minutes) / 60.0
  clock.2.completeness = CGFloat(seconds) / 60.0
}
dispatch_resume(timer)

You’ll see something like this, before the playground terminates after some seconds:

UIBezierPath visualized

That’s it! Now it’s just a tiny step to implement the @IBDesignable and use the class inside interface builder:

@IBDesignable
public class Polar: UIView {
  @IBInspectable public var radius: CGFloat = 20
  @IBInspectable public var completeness: CGFloat = 0.1
  @IBInspectable public var strokeWidth: CGFloat = 10
  @IBInspectable public var stepSize: CGFloat = 0.25
  @IBInspectable public var color: UIColor = UIColor.redColor()
  // … omitted
}

Now you can also use your Polar view inside interface builder:

UIBezierPath visualized

I love that you can easily prototype complete views with Playgrounds. The only catch is that only Swift is supported right now, so hopefully you can live without a debugger.

All in all the XCode playgrounds are awesome to share executable code, teach and learn. You should use them probably, too.

Anyway, that’s it for today. Happy hacking!