Setup HAProxy with Nginx on OS X

This is a follow up for one of my earlier post about Replacing Pow.

I’ve been working a lot with Faye and PrivatePub lately, which cause me to stumble across some problems with the setup of mine. Primarily, moving from development to production environment.

The main problem is that WebSocket support will come to Nginx as of 1.3.x. - which is many months away.

A common solution is to use HAProxy in front of Nginx. HAProxy can handle WebSocket upgrades and messages properly. Regular HTTP requests are proxied to Nginx.

The new configuration of my development environment looks like this:

• Nginx is listening on port 8080 and running as a user service. Prior Nginx was listening on port 80 and running as a system service.
• HAProxy is listening on port 80 and running as a system service

Faye is started on a per project basis.

Installation instructions

$ brew install haproxy
$ mkdir -p $(brew --prefix)/etc/haproxy
$ cat >$(brew --prefix)/etc/haproxy/haproxy.cfg <<EOL
  global
    log 127.0.0.1   local0
    log 127.0.0.1   local1 debug
    #log loghost    local0 info
    maxconn 4096
    #chroot /usr/share/haproxy
    #daemon
    #debug
    #quiet

  defaults
    log     global
    mode    http
    option  httplog
    option  dontlognull
    retries 3
    option redispatch
    maxconn 2000
    contimeout      5000
    clitimeout      50000
    srvtimeout      50000

  frontend all 0.0.0.0:80
    timeout client 86400000
    default_backend nginx_backend

    acl is_websocket hdr(Upgrade) -i WebSocket
    acl is_websocket hdr_beg(Host) -i ws
    acl is_faye url_sub faye

    use_backend ws_backend if is_websocket
    use_backend faye_backend if is_faye

  backend ws_backend
    option forwardfor
    option http-server-close
    option http-pretend-keepalive
    timeout queue 5000
    timeout connect 86400000
    timeout server 86400000
    server server1 127.0.0.1:9292 maxconn 2000 check

  backend faye_backend
    option forwardfor
    option http-server-close
    timeout connect 4000
    timeout server 30000
    server server1 127.0.0.1:9292 maxconn 2000 check

  backend nginx_backend
    option forwardfor
    option http-server-close
    timeout connect 1000
    timeout server 6000
    server server1 127.0.0.1:8080 maxconn 2000 check
EOL
$ sudo -i
$ cat >/Library/LaunchDaemons/org.homebrew.haproxy.plist <<EOL
<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>KeepAlive</key>
  <true/>
  <key>Label</key>
  <string>org.homebrew.haproxy</string>
  <key>NetworkState</key>
  <true/>
  <key>Program</key>
  <string>$(which haproxy)</string>
  <key>ProgramArguments</key>
  <array>
    <string>haproxy</string>
    <string>-f</string>
    <string>$(brew --prefix)/etc/haproxy/haproxy.cfg</string>
  </array>
  <key>StandardErrorPath</key>
  <string>/var/log/system.log</string>
</dict>
</plist>
EOL
$ launchctl load -w /Library/LaunchDaemons/org.homebrew.haproxy.plist
$ exit

The above configuration has a few assumptions:

• You are running on OS X
• You are using homebrew as a package manager
• You have Nginx running and configured to listen on port 8080

Getting the above to work can be quite troublesome if you had Nginx running as a system service listening on port 80. Console is your best friend in most debugging cases.

Gains

Now your development environment more closely resembles your production environment - You’ll encounter most production-related problems earlier. Which is a good thing!

Happy hacking!