Moving to Canada (2)

This is one out of three blog posts I’ve planned for my immigration to Canada. This post will primarily be about my dog Yumi and how he relocated to Canada with us.


Yumi was born September 8th 2015 - by the time we travelled he was close to his 2nd birthday.

We did plenty of research to make sure that his immigration went as smooth as possible. First, we had to organize a properly sized transport box for the flight, and secondly comply with the Canadian dog import regulations.

We bought the biggest box available at the local pet shop in Hamburg. The crate included a dog bowl for water and food, which is required. Also Yumi could stand in it comfortably, which is important.

When booking the flight we immediatly added a dog transportation to the ticket. For unknown reasons we had to book twice, as the online booking failed the first time so we called the airline multiple times to make sure the booking was correct.

Having booked the flight and bought a dog crate we set up two appointments with a veterinarian to have Yumi checked out: one some months ahead to refresh his rabies vaccination and one two weeks before the flight. The veterinarian provided a health certificate for Yumi on the latter date. So far so good.

We checked in our luggage the night before the flight. When we asked about the procedure to checkin Yumi the agent told us there was no dog on our booking. Staying calm we discussed with the agent for a while. I had printed proof that Yumi was booked, and after some searching he was able to find his booking. It seemed the booking was only visible on my ticket, not the one of my wife; the agent had only looked at her booking aparently. Luckily we asked about our dog, had printed proof of his booking and were persistent about the issue.

Problems did not stop there, sadly. The day of the flight we went to checkin Yumi and the agent told us the crate was too small. Yumi only had about 5 cm headroom when standing inside the box, and regulations require 10 cm. Interestingly, the inspector present at the checkin counter said he’d be fine, but ultimatly the decision was up to the travel agent.

This gave us about 60 minutes to find a bigger crate, or Yumi had to stay in Frankfurt.

Together with Yumi I rushed off to buy a bigger crate - inside the backage claim area there was a crate shop, and about 30 minutes later I had bought the biggest crate available and moved it back to the checkin counter. We assembled the crate in a hurry and checked him in, and then moved on to our own checkin.

After arriving in Vancouver we first went to the immigration area, and while waiting I walked over the the airline counter to inquire about the pick up procedure of Yumi. He was transported to the bulky cargo area, it took about an hour, but he was calm when I got him so I assume that his first flight was fine.

As Yumi is a healthy dog the immigration agent only checked his papers and we quickly moved on to get out of the airport.

Dispite all problems, with proper preparations and a bit of luck we made it to Vancouver, Canada. Needless to say that we would not have taken the flight without him :)

And now we’re all in Canada, enjoying the adventure together :)





Moving to Canada (1)

As mentioned earlier 2017 will be a hectic year - I’m looking forward to immigrating to Canada. I’ll be relocating to Vancouver, British Columbia, to start working at Amazon.

Living abroad and working at one of the big five tech companies (Amazon, Apple, Google, Microsoft, Facebook) have been two of my goals for a long time, and both will be reality end of August. I can’t articulate how excited I am - it’s a huge opportunity to growth for me, and it puts me well outside of my comfort zone. I’ll learn a lot, and hopefully get to know beautiful Canada.

Anyhow, I’m looking forward to sharing more pictures again, some new technical posts, and a great time with my family in Canada! 🇨🇦


Modifying binaries to replace proprietary APIs

Note This is a follow up post on exploring private apis from late May.

Soon I want to use the Things 3 macOS application with my own API. To achieve this goal I have built a working SDK for the things cloud to understand the structure of the communication between client and server. This time I want to modify my Things 3 binary so it actually talks to an API of my choice. Let’s get started.

We know Things 3 talks to cloud.culturedcode.com. So this must be encoded inside the binary somewhere. To find out where, let’s use strings:

Strings looks for ASCII strings in a binary file or standard input.

$ strings /Applications/Things3.app/Contents/MacOS/Things3  | grep "cloud\."

The empty output tells me it’s not part of the main binary, so it must be part of some dependency. Next, I need to find out which dependencies Things 3 has, which can be done using otool:

The otool command displays specified parts of object files or libraries.

We’re specifically interested in shared libraries which come with the binary:

$ otool -L /Applications/Things3.app/Contents/MacOS/Things3 | grep "@"
  @rpath/FoundationAdditions.framework/Versions/A/FoundationAdditions (compatibility version 0.0.0, current version 0.0.0)
  @rpath/CoreJSON.framework/Versions/A/CoreJSON (compatibility version 0.0.0, current version 0.0.0)
  @rpath/KissXML.framework/Versions/A/KissXML (compatibility version 0.0.0, current version 0.0.0)
  @rpath/Base.framework/Versions/A/Base (compatibility version 0.0.0, current version 0.0.0)
  @rpath/ThingsModel.framework/Versions/A/ThingsModel (compatibility version 0.0.0, current version 0.0.0)
  @rpath/SyncronyCocoa.framework/Versions/A/SyncronyCocoa (compatibility version 0.0.0, current version 0.0.0)
  @rpath/ThingsTools.framework/Versions/A/ThingsTools (compatibility version 0.0.0, current version 0.0.0)
  @rpath/QuartzAdditions.framework/Versions/A/QuartzAdditions (compatibility version 0.0.0, current version 0.0.0)
  @executable_path/../Frameworks/TXOnboardingPopUpKit.framework/Versions/A/TXOnboardingPopUpKit (compatibility version 0.0.0, current version 0.0.0)
  @executable_path/../Frameworks/TXVisualDebugKit.framework/Versions/A/TXVisualDebugKit (compatibility version 0.0.0, current version 0.0.0)
  @executable_path/../Frameworks/TXTrialIndicatorKit.framework/Versions/A/TXTrialIndicatorKit (compatibility version 0.0.0, current version 0.0.0)
  @executable_path/../Frameworks/TXPopUpMenuKit.framework/Versions/A/TXPopUpMenuKit (compatibility version 0.0.0, current version 0.0.0)
  @executable_path/../Frameworks/TXLinkDetectorKit.framework/Versions/A/TXLinkDetectorKit (compatibility version 0.0.0, current version 0.0.0)
  @executable_path/../Frameworks/TXListKit.framework/Versions/A/TXListKit (compatibility version 0.0.0, current version 0.0.0)
  @executable_path/../Frameworks/TXToolTipKit.framework/Versions/A/TXToolTipKit (compatibility version 0.0.0, current version 0.0.0)
  @executable_path/../Frameworks/TXCloudIndicatorKit.framework/Versions/A/TXCloudIndicatorKit (compatibility version 0.0.0, current version 0.0.0)
  @executable_path/../Frameworks/TXToolbarKit.framework/Versions/A/TXToolbarKit (compatibility version 0.0.0, current version 0.0.0)
  @executable_path/../Frameworks/TXTrialExpiredKit.framework/Versions/A/TXTrialExpiredKit (compatibility version 0.0.0, current version 0.0.0)
  @executable_path/../Frameworks/TXQuickEntryKit.framework/Versions/A/TXQuickEntryKit (compatibility version 0.0.0, current version 0.0.0)
  @executable_path/../Frameworks/TXDatePickerKit.framework/Versions/A/TXDatePickerKit (compatibility version 0.0.0, current version 0.0.0)
  @rpath/SMStateMachine.framework/Versions/A/SMStateMachine (compatibility version 0.0.0, current version 0.0.0)
  @executable_path/../Frameworks/TXWindowKit.framework/Versions/A/TXWindowKit (compatibility version 0.0.0, current version 0.0.0)
  @executable_path/../Frameworks/HockeySDK.framework/Versions/A/HockeySDK (compatibility version 1.0.0, current version 1.0.0)
  @executable_path/../Frameworks/TXMainWindowKit.framework/Versions/A/TXMainWindowKit (compatibility version 0.0.0, current version 0.0.0)
  @executable_path/../Frameworks/TXAppKit.framework/Versions/A/TXAppKit (compatibility version 0.0.0, current version 0.0.0)
  @executable_path/../Frameworks/TXTagKit.framework/Versions/A/TXTagKit (compatibility version 0.0.0, current version 0.0.0)
  @executable_path/../Frameworks/TXPopoverKit.framework/Versions/A/TXPopoverKit (compatibility version 0.0.0, current version 0.0.0)
  @executable_path/../Frameworks/TXCheckListKit.framework/Versions/A/TXCheckListKit (compatibility version 0.0.0, current version 0.0.0)

Lots of shared libraries, but we know we’re interested in functionality related to the cloud synchronization, so SyncronyCocoa looks like a likely candidate, as strings confirms:

strings /Applications/Things3.app/Contents/Frameworks/SyncronyCocoa.framework/SyncronyCocoa | grep "cloud\."
https://cloud.culturedcode.com/
https://development-dot-thingscloud.appspot.com/

Nice. Next, we need to modify the shared library to use a different domain which, conveniently for development, points to localhost:

cat /etc/hosts | grep cultt
127.0.0.1 cloud.culttcoder.local

Note that it’s important that the domain has the same number of characters as the one we’re trying to replace - if it’s shorter or longer the Things 3 binary will crash on launch.

Now that we have a domain pointing to our local machine we need to patch the SyncronyCocoa.framework. I’ll be using dd to modify the binary. dd in combination with strings is a great tool for making smaller modifications to binary files.

First, we need to find the offset of the string inside the library, using strings:

strings -t d /Applications/Things3.app/Contents/Frameworks/SyncronyCocoa.framework/SyncronyCocoa | grep "cloud\."
36078 https://cloud.culturedcode.com/
36110 https://development-dot-thingscloud.appspot.com/

This tells us that the string https://cloud.culturedcode.com/ is located at offset 36078. Now, we can use dd to change the library at position 36078 so it points to our local domain:

$ printf "https://cloud.culttcoder.local/\x00" > /tmp/api-dns
$ sudo dd if=/tmp/api-dns of=/Applications/Things3.app/Contents/Frameworks/SyncronyCocoa.framework/SyncronyCocoa obs=1 seek=36078 conv=notrunc

Let’s verify the patch was successful:

$ strings -t d /Applications/Things3.app/Contents/Frameworks/SyncronyCocoa.framework/SyncronyCocoa | grep "cloud\."
36078 https://cloud.culttcoder.local/
36110 https://development-dot-thingscloud.appspot.com/

Again, nice. Now we’ve successfully patched the library to talk to our local domain, but Things 3 will crash. As it’s an App Store binary everything is code signed, and our patch invalidated the code signature. Let’s fix that:

$ sudo codesign -f -s - /Applications/Things3.app/Contents/Frameworks/SyncronyCocoa.framework/SyncronyCocoa
/Applications/Things3.app/Contents/Frameworks/SyncronyCocoa.framework/SyncronyCocoa: replacing existing signature

Alright, Things 3 works again, but we don’t have a compatible API server yet. For now let’s create a local setup with a proxy which in turn forwards all requests to the real things cloud API:

For this to work we need a valid SSL certificate. A real deployment can easily obtain one using letsencrypt, but for local development a self signed certificate marked as trusted works just fine:

$ openssl genrsa -out server.key 2048
$ openssl req -new -x509 -sha256 -key server.key -out server.crt -days 3650 -subj '/CN=cloud.culttcoder.local/O=Private/C=DE'
$ sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain server.crt

Note that the certificate must be valid for the domain I chose before. Next up I’ve setup a tiny HTTPS proxy written in Go:

package main

import (
  "flag"
  "log"
  "net/http"
  "net/http/httputil"
)

func main() {
  listen := flag.String("listen", ":443", "port to listen on")
  flag.Parse()

  director := func(req *http.Request) {
    req.URL.Scheme = "https"
    req.Host = "cloud.culturedcode.com"
    req.URL.Host = "cloud.culturedcode.com"
    req.Header.Set("Connection", "close")

    dump, _ := httputil.DumpRequest(req, true)
    log.Printf("%s\n", string(dump))
  }
  proxy := &httputil.ReverseProxy{Director: director}
  log.Printf("Listening on %s\n", *listen)

  err := http.ListenAndServeTLS(*listen, "server.crt", "server.key", proxy)
  if err != nil {
    log.Fatal("ListenAndServe: ", err)
  }
}

This proxy will forward any request received to https://cloud.culturedcode.com. Now, when we start Things 3 it can talk to the things cloud, just as before, but through our local proxy.

things3 talking to my local thingscloud proy

Soon, it won’t be talking to the things cloud API at all…

That’s it for today, happy hacking!


Moving Forward

Everything changes and nothing stands still. Heraclitus

And 2017 will be quite a hectic year for me because of this. I’m looking forward to sharing more non-technical things in the next few months, too. Mostly pictures I guess, but we’ll see what I’ll find interesting enough to share (: