Watch App Development Blog - Week 1

January 09, 2015 - app apple-watch haskell

Okay, I’m trying something new - a weekly blog post talking about my progress developing an app. The theory is that the thought of my massive readership expectantly waiting for the next update will give me enough of an incentive to get something finished. Everyone practise their sad face to make me feel guilty if I don’t post an update.

I’m keen to get out an Apple Watch app. To be honest, I don’t think they’ll make much money, but most apps I build are for other people; it would be nice to put out something good, so I can say “I did that!”

The Concept

Over the last few months, I’ve tried to be aware of instances where I need some information off my phone, but pulling it out & launching an app seems like too much of a hassle. One scenario I noticed was when I was heading to the train station - I used to know the departure times off by heart, but now I’m not sure whether to run or dawdle. It would be great if there was an app on my watch that gave me live departure times for my nearest station - challenge accepted!

Step 1: The API

Transperth don’t have a public API, although there’s an unofficial third-party one that scrapes the website. Unfortunately some parts were broken with a recent site update, and the developer now lives in Melbourne. I also have a few ideas for some custom API behaviour, so I made a probably ill-advised decision to build my own scraping API.

I really wanted to try out a web project in F#, but I didn’t want to develop on Windows and I ended up running into significant problems with Xamarin - broken project templates, unimplemented parts of the aspnetwebstack, etc. ASP.NET vNext looks promising, but I had issues with it also.

So I thought I’d give Haskell another go - I’ve tried this in the past, but I’m much gooder at Haskell now. The state of web frameworks in Haskell has also improved significantly since 2010. I went with Scotty - I like the simplicity of the Sinatra/NancyFx model, and there’s a great walk-through by Aditya Bhargava.

Haskell Web Development on OS X

If you’re playing along at home, you'll need to follow the following steps to run the API:

  1. Install the Haskell Platform
  2. Run cabal sandbox init in your project directory. Cabal sandbox installs dependencies in a project scope, similar to Bundler in Ruby.
  3. Create a cabal file specifying your dependencies. This process I found a little odd - effectively you’re specifying your executable as a library, but it allows you to leverage cabal dependency resolution. Use Adit’s cabal file as a base.
  4. Create a Main.hs and add your Scotty routes (check out the examples).
  5. Run cabal install && .cabal-sandbox/bin/<executable name>

After an extended compile time, you should now have a web server running on localhost:<port>.

JSON Response Types

Returning JSON can be done by defining record types that implement the ToJSON type class (from Aeson):

{-# LANGUAGE DeriveGeneric #-}
module Types where

import Data.Aeson
import GHC.Generics

data Departure = Departure { time :: String, destination :: String, pattern :: String, status :: String } deriving (Generic, Show)
instance ToJSON Station

HTML Parsing

Parsing the DNN-generated web page is done using tagsoup. This differs from most other HTML parsing libraries I’ve used in that it doesn't define a query API or CSS-like selector syntax over a DOM, it just converts the HTML into a flat list of nodes that can be manipulated using regular list functions.

My scraping function, which is probably not brilliant Haskell, looks like the following (excluding some helpers):

getTrainTimes :: String -> IO [Departure]
getTrainTimes x = do tags <- fmap parseTags $  openURL $ "http://www.transperth.wa.gov.au/Timetables/Live-Train-Times?stationname=" ++ (urlEncode x)
                     let table = head $ tables tags -- first table in page
                     let rowArray = reverse . tail . reverse . tail $ rows table -- strip first & last rows
                     let times = map (f . cells) rowArray -- convert each row into a Departure
                     return times
                  where f cs = Departure (textFromCell $ cellAtColumn 0 cs) (destFromCell $ cellAtColumn 1 cs) (patternFromCell $ cellAtColumn 2 cs) (textFromCell $ cellAtColumn 3 cs)

‘Stations Near Me’

In addition to querying for live times, I also have a flat text file of station names & locations I lifted from Darcy’s project. This is used to respond to the ‘all stations’ API call, and also supports a geospatial query endpoint using the gps package. Initially I started getting build failures with this dependency - it was trying to compile GPX file support, which I don’t need. The latest version (1.2) of the gps code has removed this dependency, but it's not on Hackage yet.

Happily, this is solvable:

  1. Specify the specific version of the package in your cabal file: gps >=1.2
  2. Put the source code in your project directory (e.g. git submodule add git@github.com:TomMD/gps.git)
  3. Specify the new source directory with cabal sandbox add-source gps

Using the Geo.Computations model, it's then fairly straightforward to filter the list of stations based on distance to a given point.

Bringing it Together

Once the stations, live times, and geospatial filtering was done, it was just a case of defining the appropriate route functions in Scotty:

  scotty port $ do
    get "/train/" $ do
      list <- liftIO stations
      json list

    get "/train/near" $ do
      y <- param "lat"
      x <-param "long"
      list <- liftIO $ stationsNear y x
      json list

    get "/train/:station" $ do
      stationId <- param "station"
      station <- liftIO $ station stationId
      case station of
        Just s -> do
          times <- liftIO $ getTrainTimes $ name s
          json times
        Nothing ->
          Web.Scotty.status status404

I haven’t touched cache control or more advanced error handling, and it would be nice to fall back on timetables if live times aren't available, but I now have enough of an API running to support the basic functions of the watch app. One of the things I liked about doing it in Haskell was that once it compiled, it generally worked. It’s a pretty nice feeling.

I’ve put the code up on bitbucket - feel free to have a look through it and send some feedback if you can’t stand my beginner Haskell.

Next Steps

Next is hosting - building and deploying my dinky API somewhere I can reach it. Tune in next week for another thrilling instalment!