How to Develop an F# .NET Web App on a Mac using Atom + Ionide + Mono

I've been developing .NET apps for 8 years and for most of my career the thought of developing a .NET app outside of visual studio was crazy. Not all developers like heavy handed IDEs like visual studio, and to make matters worse as a .NET developer you are stuck on windows (I actually like Windows OS though). But, all those things are changing as Microsoft seems to be embracing open source (hello coreCLR!) and are winning back the developer community. It's now possible to write professional .NET apps on a Mac that can run on Mac OS, Linux, and Windows.

I'll walk you through a sample project I completed on my Mac at home. The app is a simple web service called "Sports Stats" (here is the full source code) and it's a web service that retrieves stats for a specific baseball player or golfer.

The Setup

Step 1 - Install Atom. It's also possible to use Visual Studio Code, but I chose Atom for this project. Atom is a text editor created by GitHub.

Step 2 - Install Mono. Mono is an open source implementation of .NET that allows you to run .NET applications cross-platform. Eventually you will be able to use coreCLR for this purpose but it's not quite ready yet.

Step 3 - Next comes Ionide. Ionide is an awesome open source package that allows F# development in VSCode and Atom. Install these Ionide packages in Atom.

Step 4 - Install fsharp yeoman. Yeoman will create scaffolding projects and solutions so you don't have to manually create the visual studio project xml files.

Step 5 - Now that everything is installed we start by using yeoman to create the solution. Just start by typing "yo fsharp" in the command line. In this project I created a simple web service so I used the Console application template.

The Dependencies

1. FAKE - F# library for building.
2. Fsharp.Data - F# HTML type provider used for parsing stat web pages.
3. FsUnit - Unit testing library to make f# tests read well. So instead of using Assert.Equals tests could look like this:

result |> should equal 10

4. Newtonsoft.Json - Library for serializing JSON
5. Suave - Amazing F# library that allows setting up a fast, lightweight, and non blocking web server.
6. xUnit - Unit testing library that works better than nUnit with F# projects.
7. Paket - Dependency manager for .NET (much better than NuGet).

The Code

I used FAKE to build all the projects. The build script uses Paket to download all dependencies, then builds the projects, then runs tests. Here is the full build.fsx file for reference.

One of the reasons I love writing code in F# is the ability to easily use the REPL. After programming this way there is no going back for me. The F# REPL in Atom isn't perfect yet but it really helps development. This allows for quickly testing of small functions and promotes the use of pure functions in your program.

REPL example

I used Suave for my web server and there are a lot of things I like about this library. It makes writing asynchronous code very easy, and it provides full flexibility without requiring you to write a ton of code. Here is the entry point of my program which uses Suave. It's very simple to understand. It forwards the HTTP routes specified to the appropriate function. This is much nicer than the WebApi controller classes that were necessary when using Microsoft.Owin.

Entry point:

let routes (db:IDB) =
  choose
    [ GET >=>
      choose [ path "/Golf/LowestTournament" >=> SportService.getLowestTournament db
               path "/Golf/LowestRound" >=> SportService.getLowestRound db
               path "/Golf/TotalEarnings" >=> SportService.getTotalGolfEarnings db
               path "/Baseball/Homeruns" >=> SportService.getHomeruns db
               path "/Baseball/Strikeouts" >=> SportService.getStrikeouts db
               path "/Baseball/Steals" >=> SportService.getSteals db ]]

[<EntryPoint>]
let main argv =
    startWebServer defaultConfig (routes Database.DB)
    0

The other good thing about the routes function is that it's fully unit-testable. The database connection is passed in at runtime so it's possible to test HTTP request and responses by simply testing the routes function.

Here is an example of a unit test that does just that.

let fakeDB (response:Response) =
  { new IDB with
      member x.GetLowestTournament first last = response
      member x.GetLowestRound first last = response
      member x.GetTotalGolfEarnings first last = response
      member x.GetHomeruns first last = response
      member x.GetStrikeouts first last = response
      member x.GetSteals first last = response
  }

[<Fact>]
let ``Golf lowest tournament total Tiger Woods``() =
  let expectedResponse = "{\"FirstName\":\"Tiger\",\"LastName\":\"Woods\",\"Stat\":{\"Case\":\"LowestTournament\",\"Fields\":[-27]}}"
  let athlete = defaultAthlete "Tiger" "Woods" (LowestTournament -27)

  result "Tiger" "Woods" "Golf\LowestTournament" (fakeDB athlete)
  |> validateSuccess expectedResponse

This unit test creates a fake database on the fly and passes that database into the routes function. The HTTP response is then fully validated. This unit test provides a lot of value and actually helped me quite a few times in development when I broke some of the routes by accident.

Eventually after the route is matched and its corresponding function is called the Fsharp.Data HTML type provider is used. The type provider loads the specified HTML page and parses through it appropriately. The parsing code I wrote is a little dirty because the page I used for getting the stats is created dynamically and didn't have good class names. Here is the parsing code for the golf stats.

let stat (html:HtmlDocument) (input:GolfInput) =
  let tables = html.Descendants ["table"]

  match Seq.length tables with
  | 0 -> Failure RecordNotFound
  | _ -> let value =
           tables
           |> Seq.head
           |> (fun x -> x.Descendants ["tbody"])
           |> Seq.head
           |> (fun x -> x.Descendants ["tr"])
           |> Seq.map (input.MapFunction input.Data.ColumnIndex)
           |> Seq.filter input.FilterFunction
           |> input.TotalFunction

         Success { FirstName = input.Data.FirstName
                   LastName = input.Data.LastName
                   Stat = input.Data.ValueFunction value }

This is also fully unit-testable. I simply pass in a sample HTML page and verify the result like so.

[<Literal>]
let golfHtml =
  """<html>
         <body>
             <table>
                 <tbody>
                     <tr>
                        <td>login</td>
                        <td>Win</td> <!-- Final finish -->
                        <td>61-67-70-71=269</td> <!-- Final score -->
                        <td>-27</td> <!-- Final score to par -->
                        <td>$864,000</td> <!-- Final money -->
                        <td>fedex</td>
                    </tr>
                    <tr>
                        <td>login</td>
                        <td>T15</td> <!-- Final finish -->
                        <td>66-71-70-71=278</td> <!-- Final score -->
                        <td>-28</td> <!-- Final score to par -->
                        <td>$1,997,000</td> <!-- Final money -->
                        <td>fedex</td>
                    </tr>
                    <tr>
                        <td>login</td>
                        <td>Win</td> <!-- Final finish -->
                        <td>72-71-70-71=284</td> <!-- Final score -->
                        <td>-18</td> <!-- Final score to par -->
                        <td>$322,000</td> <!-- Final money -->
                        <td>fedex</td>
                   </tr>
                   <tr>
                        <td>login</td>
                        <td>T33</td> <!-- Final finish -->
                        <td>58-77-64-60=259</td> <!-- Final score -->
                        <td>-17</td> <!-- Final score to par -->
                        <td>$659,000</td> <!-- Final money -->
                        <td>fedex</td>
                   </tr>
               </tbody>
          </table>
      </body>
  </html>"""

[<Fact>]
let ``Golf lowest round``() =
  let input = { FirstName = "Tiger"; LastName = "Woods"; ColumnIndex = 2; ValueFunction = LowestRound }
  let golfInput = { Data = input; MapFunction = GolfStats.lowestRoundMap; FilterFunction = (fun x -> x > 50); TotalFunction = Seq.min }
  let expected = Success { FirstName = "Tiger"; LastName = "Woods"; Stat = LowestRound 58}
  let doc = HtmlDocument.Parse golfHtml

  (GolfStats.stat doc golfInput)
  |> should equal expected

Here is the end result. A beautiful front-end showcasing my work!

Simple front-end using the API

Results: 

The bad - I couldn't get the FSI REPL to work with the FSharp.Data type provider. This was a shame because (as far as I know) debugging is not enabled in Atom with Ionide. Because of the limitation it made it difficult to write some of the HTML parsing code. Also, adding new files to the project was painful because manually editing the .fsproj files was error prone.

The good - Love the fact I can create F# .NET apps on a Mac without running a windows VM. Atom and Ionide work well together and this app was created with all open source packages and software. Given this process would also run on linux it is possible to create first class, scalable web services that would be inexpensive to host. It's close to becoming a viable option for a startup in my opinion.

What is Box in F#?

The first time I had to use the box function in F# there was some confusion on our team what exactly was happening under the hood. Our situation came when we had set an HTTP parameter to required in our WebAPI controller.

The Problem - The variable was in Int, it was required, and if it was not provided our application was supposed to return an error code. When the parameter was not provided WebAPI would set the required parameter to null even though Int cannot be null. This was very confusing, and to make matters worse we were not able to make the required parameter a Nullable<Int> with WebAPI.

The Solution - Box is the solution! From our searching, by boxing the int we could check if it is null even though it's not nullable (confusing). But what does the box function do? According to MSDN, the box function in F# "boxes a strongly typed value". Ok... Thanks documentation.

That's not all very helpful, but with further digging in this MSDN article the answer is "Boxing is the process of converting a value type to the type object..." It goes on to say "it wraps the value inside a System.Object and stores it on the managed heap..."

So, boxing a value wraps the value type inside a System.Object and therefore we can check if the object is null. Then we have to unbox it back to an Int after the null check.

Now the next time you box you'll hopefully have a better idea of what's happening. Happy boxing everyone!

From Development to Production

Each development team has their own way of developing software. I wouldn't advocate for only one development method, but here is a method we have used in the past that worked well.

The steps:

1. Have a list of well thought out and priority ordered tasks ready to go. Having a well groomed backlog is the best way to identify risks early in the process and is also a team moral booster. Estimate items at a granular level at a scale like small, medium, or large and only allow items less than large to be marked as ready.

2. It's also important to note QA should be involved and a necessary step to calling an item ready is everyone in the development cycle knows what work needs to be completed to call the task finished.

3. The developer picks a task off the top of the list and marks it as in progress, then starts work in a development branch off of the team integration branch.

*Branching aside: All our development was done in individual development branches and testing done in an integration branch.

4. Once development is complete the task is marked as resolved and a pull request is created into the integration branch.

5. Another team member will grab the task and mark it as in progress code review. Then the code reviewer proceeds to code review the pull request and when finished merges the pull request into integration and marks the task as dev complete.

*One additional note here is that in order to get to this stage a build would have to be completed on TeamCity, which includes all unit tests runs. If any of the tests failed the build would fail.

6. QA is then notified of a completed task, so the QA member will take the task and create a Git tag (release) off the integration branch. This is done so development can keep on churning without additional commits being added to the integration branch that would require re-testing.

7. QA writes automation and tests the task and when complete marks the task as QA complete and assigns the task back to dev.

8. Dev then merges the tag into master and creates a PROD release with a proper version number (we  did our best semantic versioning attempt).

9. The PROD release is then built in TeamCity (along with unit test runs). We used Octopus release promotion so this release was required to be deployed to a QA environment where all automation was automatically run again (any of these failing would cause release to halt). Once completed and successful then the release would be deployed to production with a click of the button.

This process isn't completely novel, but it worked really well for our team. The particular pieces I liked were our use of code reviews and git releases. Code reviews helped with defects and team cohesion, whereas git releases made it really easy to separate and document our work.

How to Watch ESPN While Traveling Internationally

I'm currently traveling internationally for a few months and one of my biggest dilemmas was how I was going to keep up with my sports teams. ESPN and other content providers will often only serve content within the U.S. To get around this we have a few options.

Option 1 - Use a DNS proxy
This may be the simplest option and by using Smart DNS Proxy setup is a breeze. The proxy server needs to be located in an approved content region for your provider (in my case the U.S.) The DNS server obscures your client IP address by routing certain traffic through other proxy servers. This is a simple solution but not as fast or secure as using a VPN as in option 2.

Option 2 - Set up a VPN
For the best solution, set up a VPN using Hide My Ass. This is the fastest, most secure, and reliable option. Hide My Ass sets up a private network and routes your traffic through their proxy servers, which masks your location.

What all of this means is it was possible to watch my beloved Kentucky Wildcats while on the beach in Fiji.