First Real Application Post-Mortem

Published: 2014-02-17

Tagged: learning

All things come to an end. So did this one. Back in December I decided to create my first "real" web application, built pretty much from the ground up. What's the ground in this case? The authorization/authentication system. It was also the most fun.

It's aim was simple - serving media, mainly movies, from a local server to my browser. Being able to stream media without transcoding it was a top priority, so I found the VLC web plugin for Firefox and I was happy. I knew VLC would handle almost any format I would throw at it. My application would be a simple layer over a database with users and media. It would allow a user to upload their file and later retrieve from a quickly-searchable database. Simple.

Some quick localhost tests confirmed the viability of the VLC web plugin for this task. Then I set out to build the, fairly RESTful I might add, layer over the database. This was a fun and easy introduction to Golang's net/http package as well as Gorilla's tmux. I experimented with breaking the code apart into sub-packages, but I got stuck after separating out models. Basically, the application was a main server-controller file, a file sketching out the models and some useful functions, and a folder filled with templates.

Uphill going

The first real obstacle that presented itself was the authorization and authentication system. This wasn't a necessary feature, but I thought it would be fun and educational and damn me if it wasn't. Keep in mind the fact that I come from Rails, a world where most things require adding a line to thing called a Gemfile and later executing a simple bundle install. Devise, the most commonly used authentication/authorization system for Rails as far as I know, requires those two actions, plus perhaps 10 minutes of configuration. Authentication was simple to implement, all I needed was a way to store and generate passwords. Cryptography I, a well known coursera course, added a lot of understanding to what I was doing. The next step sounded harder to implement - authorization, but some quick googling led me back to Gorilla and their awesome sessions package. It probably saved me a day or two of coding a way to turn a stateless protocol into a stateful one - or in other words - to know whether a user is logged and which user is logged in.

Around this time I started going back and forth between using AngularJS and not using it. The structure I had at this point was too set in place with all the templates so making a single-page app would require a lot of changes. However, I thought I could sprinkle some Angular magic here and there, but then I reconsidered, as I would have to expand some controller functions to handle jsonifying the data. I went through this process two times, probably using a good two days of work. Finally, I decided against Angular and used a tiny bit of jquery to achieve a similar effect.

That was a big push, but then I had to make my controller function (the ones that take care of http.ResponseWriter and http.Request) aware of the *context. There's also a Gorilla package for that, but I chose to quickly make something myself. I thought that I could simply pass a context variable, an interface{} map, to each handler and have the logic use that. It would also be available in views to determine some user-role specific issues such as showing the admin navbar or even toggling the log in/log out link.

This happened to use something I've seen in a Golang tutorial, which currently escapes me, but basically it introduced the idea of wrapper function for the controller functions. At the start, all these function did was catch panics and log them, but at this point I added a function that pulled the user id from a session, found that user and added it to the context. At that point I saw the beauty in how much a simple wrapper saved me time and how much design and architecture matter in building an application.

If I had simply thrown code at the problem, I would have gotten a solution... and a rewrite a week later. However, naively as it were, using that wrapper in the beginning saved my a helluva load of work later down the road.

Funny enough, despite Golang's different approach to OOP (lack of classes, methods and interfaces everywhere), it also taugh me a good deal about OOP. It was hard at first, I admit, but later it turned out to be an invisible guide to structuring the application. I know it's far from clean code, but I think there's an overall improvement in my craft from having a go at Go.

Failure

I spent about 3-5 days getting my code to work on my RasPi. An issue with the sqlite db adapter. I tried my hand at solving it, but everything failed me, even my leet google skills. I decided that while this would increase the overhead, I'll fully use the blessing of using an ORM, and simply switch out the database to Postgres. Unfortunately, even though it worked fine on my laptop, I was not able to get Postgres running on my Pi. I think it's an issue with the stripped down Raspbian that I'm running, but I didn't want to use any more time and reached for MySQL. Finally, a breakthrough!

I thought I was home. I uploaded a 20 minutes mp4 file and loaded it up in the application. It played right away. How wrong I was - it's the details the come back to bite us, right?

A few days later I uploaded a good 700mb mkv file, gleefully navigated my app to the right URL and... the plugin crashed. I checked the file first, thinking that maybe it became corrupted during the upload. Nope. I tried downloading the file, which showed me that something funny was happening: the file would download for a few seconds, then stall, download some more, then stall again. After looking at the code, I think that the http.ServeFile function is the most likely culprit. Or maybe I got the headers wrong? Whatever it may be, I decided to put it back on the shelf for a while.

Tactical withdrawal

The pain point the application was aimed to solve was not as big as the demand on my time to solve it, even when I factored in everything that it taught me. I already had a project in mind, in which I could apply the lessons learned and perhaps hammer them in further. So here ends the story of GopherPi, at least for now.

A very cool by-product of this adventure is that the 1-2 weeks spent on researching and implementing that, I was able to pick up how to set up authentication and authorization in a Flask application in a matter of minutes. Flask has a somewhat similar extension called Flask-Login, which performs a job like Gorilla sessions. Having the safe storage of a user credentials on a browser, all I had to do was add a password checking routine and a hashing routine using py-bcrypt. Without having gone through the adventure with golang, this would have taken me ages longer.

The most important thing this taught me was the importance of two things - being able to quickly test an idea and user experience. UX actually will shape the way code works, so from now I'll apply a design-first approach to applications. The method I'm going with now is to set expectation and make simple wireframes for an application before I set down any code. I'm also checking out some resources on general design to get a better feel of how folks interact with things and how to make it easier for them.

As for testing out ideas - if I knew that the VLC web plugin crashes for files served with Golang's file-serving function, I wouldn't have sunk so much time working on something that doesn't work in the end. This shows the importance of prototyping things that I haven't worked with earlier. It would have been dreadfully simply to test this out, probably under 30 minutes, and would have saved me a good 20-30 hours of development time. But then, had I not made this mistake, would I have learned this valuable lesson?