How did you get to Gilt?
In late 2011, I was working as a contractor at another eCommerce company, and Gilt was at the end-stage of opening its Dublin office. A Gilt recruiter contacted me, and a few months later–in May 2012–I started here.
What were you hired on to do?
At first I was hired on to be a performance engineer, then I almost immediately switched to infrastructure engineering.
Did you have a background in infrastructure engineering?
Not specifically. For much of my career I’ve worked at startups as a full-stack engineer. I’ve also founded a couple of startups, and worked in senior roles. In a startup, you end up learning a lot about how to write software and sell it, but you often don’t learn how to scale it in an organization. This is the first time in my career other than my last job where I’ve seen companies out of their startup phase.
When you got to Gilt, what did you do first?
Soon after I started we came up with a project, Galactica, and I began working on implementing parts of the idea. Gilt being Gilt, I kind of found work that I wanted to do, that wasn’t what I was originally hired to do.
The scale of Gilt is nice: it’s small enough that you can get your head around its infrastructure, but large enough to be impressive.
Tell us more about what Galactica is and does.
Galactica is a family of services that allocates containers via Docker, allows for provisioning of other services, and implements a meta-data model. It echoes the immutability of Scala, the language we use at Gilt. Immutability is important to us–it enables engineers to do things concurrently and safely, and enables you to avoid situations in which two things try to change your data at the same time.
When complete, Galactica will give us the confidence to deploy quickly and often, and if we do break something, we can recover quickly by switching back to the previous version. The previous will still be running, it just won’t be taking any traffic. For now we’re working on or staging, integration and pre-production environments, but will eventually tackle our full production environment in the same way.
What do you think of Docker?
Docker is lighter weight than any other comparable technology out there, it’s open source, and it’s much easier to experiment with. It’s not a system for providing computing resources as a service, per se–it really just does one thing, which is to start processes inside containers. Docker lets us to build our own services around it.
How long have you been working in Scala?
I started using it not long before I started working at Gilt. During the hiring process, I was asked to write an implementation of an interview question that I’d described during my initial interview–I wrote it in Scala. It wasn’t great Scala, but it was definitely a good thing to have done (it got me the job!).
How has Scala made you a happier engineer?
Well, before I made the switch I was working in Java and programming functionally in Guava. This was extremely frustrating: I was typing ten times as many characters as I felt was really necessary. Scala was a huge relief: it’s got a steep learning curve, but once you achieve a point of proficiency it’s great. For a learning tool I recommend Typesafe’s courses on Coursera–some of us took the courses, and it actually changed elements of our style because we saw (Scala creator Martin) Odersky doing certain things, and it gave us permission to do them as well (the content is also extremely good).
Can you name one of the ways Typesafe’s courses altered your style?
One little thing was using if and else more often. There’s a tendency in Scala to overuse some of its monadic features. Quite often, if you take a slight step backwards and take advantage of its functional programming features, you can produce code that is safe and functional, but also looks better.
You’ve actually taught some Scala–talk about that.
Last fall I helped teach a Scala session to some engineers in the Dublin community. I’ve also helped others on the team. I’ve always liked mentoring because it’s rewarding–it helps you to learn. When you’re teaching someone you find gaps in your own knowledge. Students ask the best questions! We’ve talked about creating an advanced-level Scala course–I’m going to help with the curriculum.
Did you study programming in school?
Yes, I have a CS degree. I started programming when I was eight!
Are you experimenting with any other languages at the moment?
One of the earliest things I did at Gilt was to write a Lisp interpreter. Lisp is something I come back to, because it’s extremely easy to write a Lisp interpreter; also, a fundamental understanding of it enables you to explore other languages quite well. Aside from that, I have checked out Go but haven’t gone deep into it. Clojure is definitely on the list of things I’d like to experiment with.
How active are you in the open source community?
I’m a contributor to Gatling, a tool for load testing. It generates requests to send to a web service or application at a very high volume. It also measures how long those requests take, and how your application performs under different levels of load. Last year I gave a talk at the Scala.io conference in Paris with one of Gatling’s founders–we discussed load testing and using Gatling at Gilt.
How are we using Gatling, exactly?
At Gilt, we’ve found that writing tests in code is much more effective in encouraging people to write tests, period. We don’t have a team of performance engineers writing tests, but a team that writes tooling. We want developers to write tests, and have it become second nature. It’s harder when testing means using a GUI–you have to learn something new to write the test. The way Gatling works was very appealing to us, and the reports looked fantastic. We took it on, built some tooling so we can run tests against EC2, and now test the site three times a day. We receive alarms if there’s a failure, so we’re now working on a tool called the Smoking Gun Detector–it will run after every load test, analyze behavior as the log tests saw it, analyze the behavior of the requests the log test made, and identify or suggest backend services that are indirectly accessed as a result of the load test. The intention is that it will automatically identify anomalies. If a load test fails or performs slowly, the Smoking Gun Detector will hopefully point out where the source of the problem is. Eventually we’d hope to have it running all the time, comparing the performance of our services and identifying irregularities.
How would you describe Gilt’s attitude toward open source?
Gilt is a great company for open source. Many companies restrict their engineers’ ability to contribute to projects or launch their own–even on their own time. Here, as long as you’re responsible for what you do, and you don’t do anything stupid, it’s acceptable.
It’s the same with other things about our culture–for example, contributing to the tech blog. It’s kind of rare that you have companies of our size that encourage everyone in tech to blog on the company’s site–usually, there are four or five people who blog, and they present a micromanaged view of how the company works.
Final question: What are your goals?
I have three types of goals. One is project-based: I’d like to complete Galactica. Another relates to evangelism, both internal and external: teaching Scala, writing blog posts, etc. The final goal is DevOps-oriented–to continue to drive reliability. Because of the time difference, the Dublin team is up and awake long before the people in NYC arrive at work. Usually if something problematic happens overnight, the Dublin team is working on it. I would like to help us achieve a pure devops culture, where everyone is in ops and has full privileges. I’m trying to encourage that mindset across the organization–i.e., if you have an interest in how systems work, make sure you get privileges and join in on keeping them working. The way we build now, it’s hard to severely damage something–but if you do, just ask for forgiveness.