If you’re a Business Intelligence Engineer, we’d like to hear about your favorite tools and methodologies. Email us at lapple at gilt dot com.

Hello, George! Tell us a bit about how you arrived at Gilt.

I got here in 2012 after working in business intelligence at Everyday Health, MTV/Viacom, and Hewlett-Packard, and before that I went to school at Cornell.

How have things changed since the early days?

When I first joined the team, we were still trying to figure things out. Our data warehouse was new, our self-service framework wasn’t built out yet, and our goals weren’t as well defined. Since then we’ve streamlined our processes and become a lot more efficient, within the team and out. We were doing a lot of repetitive work and now there’s much more time to work on projects that will make a difference and that we’re passionate about.

What drew you to Gilt in the first place?

Just walking into the office I felt a buzz and got the sense that there’s a healthy atmosphere here. Gilt has a great reputation, with a lot of smart people working here. Everyone I spoke to during the interview process, I had a good connection with. Technology-wise, we’re pretty cutting-edge and I was excited to work with the team and learn new skills with the latest tools.

What does an average day look like for you?

Every day is different, but typically I spend half the day doing analyses, then working on projects for different teams. For example, I’m working with our customer transactions team to get credit card data into our data warehouse, and our loyalty team on getting new AB testing data into the warehouse as well. The data team will help the engineers figure out how the data model will work and look. Projects vary in length and scope, but the ones I’m working on now have lasted a couple months.

The data team’s recent work has been involving a lot of open source technologies.

Yes. We’re doing much of our current work through new event streams, in which services relay messages to Kafka, and the data warehouse reads those messages. Eventually we’ll use Hadoop for the file storage, with Kafka relaying the messages there and the data warehouse reading from Hadoop. In our work with Kafka, the idea behind it—and we’re not there yet—but it is to create easy access to all of the data within the company, so no one will have to ask an engineer to create a process to access a file or send data to the data team; they can just publish events to Kafka.

In terms of technology skills, what should a business intelligence engineer possess?

Prior experience using a range of business intelligence tools, a solid background in SQL, and an understanding of how a data warehouse works. But the primary qualification for the job is a good mind for analytics—the ability to draw connections between different data sources, understand data models, see patterns, and consolidate them into insights decipherable by anyone.

At Gilt, a lot of our data isn’t exact, so working with it involves a lot of research. At a more granular level, that means working with various engineering teams to understand the data they’re producing and putting the data together from multiple sources (inventory, site-live inventory, email, click-stream, order transactions) to fit it into a model. Then we run the data through the model to understand its various attributes, and highlight the important attributes to support the various business decisions we make.

One of the things we pride ourselves on here is our flat structure. Do you get to work with tech leadership much?

I do work with the tech execs occasionally, but I work more frequently with the heads of other departments, like Marketing and Merchandise Planning. The data team works with every department in the company outside of tech: marketing, finance, operations, merch planning, etc. The projects we work on relate to everything from supply chain management, to new marketing initiatives, to understanding our customers and demographics, to AB testing and understanding more about the user experience. Across these initiatives, my role is to understand where all the data comes from, put it all together and make it easy to use for everyone else in the company who needs it. Within the data team, I work closely with our chief data scientist to run data models and see if certain assumptions about our data make sense or not.

Talk a bit about the tech stack our data team uses.

Our stack currently includes Aster Teradata, Hadoop, Kafka, AWS and AWS Kinesis, real-time data events streams, and lots of micro-services written in Scala. On the front-end we work with Cognos, Looker and Spotfire.

What do you enjoy most about working with this stack?

Our data warehouse is pretty amazing in terms of speed and processing power. The amount of data we can process, run analytics on and get responses back—and quickly—is pretty awesome. A lot of other places I’ve been to, especially startups, don’t have the infrastructure to do that. It’s great to be able to iterate on your analytics quickly—you don’t have to wait for the data to come back.

The MR functions that Aster provides, and the functions we can create on our own, make our analytics even faster. And having Scala programmers on our team, who can help write MRs so that we can do things Aster can’t do out of the box, is really powerful.

Which technology do you personally use the most?

Most of my work involves working with SQL, but also Cognos and Looker. We do a lot of data mining and looking at large sets of data to find patterns.

We’ve been a bit bullish about Looker since we started using it. What are its advantages?

Looker has empowered people across the business to do their own analysis. It’s generated a collective sense of ownership when it comes to our data and analytics. The more powerful self-service is, the more insightful our analytics can be, and the more the data team is freed up from mundane tasks involving pulling data for people. We’re free to discover new things and take on new projects, not just build reports for the rest of the company.

What are your favorite things about your job?

Interacting with all the different teams—it helps me to understand the business more. Designing new solutions and participating in the discussions on how to best approach an analysis, choose the data to work with. I also appreciate the diversity of projects. Usually the data team will focus on helping one department at a time for 2-3 months, but even then we’re still working with everyone.

What do you like most about the data team’s culture?

Everyone is pretty independent, yet we’re pretty close. We’re all there for each other, but we give each other a lot of leeway to build and develop the way we want to.  

And what about Gilt’s culture in general?

We’re quick in getting things done.  Everyone is really helpful.  We act on new ideas quickly and we’re not afraid to fail.

Earlier today the folks behind Gatling announced the release of Gatling 2.0.0-RC1—and also their new website, which mentions Gilt as a user! In case you’re not familiar, Gatling is an open source load-testing framework based on Scala, Akka and Netty. Congrats to the Gatling team for both achievements. 


August 8-9 marks the first-ever Scala By the Bay conference: two days of training, talks and reverie for the San Francisco-area Scala and functional programming community. Scala By the Bay is made possible by Alexy Khrabrov, conference co-organizer (along with Jason Swartz) and founder. Alexy’s also the founder and co-organizer (along with Swartz) of SF Scala, whose +2K members “use Scala to dominate the world.” Exciting! We chatted with Alexy to find out more about next week’s conference—try to go, if you can!

Compared to last year’s Silicon Valley Scala Symposium, which you also organized—and have rebranded as SbtB—what’s different and new about this year’s conference?

This year everything is different. We renamed the conference Scala By the Bay to attract folks from all over the world to our beautiful area. We’ve rented Fort Mason for the location and are holding the conference over two days instead of one. We ran a proper call for proposals, and selected a very competitive program. We decided to do just one track to make the conference more communal, and have a hackathon and an unconference scheduled for the evenings.  

You’re also offering training this year.

Yes, Scala and Spark training sessions; both are filling up rapidly.  My company, By the Bay LLC, is a Typesafe Training Partner, and we offer Spark training jointly with Databricks. After the conference I’ll do training regularly.

In terms of topics in Scala, what’s new or different about this year’s conference?

The Big Data/Scala theme is even more powerful this year, with a separate keynote on Spark in Scala given by Databricks CTO Matei Zaharia, who wrote the original Spark in Scala. The other keynote will be by Marius Eriksen, whose work at Twitter has demonstrated how Scala can power web-scale companies in real time.

How many people submitted talks?

We received nearly 50 proposals for about half that number of speaking slots. Some of the proposals we compressed to halftime. It’s still plenty of time and much more informative than lightning talks, and lets folks get a taste of the tools and approaches (which are hard to pick up on your own, or not as exciting).  We hope folks who didn’t get into the main program will propose their talks to the unconference.

How many proposals came from women?

We do have talks from women, but should work more on the outreach.  I’m personally following up on it, and this work started to pay off.

Did you notice any patterns, trends or similarity topics-wise?

Big Data is ascendant, with Spark a darling of the whole Big Data community at large.  We see people registering to learn Spark, which they then realize is in Scala, and so they join SF Scala next and ramp up on Scala itself. Akka is a perennial favorite, with not one but two proposals showing how to run farms of R servers with it—building a SparkR at home, so to speak. Various aspects of Play and web applications are dominating, showing how Scala-based web apps are taking hold.

Who is traveling the longest distance to speak?

Someone is coming from Hungary, and another from Argentina. 

What surprised you about this year’s proposals?

The sheer strength and depth.  Also the amount of type-related work, which got many votes. Scala folks are intellectually rigorous and efficient at the same time, and that’s why we’ll eventually take over the world.

How has the SF/Bay Area Scala community grown/changed/evolved over the past year? What would a visitor expect to find?

Play, Akka and Spark are growing exponentially and bringing more and more Scala beginners to the community.  The importance of professional training is obvious, hence our offerings.  More startups are relying on Scala, and whole categories of businesses are aligning around it, such as healthcare-related API startups. 

If I’m a Java programmer who’s new to Scala, will I be able to follow along with the talks?

You’ll have a much better time if you begin by taking Fast Track to Scala with Brendan McAdams on August 6-7. Brendan is a bona fide Unix/Scala/Akka longbeard, and he authored both the original and reactive Scala drivers for MongoDB. But our Scala speakers are excellent communicators, so every level of ability will be able to advance to the next one.  See you shortly By the Bay!

Here’s Typesafe Senior Software Engineer/Scala In Depth author Josh Suereth earlier this week at Gilt’s NYC office, teaching his sbt workshop. Josh spent more than two hours covering settings, configurations, dependency management, and other topics. More than 50 non-Gilt technologists from the metro NYC area joined members of our engineering team for dinner and the class.

If you’re curious about sbt but couldn’t attend the workshop, check out Josh’s presentation from the 2014 Northeast Scala Symposium:

#gilttech’s excited to announce that Roland Tritsch will be the featured speaker at the next Dublin Scala Users Group meetup! Roland, who is Director of Business & Application Services at Fujitsu UK and Ireland, will present “Scala, Android & Bluetooth Low Energy: How to Make It All Work Together?" He’ll show us how to build Scala apps for Android and will then introduce the Scaloid framework as a way to do Scala development “the Scala way” on Android. As part of the presentation, we will build a simple BLE scanner (looking for beacons).

Roland has more than 25 years of experience building, running and fixing software engineering organisations and large distributed systems. His background includes positions at IONA Technologies, Gilt and HP. He likes computers, software and cycling, and these days is most interested in finding ways to make the “Internet of Things” work.

Roland’s an excellent speaker and teacher—we have first-hand knowledge of this!—so this will be a don’t-miss. Go here to RSVP!

More than 90 Dublin technologists came out to Docker Dublin’s inaugural meetup, co-organized by Gilt! Here’s Docker Dublin founder/organizer John Zanchetta kicking off the event:


And here’s Gilt Co-Founder and CTO Michael Bryzek, the main presenter of the night, describing how our engineering team has been using Docker and AWS to deliver immutable software releases to production:


The Docker Dublin group has tons of potential, and we’re very excited to be working with John to help the group expand. If you’d like to give a talk at an upcoming meetup, please send an email (include your suggested topic) via the DD meetup page. Then follow the group’s Twitter account to receive news and updates!


If you’re a NYC-area Scalasmith who can’t attend our July 3 Scala libraries course because you’re starting your patriotic gardenburger-grilling a day early, here’s another opportunity to learn Scala things at Gilt for free: On July 14, we’re hosting a free sbt workshop with Typesafe Senior Software Engineer Josh Suereth! Josh, the author of sbt in Action, will give an introduction to sbt and show how it can make your development flow better. The talk will be accessible to people who have not used Scala or sbt extensively, but will also include useful examples and tips for seasoned sbt users.

When: Monday, July 14, 2014 from 6-9 PM
Where: 2 Park Avenue, New York, NY
Cost: FREE (includes dinner)
Who: Gilt technologists, maybe you too

Fill out the form below to submit your contact info. (You must be age 18 or older to attend.) Please note that submitting this form does not guarantee you a seat—but we’ll try our best. We’ll also keep your contact info on file to let you know about future classes!



As part of our ongoing efforts to offer free Scala education to the Dublin and NYC tech communities, we’re hosting a free, full-day class on July 3 that focuses on Scala libraries! Targeted at experienced Scala engineers, the class focuses on safe and true abstraction with three of the typelevel.scala libraries: Scalaz, Shapeless and Spire.

Teaching the class will be Stephen Compall, a longtime Scalaz developer whose Scala knowledge comes from having studied with various typed functional programming gurus. While pursuing an understanding of a higher kind, he’s also contributed to Shapeless and the Scala standard library, and writes about type-oriented thinking and amusing compiler games for the typelevel.scala blog. As far as he can tell, he’s the only one with his name.

More on the libraries you’ll learn about:

Scalaz: Functors are everywhere

Scalaz is a general-purpose functional programming library.  Programmers run into trouble when trying to imagine their seemingly specific and “business-y” code as merely instances of abstract concepts already provided in Scalaz. We will look at some such code and see how we can borrow the power of Scalaz, by way of higher-kinded types, to avoid writing some of the functions (and eliminate the possibility of certain errors in others).

Shapeless: Don’t talk about configuration

Shapeless is famous for pushing the limits of Scala’s type system by way of its extremely abstract system of describing data structures and the core functions built over them. As with Scalaz, it’s a question of seeing where Shapeless is already part of your code. We will practice the art of Scala constraint propagation to combine highly abstract functions from Shapeless into new highly abstract functions, and use Shapeless to solve the problem of delivering different types of configuration to different modules of code via dead-simple dependency injection.

Spire: Make numerical code work fast

Spire is the library for numeric code, from people who know Scala specialization for fast, unboxed, optimized code. It manages amazing performance tricks without sacrificing a bit of type safety; in fact, it encourages stronger type safety, even in purely numerical code, by the usual typelevel means: breaking big concepts down into their true underlying abstractions.  We’ll see why Spire math functions follow some odd conventions, learn to write unboxed abstract code, numerical and not, by imitating the Spire authors, and get our newly abstract math functions to slow down for testing and speed up in production.

 When: Thursday, July 3, 2014 from 10 AM-6 PM
Where: 2 Park Avenue, New York, NY
Cost: FREE (includes breakfast and lunch)
Who: Gilt technologists, maybe you too

Fill out the form below to submit your contact info. (You must be age 18 or older to attend.) Please note that submitting this form does not guarantee you a seat—but we’ll try our best. We’ll also keep your contact info on file to let you know about future classes!


Since 2012 the Gilt engineering team has been using sbt to build and deploy microservices and webapps—from our newest Scala microservices to our most ancient Java applications. Microservices tend to have common functionality across the fleet, from Play Framework and Akka, to our own internal libraries.

Dependencies (and their complex transitive dependencies) have always been challenging to understand and manage. Not surprisingly, the open source community has produced some tools to make it easier. One open source tool we’ve relied on for managing dependencies is Johannes Rudolph’s awesome sbt-dependency-graph plugin.

Like Maven’s dependency plugin before it, sbt-dependency-graph produces an ASCII-art representation of an application’s dependency graph, making it easier to understand and spot problems. Unlike Maven, though, sbt-dependency-graph can generate different machine-readable graph formats, which enables composing interesting behavior on top of it.

Most of the time, developers just want to see the dependency graph, and sbt-dependency-graph’s default visual output comes up short. By leveraging the machine-readable formats, we can generate even more powerful visualizations. To that end, the Gilt team has created sbt-dependency-graph-sugar: an open-source plugin that provides some sbt-dependency-graph “sugar” to simplify understanding and managing dependencies.

If you’re working on a Mac with graphviz installed, sbt-dependency-graph-sugar “just works” to convert the dependency graph to an SVG file and automatically open it on Safari:


To use sbt-dependency-graph-sugar with sbt 0.13.x, you can add it as a plugin to a particular project by putting this line in a .sbt file in the project subdirectory:

addSbtPlugin("com.gilt" % "sbt-dependency-graph-sugar" % "0.7.4")

Or you can make it magically available to all your sbt projects by putting that line in a .sbt file in ~/.sbt/0.13/plugins/.

Once it’s installed, you can view the dependency graph for an application and its subprojects by running:


This will compute the dependency graph, convert it to svg, and open it with Safari.

You can also customize the plugin to open files with Chrome and other applications.

We’re hoping to contribute this feature to sbt-dependency-graph after we gain some traction, so give it try, give us feedback and stay tuned for updates. For now, enjoy!


Today Typesafe celebrates its first-annual Play Day with a full day of webinars—and #gilttech is honored to be a part of the activities! The sessions kick off at 9 AM PST with Adam Evans, Technical Lead for BBC Children’s Future Media, “Making The Case For Play,” and run through 5:45 PM PST with Typesafe Lead Developer James Roper on “Reactive Streams and Play 3.0.”

The Play Day module we’re most excited about (we’re a little biased :) ) comes at 2 PM PST, when Gilt Senior Software Engineer Giancarlo Silvestrin (Personalization Team) shares “Lessons Learned From Implementing Play Across Lots of Small Applications and Microservices.” Details on Giancarlo’s presentation:

This session will focus on Gilt’s implementation of Play Framework across its architecture—starting with Gilt Live, a real-time page showcasing purchases made in real-time, and eventually branching out to many critical parts of the company’s tech organization (from search to inventory status updates). Play has offered Gilt numerous advantages in terms of accessibility, scalability and performance. You’ll hear more about lessons learned and the many advantages Play has offered to Gilt, a rapidly-moving eCommerce company with more than eight million users.

Hundreds of people have already signed up for Giancarlo’s webinar—and you should, too! To register, just go here.