The Gilt technology organization. We make gilt.com work.

Gilt Tech

Gilt Tech + Jekyll = ♥

Andrew Powell tech blog

Gilt Tech Blog Moves to Github

In continuing the march towards a more open Gilt Tech, we've made the switch to Github Pages and Jekyll from Tumblr.

You can view the entirety of this blog's code and content here: https://github.com/gilt/tech-blog.

Why We Did This

First and foremost: control. We've got control over the layout and design of the blog from top to bottom, without restriction and without any kind of a funky theming engine to wrestle with. We're really good at this kind of thing (we should be, after all) and we've ended up with a blog design that matches elements of gilt.com.

We've also got better control over how we write posts. We're a fairly large, distributed organization, and coordinating the setup, permissions, et al for Tumblr was kind of a pain. There was also very little positive feedback about the authoring experience. With the move everyone is able to author in their editor of choice and go through the already familiar process of submitting the post to the tech-blog repository.

We Used Technology

Jekyll Bootstrap is a fantastic bootstrap/scaffolding package put together to quickly spin up a blog. While we customized the bajeezus out of it, which was incredibly quick and painless, and rolled our own 'gilt' theme, we used a healthy chunk of what it ships with by default. We did roll our own helpful tidbits, like the authors page, for which you can view the source here.

Github Pages. If you haven't looked into Github Pages for simple hosted websites or project websites yet, give it a gander. It's a phenomenal way to quickly spin up a site, and it's Jekyll support is pretty great.

Fin

We hope you like the new digs, and we're forecasting a lot more great content to come.

jekyll 1 github 3 blogging 1
Tech
Gilt Tech

Sean Smith to Present at Microservices Day!

Sean Smith conferences

What is Microservices Day?

from microservicesday.com

Microservices Day is a conference by the enterprise, for the enterprise. It brings together people from organizations that have adopted (or are planning to adopt) a microservices approach to running their business and focuses on the issues that companies face. It gives participants a forum to discuss and share their experiences with microservices through advice, tips and tricks as well as driving forward both the technology and business community. Come join us at The Altman Building, 135 West 18th Street in New York on July 13th! The event will consist of a range of talks from industry leaders who have deep experience building and operating microservices systems at scale. Panel discussions and active audience participation will be a further part of the event, as the implications of enabling business to innovate at a faster pace than ever before will be a key discussion point. Come join the early adopters at the dawn of a new era in business computing!

Sean Smith is a Software Engineer from Canada. He currently works at Gilt as a Principal Software Engineer. He has worked on a variety of services at Gilt, ranging from third party warehouse systems integration to email content generation and delivery.

Register now for this great event!  See you on July 13th!

microservices 18 meetup 6 conference 3 microservices day 1 software engineering 66
Tech
Gilt Tech

Introducing Gumshoe

Andrew Powell analytics

An analytics and event tracking sleuth.

In late 2014 Gilt received word that a core feature of Google Analytics (henceforth known as GA) would be removed in the next iteration of the platform. The ability to redirect collected data and events to a secondary endpoint had grown critical to our the analization of user behavior, patterns, and effectiveness of our user experience, as well as sales forecasting and predictive modeling. Our two most viable options for retaining this ability were to implement another tracking platform from a different vendor or create and implement our own, tailored to our specific needs.

https://github.com/gilt/Gumshoe

After taking a hard look at the heavy hitters of the analytics platform sphere it was decided that we’d take the divergent route and roll our own analytics and event tracking library. We won’t go into detail on why we passed on big players like Snowplow, Omniture, CoreMetrics, et al. Rather, we’ll focus on Gumshoe and its merits.

What Did We Need?

To anyone in development, marketing, or data sciences on the web, our list of needs will sound quite familiar.

First and foremost we required parity with GA’s base page data. The industry has come to be reliant and has standardized on the kind of data that GA collects for various reporting, including vital year-over-year reporting. So no shockers there.

As performance-minded developers we also had a low page footprint in mind.

We desperately needed organized event names and data. One major shortcoming of GA was the lack of enforcement of any kind of standard in event naming and data. Arbitrary event names and varying event data formats produced migraines the size of Mount Fuji for our Data Scientists. It wasn’t uncommon to have the same target event with several different names, in different applications, with different data in different delimited formats.

Related to the variance in event data, we were looking for a high degree of data integrity and confidence.

And last but not least, a low delivery failure or miss rate for the data collection.

Rolling Our Own

Events are events are events. In Gumshoe, everything is an event. There’s no disambiguation between page views, virtual page views, or custom user events. Everything is an event, and it’s a core tenant of Gumshoe.

Right off the bat, parity with GA was key. We started by breaking down the data that GA collected from each page view, and moved forward from there. A basic Gumshoe event consists of the following:

pageData
eventName
eventData

pageData consists of the GA parity data.
eventName is self-explanatory; the name of the event we’re passing.
eventData is custom data passed by the consumer which should be associated with an event.

Wunderbar! We had the recipe for sending a page view event. But how do we set this up so that others can easily send these events to where ever they’d like? Enter the transport.

Transports are the mechanism(s) by which consumers instruct Gumshoe on how it should send the data for each event. One can configure Gumshoe to send to multiple transports, or a single transport, it’s your choice. A brief example from the repository:

(function (root) {

  var gumshoe = root.gumshoe;

  gumshoe.transport({

    name: 'example-transport',

    send: function (data) {
      console.log('Gumshoe: Test Transport: Sending...');
      console.log(data);
    },

    map: function (data) {
      return {
        customData: {
          someData: true
        },
        ipAddress: '192.168.1.1'
      };
    }

  });

})(this);

The send method is responsible for actually sending the data. Gumshoe bundles the https://github.com/gilt/reqwest library, which any transports can freely use. At Gilt we use reqwest.js to send the data to our backend event stack, which we’ll cover in subsequent blog posts.

The map method allows consumers to extend the data which Gumshoe is sending through the transport. At Gilt we use this method to attach a giltData object that is sent along side pageData and contains vital Gilt-specific data for every single event that we send.

Using Gumshoe

Once you’ve got Gumshoe included on your page and you’ve specified a transport, you’ll need to initialize the library. It’s as simple as specifying the transport:

// tell gumshoe to use our transport  
window.gumshoe({ transport: 'example-transport' });  

And sending an event:

window.gumshoe.send('page.view', {});  

What’s Next

In forthcoming posts we’ll be talking more about the other goals we listed, specifically how we internally solved event organization, event naming standardization, data integrity, and the remainder of the stack that takes over once Gumshoe has delivered the payload.

We’ve been using Gumshoe on gilt.com for some time now, and have been running constant parity comparisons against GA and we’re very happy with the results thus far. However, we’d be bonkers to claim that Gumshoe is perfect. We’ve tailored Gumshoe to be extensible but so far it’s only been targeted for use at Gilt. We’d love feedback from the community at large. And if you feel you might be able to use Gumshoe, but find it lacking, we’d love to talk about how it can be improved.

gumshoe 1 google analytics 1 gilt 87 events 26 event tracking 1 andrew powell 1
Tech
Gilt Tech

Welcome to our Summer Interns!

Ryan Caloras culture

Introducing Gilt Tech’s first official Summer Apprentice Program class! Here’s our class described in their own words :D

Alex Luo

  • Has no middle name
  • Lived for more than two years in three different countries (China, US, Israel)
  • Walks to class in the snow in April because he goes to Cornell
  • Once volunteered by distributing medicine to poorer families in rural Dominican Republic
  • Voted “best eyes” for high school yearbook
  • Ate a cricket once
  • Fixed a graphics card by baking it in the oven
  • Broke a bone from accidentally falling off a building
  • Got kicked out of the public library once for “hacking” a computer
  • Actually can believe it’s not butter
  • Named by Time Magazine as 2006’s Person of the Year
  • Purposefully uses Comic Sans in documents to tick people off

Courtney Ligh

Courtney is a native New Yorker studying Computer Science at Dartmouth College. She is a future software developer and is excited about her summer internship here with Gilt’s Development Team. Besides coding, Courtney has a passion for food and prides herself with knowing all the best eateries around the city.  

Yogisha Dixit

The only thing you need to know about me is that I’m obsessed with chocolate. Literally…

Okay, maybe not. I’m interested in harnessing the power of computers to help make the world a better place. And I’m really looking forward to learning a lot and being challenged this summer.

That’s it. I swear.

And a very special Welcome Back to

Helena Wu

Helena was born and raised in São Paulo, Brazil. As a Computer Science student at Cornell University, she’s been on the Engineering Merit List and volunteers for different international organizations on-campus. When she’s not hacking on a project, you can find her rocking to metal songs! She also loves to stargaze in open fields on warm nights. She is excited for the summer and ready to explore NYC!

internships 1 Alex Luo 1 Helena Wu 3 Courtney Ligh 1 Yogisha Dixit 1 Gilt Tech Apprentice Program 1
Tech
Gilt Tech

Is Agile a Competitive Advantage?

Heather Fleming agile

Registration is open for this Webinar on May 27th!

Summary below from Atlassian.com:

A new Harvard Business Review Report asserts that agile development practices have steadily risen to become one of the most trusted and preferred methods of development across software teams in almost every industry. The study also discovered that by using agile frameworks, organizations can respond to market changes faster, deliver higher quality software, and gain a significant competitive edge. For these reasons HBR has dubbed agile development the competitive advantage for a digital age.

Join Jake Brereton, Sr. Brand Manager at Atlassian, as he discusses these HBR findings with Heather Fleming (Sr. Director, PMO, Gilt), Nate Van Dusen (Engineering Program Management Director, Trulia), and Maira Benjamin (Director of Engineering, Pandora); three seasoned agile experts who have successfully lead agile transformations and witnessed tangible, positive results firsthand.

In this webinar you’ll learn:

  • How Trulia benefited from taking engineering agile principles and spreading them throughout their entire Enterprise
  • What agreements engineering made with Gilt leadership to improve their unique agile development process
  • The way Pandora’s agile approach to implementing agile allowed them to keep what worked and throw out what didn’t, quickly
webinar 5 atlassian 3 gilt 87 pandora 1 trulia 1 HBR 1 Harvard Business Review 1
Tech
Gilt Tech

Last Chance to Join the Meetup Tonight!!!

Heather Fleming conferences

Adrian Trenaman, VP Engineering at Gilt will be speaking tonight at our Gilt offices on how we adopted Solr at Gilt to power our personalized search results!

We will be raffling off copies of Solr In Action after the presentation! Drinks and sandwiches will be served.

Registration closes soon, so sign up now!  It’s free!

Gilt Security requires all reservations to have a FIRST AND LAST name in order to enter the building.  Please make sure you include this when you RSVP.

meetup 6 Gilt 340 ade trenaman 2 solr 4
Tech
Gilt Tech

Importing Google Trends data

Igor Elbert analytics

Google Trends offers a trove of data for analysis. It’s not used nearly enough partially because good folks at Google did not provide an API to access the data. You can play with Trends in you browser, embed it into your webpages but it’s not that simple to get the raw data behind it to use it in your analysis.

There is a number of packages in Python, Perl, or R that pull the data for you but none of them did when I needed: compare hundreds of trends against each other.

You see, not contend with the lack of API Google returns trends on 1 to 100 scale so it’s hard to compare numbers for many different trends. You can plot several trends on the same graph but you will not be able to tell how they stand relative to another set of trends.

For example:

image

Above: “Game of Thrones” vs. “House of Cards”.
Below: “Orange is New Black” vs. “The Newsroom” 

Since each set is rescaled there is no way to tell how “House…” stands against “Orange…” 

image

I needed to get the trends for hundreds of fashion brands Gilt deals with and compare them against each other.

The logical solution seemed to use one search term as a baseline in every set and then rescale the results relative to a baseline term.

I borrowed heavily from GTrendsR package and came up with a script in R that pulls trends and rescales them relative to baseline.

It took some hacking: I took Google Trends export link that looks something like

www.google.com/trends/embed.js?hl=en-US&q=/m/06bkdx,+/m/0tlwzvq,+/m/0b6hm_f,+/m/0c5_m3,+/m/0f4w93&geo=US&date=2/2013+25m&cmpt=q&tz&tz&content=1&cid=TIMESERIES_GRAPH_0&export=5

and after looking at various export options found the one (export=3) that returns raw data in JSON-like string. For example:

http://www.google.com/trends/fetchComponent?q=%2Fm%2F06bkdx%2C%2Fm%2F0624dh&date=today%207-d&geo=US&cat=0-18&cid=TIMESERIES_GRAPH_0&export=3

returns:

// Data table response
google.visualization.Query.setResponse(
{"version":"0.6","status":"ok","sig":"881376537",
"table":{"cols":[
 {"id":"date","label":"Date","type":"date","pattern":""},
 {"id":"query0","label":"Agent Provocateur","type":"number","pattern":""},
 {"id":"query1","label":"Kate Spade","type":"number","pattern":""}
],
"rows":[{"c":[
 {"v":new Date(2015,3,18),"f":"Saturday, April 18, 2015"},
 {"v":0.0,"f":"0"},{"v":77.0,"f":"77"}
]},
{"c":[
 {"v":new Date(2015,3,19),"f":"Sunday, April 19, 2015"},
 {"v":0.0,"f":"0"},{"v":100.0,"f":"100"}
]},
{"c":[
 {"v":new Date(2015,3,20),"f":"Monday, April 20, 2015"},
 {"v":0.0,"f":"0"},
 {"v":65.0,"f":"65"}]},
{"c":[
 {"v":new Date(2015,3,21),"f":"Tuesday, April 21, 2015"},
 {"v":0.0,"f":"0"},{"v":62.0,"f":"62"}
]},
{"c":[
 {"v":new Date(2015,3,22),"f":"Wednesday, April 22, 2015"},
 {"v":0.0,"f":"0"},{"v":57.0,"f":"57"}
]},
{"c":[
 {"v":new Date(2015,3,23),"f":"Thursday, April 23, 2015"},
 ,
 {"v":null}
]},{"c":[
 {"v":new Date(2015,3,24),"f":"Friday, April 24, 2015"},
 ,
 {"v":null}
]}]}});

Which is turned into a valid JSON with 4 lines of R code and then parsed into R data structures with rjson package.

The result looks like:

image

And can now be analyzed, plotted, and joined with other data.

image

Because of the powers of SQL/MapReduce in our Teradata Aster database we can pull the trends and join with our relational data on the fly in SQL:

SELECT *
FROM STREAM (
      ON (SELECT brand_name from dim_brands)
      PARTITION BY brand_name
      SCRIPT(‘GTrends_MR.R’)
      OUTPUTS('week DATE, term VARCHAR, trend INTEGER’)
)

Any comments on Google Trends, R style and overall approach are appreciated.

Happy trending!

Google Trends 1 Data 1 How-To 1
Tech
Gilt Tech

Java has log4j...Objective-C has CocoaLumberjack...and now Swift has CleanroomLogger

Evan Maloney swift

Gilt Tech is proud to announce the release of CleanroomLogger 1.0, an open-source logging API for iOS.

CleanroomLogger provides a simple, lightweight and performant logging API written in Swift that will be readily understood by anyone familiar with other logging packages like CocoaLumberjack and log4j.

CleanroomLogger Highlights

CleanroomLogger provides these key features:

  • Classify log messages according to one of five severity levels: verbose, debug, info, warning and error
  • See the source file and line number of each log message recorded
  • Silently ignore messages below a given severity threshold
  • Far more respectful of the calling thread than NSLog()
  • A trace() function that can be used to trace an application’s code paths by logging the location of its caller
  • Configurable log behavior that puts the application developer in control of embedded code
  • Extension points that allow custom implementations for (1) log message formatting, (2) message filtering logic, and (3) the recording of log messages to an underlying facility.

Message logging

Each message sent to the log is associated with a severity value indicating the relative importance of the message.

You would send an informational message as follows:

Log.info?.message("The user has added \(count) items to the cart")

If a condition occurs that itself isn’t an error, but that might signal future trouble, you might issue a warning:

Log.warning?.message("We're running low on space!!!")

Finally, when things hit the fan, you’ve got:

Log.error?.message("Oh my, this is extremely embarrassing")

Each log message is recorded with the file and line that issued it, so you can very quickly figure out what code was responsible for a given message.

Execution tracing

As you’re developing and testing your code, you might want to understand the flow of execution your code takes. You can add trace() calls to various points in your code:

Log.debug?.trace()

The code above would print out the file and line number containing the trace() call, as well as the name of the calling function. The output looks like:

ModularTable.swift:364 — tableView(_:cellForRowAtIndexPath:)

Logging arbitrary values

Sometimes life hands you an NSError. Why bother constructing a new message? The NSError is the message:

Log.error?.value(err)

This function accepts Any?, meaning it’ll handle anything you throw at it.

Extensibility

CleanroomLogger’s extension points allow you to fully customize all aspects of logging without having to worry about the mechanics of how a logging engine is implemented.

The ability to provide one or more custom LogRecorder implementations is particularly powerful. You could implement your own LogRecorder to do things like:

  • Write messages to a file or set of (potentially rotating) files
  • Store messages with a warning or error severity in a database for subsequent diagnostic evaluation
  • Send messages with an error severity to a network endpoint to enable rapid response to codebase instability

And because more than one LogRecorder can be used at a time, you can multiplex your log messages to multiple destinations without writing any additional code.

About CleanroomLogger

CleanroomLogger is part of the Cleanroom project a set of open-source Swift projects launched as “an experiment in re-imagining Gilt’s iOS codebase in a legacy-free incarnation that embraces the latest Apple technology.”

The source code for CleanroomLogger is available on GitHub, and may be used, modified and redistributed freely under the MIT license.

swiftlang 1 swift 1 ios 5 xcode 1 log4j 1 cocoalumberjack 1 opensource 9 github 3 mobile 21 apple 2 iphone 2 ipad 2
Tech
Gilt Tech

Solr@Gilt-Powering Search and Listing for the Largest Flash Sale Site in America

Heather Fleming conferences

Reserve your spot before they’re all gone!

Learn how we adopted Solr at Gilt to provide personalised search results, from early days using Solr 3 on just three nodes, to a more modern clustered deployment today running Solr 4 that powers all listing pages on Gilt through our daily pulse load at noon. We’ll discuss what worked, what didn’t, and where we’re going next for search on gilt.com.

If you haven’t yet heard Ade Trenaman speak, you are in for a treat!  Be there: Thursday, May 14th at 6:30pm!

meetup 6 ade trenaman 2 solr 4 search 7
Tech
Page 1 of 64