rafer@ralua.com
303.210.5970
Denver
Rafer Hazen
Overview
- Experience building backend data systems with an emphasis on asynchronous messaging/queueing
- Building, growing and managing engineering teams
Experience
GitHub
- Staff Engineering Manager - Data Pipelines
Feb 2017 - Sept 2021
- Started the Data Pipelines team and grew it to 8 engineers
- Lead the release of project Hydro, a Kafka-backed event logging, storage and processing system now used by 30+ teams
- Hydro facilitates both asynchronous app-to-app communication and data warehouse ingestion and accepts ~250k writes/second at peak
- Wrote Hydro's data warehouse ingestion component (event ingestion and schema management)
- Lead the replacement of GitHub's background job system with a service that provides queue semantics over Kafka's log structured model
- Wrote the job queueing system's sharding algorithm to distribute messages across an in-memory dispatch tier
GitHub
- Senior Engineer
Nov 2015 - Feb 2017
- Operational support and development of GitHub's in house-event data pipeline and warehouse
- Evaluation, execution and support of Presto rollout
- Developed application to acquire data for machine learning jobs and serve results to user-facing applications
- Lead support and development of our internal A/B testing framework
Collective Intellect/Oracle
- Principal Software Engineer and Team Lead
Feb 2012 - Nov 2015
- Managed the API Team responsible for development, delivery and support of our group's public APIs
- Implemented core data models and interchange formats
- Delivered a performance-oriented overhaul of the infrastructure for our semantic categorization engine
- Lead development of "Semantic API" to expose text-processing capabilities
- Developed and delivered standardized "slug" deployment model to reduce errors and increase predictability
- Rewrote legacy data ingestion framework to allow for horizontal scaling via the introduction of a queueing layer
Foraker Labs
– Senior Software Developer and Team Lead
Jan 2010 – Feb 2012
- Worked with project managers, developers and clients to complete projects
- Spearheaded efforts to create and improve Foraker's culture of test-driven development
- Developed interactive browser-based choropleth maps
- Wrote an extensible framework to import, relationalize and map health insurance usage data
Lijit Networks
– Software Engineer
Mar 2009 – Aug 2009
- Maintained and augmented a Ruby ETL utility to warehouse search usage data from Apache log files
- Assisted in research and development of Lijit’s authority ranking metric
- Day-to-day maintenance and feature implementation for lijit.com
Sun Microsystems
– Intern
Jul 2008 – Jan 2009
- Built and deployed Oracle-backed PHP front end for an internal sales metric portal
Education
- [2006-2009] BS in Computer Science from University of Colorado, Boulder
- [2005-2006] New York University