Data Engineer at SpotHero

SpotHero is seeking a Data Engineer to join the Data Engineering squad. This squad interacts with data consumers such as Data Science, Marketing, Engineering and Business Analytics to provide data solutions that meet their day-to-day needs and long term vision.

As a Data Engineer, you’ll focus on building data products and data pipelines for our stakeholders to enable them to model, store, access, process, and analyze SpotHero’s data. You’ll also get the opportunity to work on some of the most impactful projects at SpotHero, including integrations with partners such as Apple Maps and the pipelines that process the evergrowing search and clickstream datasets.

Who we are:

SpotHero is a parking reservation service that helps drivers find and reserve parking at thousands of lots and garages in many major North American cities. We are on a mission to bring the parking industry into the future through technology. Drivers in the U.S. and Canada use the SpotHero mobile app or website to reserve convenient and affordable parking on-the-go or in advance. Also, parking companies rely on us to help them reach new customers while optimizing their business. We combine hard-won industry knowledge, a large parking demand and supply dataset, and solid software engineering to serve both sides of the parking market.

What will you do:

• Work with our Analytics, Data Science, Marketing and other squads to understand their data pipelining and processing needs.

• Thoughtfully design data models that are clear, correct, and scalable.

• Design, create, and maintain data pipelines using Apache Airflow, Apache Spark, Docker, and Python

• Contribute to tools to help our software engineers, data analysts, and data scientists design and build their own data pipelines in a self-service manner.

• Ensure that the team’s data-processing code is well-tested and well-documented

• Collaborate with Marketing, Business Analytics, and other teams to design data products using SpotHero’s data warehouse (Redshift) and data lake (Trino)

Key Responsibilities: 

• Data Modeling/Architecting

  • Design data models with a broad understanding of underlying systems.

  • Create approachable, thorough documentation of data models describing how to access their data in a performant way.

  • Consult with stakeholders on the best practices for creation and deployment of data models and data flows.

• Data Processing

  • Define and enforce service level agreements between products owned and stakeholders, including configuration of monitoring and alerting

  • Understand data lineage and dependencies between data pipelines.

  • Design, implement, and maintain complex data processing pipelines which involve multiple integration points, including those which rely on distributed systems like Kafka and Spark.

  • Influence data processing and infrastructure practices across all of SpotHero.

  • Determine the best architecture, batch or streaming, for applications being built.

Your experience:

We care about your abilities, not how you gained them.

You might demonstrate the capabilities below through any combination of relevant professional experience, experience in a research setting, formal education, self-guided learning, open source contributions, or public speaking / writing / teaching experience.

• You have 2-4 years experience in data engineering.

• You are able to design and implement high-quality software in Python.

• Experience using SQL to read and manage data.

• Experience with an ETL scheduling tool (e.g. Airflow, Luigi, Prefect, or similar)

• Hands-on experience using multiple data platforms and tools (e.g. Airflow, Hive, Kafka, Postgres, Redshift, S3, Spark, Trino)

• Experience designing and implementing software (pipelines, services and client libraries) that is run in Docker containers, automatically tested on a continuous integration (CI) system, and versioned in git. You have experience writing shell scripts, Makefiles, or other configuration to glue together these components.

• Demonstrated experience designing and supporting technology intended to be used by other stakeholders.

• Strong ability to communicate on both business and technology subjects.

Nice to Haves:

• Message driven or streaming architectures, such as those with Kafka, Spark, Flink.

• Postgres, MySQL, or other RDBMS experience.

• Redshift, Presto, or other MPP database experience.

• Experience with a JVM language (we use Kotlin, but Java or Scala experience works).

• You have experience provisioning and managing infrastructure with infrastructure-as-code tools (we use Terraform, but experience with similar tools like CloudFormation, Pulumi, or SaltStack is totally fine!)

• Ability to deploy containerized software in Kubernetes, or sufficient experience in similar technologies like Apache Mesos or Amazon ECS.

Technology we use:

• Our Android Stack is: Kotlin and XML (standard for Android apps) using MVI architecture (still working on refactoring old views), our database layer is built in Realm. Bitrise for CI/CD. We also make heavy use of Dagger, RxJava, Espresso (testing). Network stack uses Retrofit.

• Our iOS Stack is: Swift using MVC architecture, CoreData for Local Storage, XCUI for UI Testing, XCTest for Unit testing, SPM for Package Management, Fastlane for app automation and build scripts, Bitrise for CI/CD, and Sentry for crash reporting.

• Our Back End Stack is: Monolith using Django/Python/PostgreSQL. We are moving our Monolith to a Modular Monolith, using Domain Driven Design. When relevant we extract specific domains to Services currently using Java, Kotlin and Go. We also use Docker, deploy our apps via Kubernetes. We use Kafka for asynchronous-, and gRPC for synchronous service-to-service communication. Our Integrations are on a .Net CORE, moving to Kotlin.

• Our Front End Stack is: React/Redux, Sass, Jest/React Testing Library/Cypress, and Webpack. We maintain a private npm repository with shareable UI components, utility functions, Babel/ESLint/Prettier configurations, and custom tasks

• Our Data Stack is: Our Monolith Database is Postgres and Redis for caching. We also use Redshift as our data warehouse and S3 as our data lake. The data lake is queried using Trino. We use Apache Airflow, Python, and Apache Spark for ETL. For streaming data, we use Apache Kafka managed by a vendor and use Kafka Connect, Kafka Streams, and Spark Streaming for stream processing. All machine learning work is done in Python, using the PyData ecosystem. Our analysts use Looker for internal-facing business intelligence reports, and Amazon Quicksight for external-facing data visualizations.

• Our Dev Tools Stack is: AWS+Kubernetes for hosting. Terraform + Helm Charts for IaaS/Deployment. ConcourseCI for CI/CD. Prometheus/Alertmanager/VictorOps for team alerting. We’re starting to work on multi-region available services.

What we are offering:

• Excellent benefits – 

  • In the US we cover up to 90% of Medical Premiums, 50% of Dental & Vision Premiums, company sponsored Life Insurance, 401K, and generous parental leave. 

  • In Canada we offer Medical (prescription drug and paramedical coverage), Dental, Vision, Life Insurance, STD and LTD

• Flexible PTO policy and great work/life balance – We value and support each individual team member.

• Annual parking stipend – we help people park!

• The opportunity to collaborate with fun, innovative, and passionate people in a casual, yet highly productive atmosphere.

• A workplace recognized as the Best Consumer Web Company by Built in Chicago, Top Company Culture by Entrepreneur, a Top Workplace by Chicago Tribune, and one of Chicago’s Best Places to Work for Women Under 35 by Crain’s Chicago Business. 

Steps to apply: Please include a resume and any other links which you think might improve our understanding of your abilities (e.g. a GitHub account or LinkedIn profile).

At SpotHero, we Respect Fellow Drivers by providing an inclusive interview experience for everyone, including people with disabilities. We are happy to provide reasonable accommodations to candidates in need of individualized support during the hiring process. Please let our team know of your need when you apply or as you begin interviewing with our team.

SpotHero is an equal opportunity employer. We know that a diverse workforce is the strongest workforce, and are committed to building and supporting an inclusive environment for all.