Senior Data Engineer

Note for all engineering roles: with the rise of fake applicants and AI-enabled candidate fraud, we have built in additional measures throughout the process to identify such candidates and remove them.

About Us

People Data Labs (PDL) is the provider of people and company data. We do the heavy lifting of data collection and standardization so our customers can focus on building and scaling innovative, compliant data solutions. Our sole focus is on building the best data available by integrating thousands of compliantly sourced datasets into a single, developer-friendly source of truth. Leading companies across the world use PDL’s workforce data to enrich recruiting platforms, power AI models, create custom audiences, and more.

We are looking for individuals who can balance extreme ownership with a “one-team, one-dream” mindset. Our customers are trying to solve complex problems, and we only help them achieve their goals as a team. Our Data Engineering Team is the secret sauce behind all that we do and we are looking for the best of the best.

If you are looking to be part of a team discovering the next frontier of data-as-a-service (DaaS) with a high level of autonomy and opportunity for direct contributions, this might be the role for you. We like our engineers to be thoughtful, quirky, and willing to fearlessly try new things. Failure is embraced at PDL as long as we continue to learn and grow from it.

What You Get to Do

• Build infrastructure for ingestion, transformation, and loading an exponentially increasing volume of data from a variety of sources using Spark, SQL, AWS, and Databricks

• Building an organic entity resolution framework capable of correctly merging hundreds of billions of individual entities into a number of clean, consumable datasets.

• Developing CI/CD pipelines and anomaly detection systems capable of continuously improving the quality of data we're pushing into production.

• Dreaming up solutions to largely undefined data engineering and data science problems.

The Technical Chops You’ll Need

• 5-7+ years of industry experience with clear examples of strategic technical problem-solving and implementation

• Strong software development fundamentals.

• Experience with Python

• Expertise with Apache Spark (Java, Scala, and/or Python-based)

• Experience with SQL

• Experience building scalable data processing systems (e.g., cleaning, transformation) from the ground up.

• Experience using developer-oriented data pipeline and workflow orchestration (e.g., Airflow (preferred), dbt, dagster or similar)