Staff Data Platform Engineer
Senior hands-on engineer owning the end-to-end AI data platform at a deepfake-detection cybersecurity company: scaling a Databricks lakehouse with Spark/Ray distributed processing on AWS, building reproducible ML datasets and in-house synthetic deepfake generation pipelines using Python, SQL, and Terraform. NYC HQ or remote (U.S.).
- Compensation
- $210K–$260K / yr
- Location
- New York City
- Work setup
- Not specified
- Seniority
- Staff
- Employment
- Full time
- Experience
- 8+ years
- Category
- Data engineering
- Posted
Skills
Must have
- 8+ years building data platforms or data infrastructure, including staff/lead-level ownership of high-volume production pipelines
- Deep hands-on experience with distributed processing (Spark, Ray, or similar) and lakehouse platforms, ideally Databricks, on cloud object storage (S3, GCS, Azure Blob)
- Production experience with Databricks, Unity Catalog, Delta Lake, and Spark on AWS, including performance tuning, access controls, and operational troubleshooting
- Experience with large unstructured media (video, audio, images), including large files, decoding/transcoding, and metadata at scale
- Track record building reproducible ML datasets: versioning, lineage, data quality validation, and orchestration (Airflow, Dagster, Databricks Workflows, or similar)
- Strong Python and SQL, with end-to-end ownership of ambiguous platform problems on a lean team
- Infrastructure and deployment automation using Terraform and CI/CD, including AWS storage, IAM, compute, and Databricks resources
Nice to have
- Experience building synthetic data generation pipelines or running generative models (diffusion, voice cloning/TTS, video generation) at scale
- Background supporting ML research teams with training and evaluation data, especially for multimodal models
- Experience in security, fraud, trust & safety, or another adversarial domain, or at a Series A/B startup where scope outpaces headcount
Full description
Staff Data Platform Engineer
Location: NYC HQ or Remote (U.S.)| Full-time | Engineering
Who we are.
Reality Defender is an award-winning cybersecurity company helping enterprises and governments detect deepfakes and AI-generated media. Utilizing a patented multi-model approach, Reality Defender is robust against the bleeding edge of generative platforms producing video, audio, imagery, and text media. Reality Defender's API-first deepfake detection platform empowers teams and developers alike to identify fraud, disinformation campaigns, and harmful deepfakes in real time.
Backed by world class investors including DCVC, Illuminate Financial, Y Combinator, Booz Allen Hamilton, IBM, Accenture, Rackhouse, and Argon VC, Reality Defender works with leading enterprise clients, financial institutions, and governments in order to ensure AI-generated media is not used for malicious purposes.
Youtube: Reality Defender Wins RSA Most Innovative Startup
The Staff Data Platform Engineer Role.
Reality Defender's detection models are only as strong as the data behind them, and every new generative tool raises the bar on the volume and variety of media we need to train and evaluate against. The Staff Data Platform Engineer will own the shared AI data platform behind our vision, audio, and multimodal research, turning acquired and internally generated image, video, audio, and text data into reliable, reproducible datasets, from continuous ingestion through processing, governance, and export. This is a senior, hands-on role where you'll complete our Databricks and distributed processing platform and build the reusable infrastructure behind our in-house synthetic data generation, including producing the very deepfakes our models learn to catch. This is a full-time role, reporting to our Director of Data. Responsibilities include:
- Own the end-to-end data platform for AI research: continuous ingestion, preprocessing, governance, and export of image, video, audio, and text data at high volume.
- Complete and scale our Databricks lakehouse and distributed processing stack (Spark, Ray, or similar), making architecture and build-vs-buy decisions that hold up as data volume grows.
- Build reusable infrastructure for in-house synthetic data generation, so researchers can produce new deepfake variants across modalities on demand instead of one-off.
- Make every dataset reproducible and traceable through versioning, lineage, metadata, quality checks, and stable train/eval splits, so any model result can be tied back to the exact data behind it.
- Own pipeline reliability, observability, and processing cost per dataset as core metrics of the platform.
- Partner with ML researchers to understand their data needs and turn them into self-serve tools that speed up model iteration.