Staff Engineer, Data Platform
Staff engineer who sets the technical direction for LILA Sciences' data platform: designing ingestion pipelines, storage/lakehouse architecture, orchestration, and data models that serve scientific and ML workflows, while mentoring engineers and setting standards. Core stack: Python, SQL, AWS, Kubernetes, and open table formats like Iceberg/Delta Lake.
- Compensation
- $192K–$272K / yr
- Location
- Cambridge, MA USA; San Francisco, CA USA
- Work setup
- Not specified
- Seniority
- Staff
- Employment
- Full time
- Experience
- 8+ years
- Category
- Data engineering
- Posted
Skills
Must have
- Bachelor's or Master's degree in Computer Science, Engineering, or related field
- 8+ years as a software or data engineer focused on building and operating data infrastructure
- Designed and shipped data platform components from the ground up (ingestion frameworks, storage abstractions, orchestration systems)
- Fluent in Python and SQL; writes production-quality code
- Production experience with relational and NoSQL databases, schema design, and query optimization at scale
- Comfortable working across structured, semi-structured, and unstructured data
- Cross-functional experience with scientists, ML researchers, and engineers; can explain technical trade-offs to diverse audiences
- Experience with cloud infrastructure and containerized deployment (AWS, Kubernetes)
- Hands-on experience with modern table formats and open lakehouse patterns (Iceberg, Delta Lake, Hudi)
Nice to have
- Experience with workflow orchestration systems (Flyte, Airflow, Dagster, or similar)
- Experience building data infrastructure for agentic/LLM workflows (vector databases, RAG infrastructure, retrieval-optimized access)
- Background in scientific computing, life sciences, or research software
- Proficiency with AI-assisted development tools (Cursor, Claude Code, or similar)
Full description
Your Impact at LILA
Lila Sciences is building the software platform that makes automated scientific discovery possible. At the heart of that platform is data: raw outputs from laboratory instruments, experimental model results, curated public datasets, and the scientific literature that contextualizes all of it. The data platform team is responsible for the infrastructure that moves, stores, transforms, and surfaces this data across the organization.
We are looking for a Staff Engineer to set the technical direction for our core data infrastructure: ingestion frameworks, storage architecture, orchestration patterns, and the interfaces that let scientists and ML researchers work with data reliably at scale. You will work closely with software engineers, machine learning researchers, and lab scientists to understand requirements and translate them into durable platform capabilities.
This is a role for engineers who care deeply about how data systems are designed. You will establish the architectural patterns and engineering standards the broader team builds on, mentor engineers across the data platform group, and make technical decisions that compound over time.
What You'll Be Building
- Data Platform Architecture: Design and evolve the core data infrastructure that ingests, stores, and serves data across scientific and ML workflows. Make principled build-vs-buy decisions and establish architectural patterns adopted by the broader engineering organization.
- Ingestion and Integration: Build reliable pipelines that bring in data from diverse sources: laboratory instruments, public scientific datasets, and external research literature. Own the interfaces between upstream producers and downstream consumers.
- Orchestration and Reliability: Operate and extend workflow orchestration systems that run complex, multi-step scientific pipelines. Ensure observability, fault tolerance, and reproducibility across the data stack.
- Data Modeling and Schema Strategy: Define and maintain data models, schema evolution practices, and data contracts that ensure consistency, discoverability, and long-term durability of scientific and platform data assets.
- Cross-Functional Technical Leadership: Partner with ML researchers, lab scientists, and product engineers to translate scientific and research requirements into platform capabilities. Drive alignment on data standards and integration patterns across teams.
- Engineering Standards and Mentorship: Establish coding, review, and design standards for the data platform team. Mentor engineers, lead design reviews, and raise the technical bar across the group.