Senior Data Engineer (Databricks)
Senior Databricks-focused data engineer at digital product consultancy Livefront, working remotely from Peru. Day to day: designing and building production Lakehouse pipelines (DLT, PySpark, dbt), Unity Catalog governance, and data/AI foundations on AWS/Azure for consulting clients.
- Compensation
- Not disclosed
- Location
- Remote (Peru)
- Work setup
- Remote
- Seniority
- Senior
- Employment
- Not specified
- Experience
- 4+ years
- Category
- Data engineering
- Posted
Skills
Must have
- 5-8+ years of data engineering experience, with at least 4 years in production Databricks environments, preferably in consulting or client delivery
- Solid working knowledge of AWS and Azure cloud services relevant to Databricks deployments (storage, networking, IAM, compute)
- Deep production-grade Databricks expertise: Lakeflow Declarative Pipelines, Autoloader, Structured Streaming, Lakeflow Jobs, Unity Catalog
- Proven experience designing Lakehouse architectures: medallion patterns, Delta Lake table design, partitioning, Z-ordering, query optimization
- Hands-on experience with data pipeline testing, observability, and CI/CD for data (unit testing, data quality frameworks, Git, Declarative Automation Bundles)
- Strong proficiency in SQL and Python
- Understanding of data modeling, schema design, and query optimization
- Excellent communication skills with technical and non-technical stakeholders
- Strong problem-solving skills and ability to navigate ambiguous requirements
- Above-average discipline and personal organization skills
- Comfort with critique and peer review in an iterative development process
- Demonstrated hunger for personal and professional growth
Nice to have
- GCP familiarity
- Experience with real-time streaming technologies (Kafka, Kinesis, etc.)
- Hands-on experience with alternative cloud data platforms
- Experience in healthcare or fintech domains
- Hands-on MLOps/LLMOps experience on Databricks (MLflow, model registry, Model Serving, Vector Search for RAG)
- Experience with Java, Go, or Scala
- Strong illustration skills for technical diagramming and data architecture documentation
- Speaking, writing, or educating publicly about data engineering topics
- Contributions to open-source data projects
- Databricks certification (Data Engineer Associate/Professional or Apache Spark Developer), held or actively pursued
- Experience with Databricks Apps or Lakebase
Full description
At Livefront, we help companies design and build world-class digital products that command attention and inspire joy. We’ve helped household names like CVS, Samsung, General Mills, and Optum create experiences that have reached millions of people, and startups like HomeSpotter and Credly build entirely new businesses that challenge their industries’ status quo.
We're looking for an outstanding Senior Databricks Data Engineer to join our growing data practice — someone who builds the data and AI foundations that digital products and intelligent experiences run on. This is a senior-level position with the opportunity to work remotely inPeru.
Who you are
You are a Databricks-focused Data Engineer who understands that great data platforms are only as valuable as the products, AI workflows, and experiences they enable. You bring deep, production-grade expertise across the Databricks platform and know how to connect platform capabilities to real business outcomes.
You thrive in ambiguity and can quickly assess a client's data landscape to recommend and implement the right solutions. You understand that in consulting, your Databricks depth is most valuable when it connects platform capabilities to the products and experiences clients actually use — and you're as comfortable in a product design conversation as you are building a DLT pipeline.
You excel at translating complex data challenges into clear technical requirements and can confidently navigate conversations with everyone from data scientists to executives. Your engineering principles are mature and grounded in real-world experience across various industries and scales.
You have an interest in and a curiosity about data platforms and the latest advances in data technology.
What you will be doing
- Design and build production data pipelines using Lakeflow Declarative Pipelines, Autoloader, and Structured Streaming, with end-to-end ownership of ingestion, transformation, data quality expectations, and CI/CD deployment via Declarative Automation Bundles.
- Architect and implement Lakehouse solutions on Databricks — medallion architecture, Delta Lake, Unity Catalog — tailored to the client's analytics, AI, and application needs.
- Build and maintain Databricks transformation layers — DLT pipelines, PySpark notebooks, and dbt — with data quality constraints and SLAs baked in.
- Design and maintain the data and AI foundations — Unity Catalog, Feature Store, MLflow, and Model Serving — that power production ML, agent workflows, and AI-enabled digital products.
- Collaborate with product and backend engineers to design data models, APIs, and application data contracts — ensuring the platform serves the product, not just the warehouse.