Senior Data Engineer
- Compensation
- Not disclosed
- Location
- United States (Remote)
- Work setup
- Remote
- Seniority
- Senior
- Employment
- Not specified
- Experience
- 6+ years
- Category
- Data engineering
- Posted
Skills
Must have
- 6+ years in data engineering or analytics engineering.
- Advanced experience building production data pipelines with Apache Spark (PySpark and SQL), including performance optimization.
- Hands-on production experience with Databricks (Delta Lake, Jobs, Workflows, Unity Catalog).
- Deep experience with at least one major cloud (AWS, Azure, or GCP).
- Strong SQL and data modeling fundamentals.
- A track record of building and operating production-grade data pipelines, not just prototypes.
- Comfortable working directly with client stakeholders to translate business requirements into technical solutions.
- Demonstrated ability to independently own a data engineering workstream from design through production.
Nice to have
- Experience with Delta Live Tables, structured streaming, or real-time pipelines.
- Familiarity with dbt, Airflow, or other modern data stack tooling.
- A demonstrated performance- and cost-optimization mindset (cluster sizing, Photon, file layout, partitioning).
- Exposure to governed or regulated environments (e.g., financial services, healthcare).
- Prior consulting, systems integrator, or professional services experience.
Full description
About Climb
Climb is a Data and AI consultancy that partners with enterprises to design, build, and operationalize modern data platforms and production AI systems. As a Databricks partner, we go deep on lakehouse architecture, machine learning, and applied AI, with a bias toward production over proof of concept. Our team brings deep technical expertise and a builder's mindset to every engagement, and we measure our work not just by what ships, but by the business impact it drives.
Role Summary
Senior Data Engineers own complete data engineering workstreams within client engagements. You design, build, and operationalize the data platforms that power analytics and AI, taking your work from discovery and design through implementation, production, and handoff.
This is a hands-on engineering role built around ownership. You work directly with client stakeholders to translate business requirements into scalable data solutions, making the technical decisions necessary to deliver reliable, secure, and cost-efficient platforms. While the Data Architect owns the overall platform architecture, you own the successful delivery of your workstreams and the quality of the systems you build.
Our work focuses on modernizing enterprise data platforms using Databricks and the broader cloud ecosystem, enabling organizations to build trustworthy data products and AI-ready foundations that last beyond the engagement.
Key Responsibilities
- Own one or more data engineering workstreams, from technical design through production deployment and handoff.
- Design and implement scalable data models and lakehouse architectures, including medallion patterns where appropriate.
- Optimize performance and cost across Databricks and the underlying cloud: you treat compute spend as your problem, not someone else's.
- Orchestrate workflows using Databricks Workflows, Delta Live Tables, or equivalent tooling.
- Implement governance, security, observability, and lineage with Unity Catalog, and stand up CI/CD for data.
- Design and build AI-ready data platforms that enable reliable analytics, machine learning, and agentic applications.
- Work directly with client stakeholders to gather requirements and translate them into technical solutions.
- Review code, mentor junior engineers, and maintain high engineering standards across your workstreams.
- Contribute reusable accelerators, frameworks, and best practices back to the practice.
Required Qualifications
- 6+ years in data engineering or analytics engineering.
- Advanced experience building production data pipelines with Apache Spark (PySpark and SQL), including performance optimization.