INTEGRATZ
Senior Data Engineer (Databricks)
Builds and maintains scalable data pipelines on Databricks and Apache Spark, integrating diverse data sources and optimizing for performance and cost.
- Compensation
- Not disclosed
- Location
- Remote
- Work setup
- Remote
- Seniority
- Senior
- Employment
- Not specified
- Experience
- 5+ years
- Category
- Data engineering
- Posted
Skills
Must have
- 5+ years as a Data Engineer or similar role
- Hands-on experience with Databricks and Apache Spark
- Advanced SQL and Python skills
- Experience in cloud environments (Azure preferred)
- Solid understanding of data modeling, ETL/ELT processes, and data architecture
- Ability to work independently and collaborate with distributed teams
Full description
- Design, develop, and maintain scalable data pipelines using Databricks and Apache Spark.
- Define data architectures that support batch and streaming processing.
- Integrate data from multiple sources, including databases, APIs, and cloud storage.
- Optimize data solutions for performance, reliability, and cost efficiency.
- Collaborate closely with data science, analytics, and business teams.
- Provide technical leadership and guidance on data engineering best practices.
- Translate complex Databricks architectures and data solutions into clear, business-friendly explanations for technical and non-technical audiences.
- Document data architectures, pipelines, and technical decisions.
- Support production systems and troubleshoot data-related issues as needed.
- Minimum of 5 years of experience as a Data Engineer or in a similar role.
- Strong hands-on experience with Databricks and Apache Spark.
- Advanced SQL and Python skills.
- Experience working in cloud environments (Azure preferred).
- Solid understanding of data modeling, ETL/ELT processes, and data architecture concepts.
- Ability to work independently while collaborating with distributed teams.
- Competitive salary
- Flexible work environment
- Opportunities for growth and certifications
- Inclusive and collaborative team culture
- Access to cutting-edge tools and technologies