ZURICH CAPABILITY CENTER MEXICO
Data Engineer - Databricks
- Compensation
- Not disclosed
- Location
- Ciudad de México
- Work setup
- Onsite
- Seniority
- Not specified
- Employment
- Not specified
- Experience
- 3+ years
- Category
- Data engineering
- Posted
Skills
Must have
- Bachelor's Degree in Computer Science, Engineering, Information Systems, Mathematics, or a related field.
- 3+ years of experience in Data Engineering, Data Warehousing, or Big Data environments.
- Strong programming skills in Python and SQL.
- Hands-on experience with Databricks, Apache Spark, and distributed data processing.
- Experience designing and developing ETL/ELT data pipelines.
- Experience working with Data Lakes and modern data architectures.
- Understanding of data modeling concepts including Star Schema and Dimensional Modeling.
- Experience working with Git and version control systems.
- Strong analytical and problem-solving skills.
- English level B2 or higher.
Nice to have
- Experience with Azure Data Platform services, including:
- Experience working with Delta Lake and Lakehouse architecture.
- Knowledge of orchestration tools such as Airflow, Databricks Workflows, or Azure Data Factory.
- Experience supporting Machine Learning use cases and MLOps practices.
- Familiarity with MLflow and model deployment processes.
- Knowledge of CI/CD, Infrastructure as Code, and DataOps practices.
- Experience with streaming technologies such as Kafka or Event Hubs.
- Experience working in Agile environments.
- Insurance industry experience is a plus.
- Experience collaborating with Data Science teams and supporting AI/ML initiatives.
- Knowledge of Generative AI, LLM data pipelines, vector databases, and embedding workflows.
- Experience with Unity Catalog, data governance, and metadata management.
- Exposure to cloud-native architectures and enterprise-scale data platforms.
Full description
#ZCC
Zurich Capability Center is currently hiring a:
Data Engineer - DatabricksResponsibilities
As a Data Engineer, your main tasks will involve:
- Design, build, and maintain scalable data pipelines using Databricks and cloud-based data platforms.
- Develop, optimize, and automate ETL/ELT processes to support analytics, reporting, and machine learning initiatives.
- Work closely with Data Scientists, Analysts, ML Engineers, and business stakeholders to understand data requirements and deliver reliable datasets.
- Implement data ingestion frameworks from multiple structured and unstructured data sources.
- Build and maintain data models, data lakes, and data warehouses that support enterprise analytics needs.
- Ensure data quality, consistency, governance, and security across platforms.
- Optimize Spark workloads and Databricks environments for performance and cost efficiency.
- Support deployment and operationalization of machine learning solutions by providing production-ready datasets and feature pipelines.
- Collaborate with Azure, IT, and DevOps teams to implement CI/CD and DataOps best practices.
- Monitor, troubleshoot, and continuously improve data platform performance and reliability.
Your Skills and Experience
As a Data Engineer, your skills and experience will ideally include:
Required
- Bachelor's Degree in Computer Science, Engineering, Information Systems, Mathematics, or a related field.
- 3+ years of experience in Data Engineering, Data Warehousing, or Big Data environments.
- Strong programming skills in Python and SQL.
- Hands-on experience with Databricks, Apache Spark, and distributed data processing.
- Experience designing and developing ETL/ELT data pipelines.
- Experience working with Data Lakes and modern data architectures.
- Understanding of data modeling concepts including Star Schema and Dimensional Modeling.
- Experience working with Git and version control systems.
- Strong analytical and problem-solving skills.
- English level B2 or higher.
Preferred Qualifications
- Experience with Azure Data Platform services, including:
- Azure Databricks
- Azure Data Factory
- Azure Data Lake Storage