Data Engineer

This role is offered on a Contract-to-Hire basis, with an initial pre-defined contractual period. Upon successful completion of this period, the position is intended to transition to Burckhardt Compression’s permanent payroll.

Main responsibilites and tasks:

• Develop and operate the Azure Databricks data platform and enterprise data lake for analytics, AI and production data products.

• Design, implement and optimize production-grade data pipelines and transformation workflows using Python, SQL, PySpark and, where required, Scala.

• Translate business requirements into scalable ingestion, transformation, mapping and data-quality solutions.

• Administer Databricks workspaces, Unity Catalog, identities, permissions, clusters, jobs and platform integrations.

• Maintain the supporting Azure infrastructure, connectivity, security configuration and infrastructure-as-code deployments.

• Ensure reliable platform operations through monitoring, troubleshooting, upgrades, incident recovery and controlled production changes.

• Manage platform cost, technical documentation and operational continuity in collaboration with Data Science, MLOps, Cloud Infrastructure and Cybersecurity teams.

• Potentially extend development and operations scope into Microsoft Fabric for business intelligence and AI use cases.

• Strong data-engineering skills in Python, SQL and PySpark; Scala knowledge is an advantage.

• Hands-on experience with Azure Databricks, Apache Spark, Workflows, clusters, runtime management and performance tuning.

• Strong knowledge of Unity Catalog, cloud storage, identities, access controls and external locations.

• Practical Azure experience covering storage, networking, private endpoints, containers, identities and secrets.

• Experience with Terraform, Azure DevOps, Git, CI/CD and controlled production deployments.

• Ability to build and support scalable batch, event-driven and container-based data pipelines integrating files, APIs, IoT and cloud sources.

• Strong capability in platform security, monitoring, troubleshooting, performance optimization and cloud cost management; Microsoft Fabric, Power BI and Azure AI knowledge are advantages.

Education, Experience and Languages:

• Bachelor’s or Master’s degree in computer science, data engineering, software engineering or a comparable technical discipline.

• 5–8 years of relevant data engineering experience, including at least 3 years with Azure Databricks, Apache Spark and Azure cloud services.