Lead Big Data Engineer: Databricks Lakehouse on Azure
Lead Big Data Engineer at SoftServe designing and building scalable batch and streaming data pipelines on the Databricks Lakehouse platform in Azure, using SQL and PySpark alongside services like Data Factory, Synapse, Data Lake, and DevOps, from PoC through production.
- Compensation
- Not disclosed
- Location
- Poland
- Work setup
- Not specified
- Seniority
- Lead
- Employment
- Not specified
- Experience
- Not specified
- Category
- Data engineering
- Posted
Skills
Must have
- Experience designing and developing scalable data solutions on the Databricks Lakehouse platform
- Experience working within Azure cloud environments
- Hands-on experience with Azure services such as Data Factory, Synapse, Data Lake, and DevOps
- Advanced SQL skills
- Advanced PySpark skills for data optimization
- Experience delivering batch and streaming data pipelines
- Ability to contribute to architectural decisions
- Ability to support projects from PoC to production
- Collaboration with technical and business stakeholders
Full description
SoftServe is seeking a data engineer to design and develop scalable data solutions using the Databricks Lakehouse platform within Azure cloud environments. You will collaborate with technical and business stakeholders to deliver high-quality batch and streaming data pipelines, contribute to architectural decisions, and support projects from PoC to production.
You will work with Azure services such as Data Factory, Synapse, Data Lake, and DevOps, applying advanced SQL and PySpark to optimize data