Data Engineer - Python/PySpark

We are seeking a skilled and motivated Data Engineer to design, build, and optimize our next-generation data pipelines. In this role, you will play a critical part in shaping our data architecture, leveraging Databricks, Python, and PySpark to transform raw data into actionable insights. You will collaborate closely with data scientists, analysts, and business stakeholders to ensure our data ecosystem is scalable, reliable, and highly efficient.

Key Responsibilities :

• Design, develop, and maintain robust ETL/ELT pipelines to ingest and transform large-scale datasets from diverse sources.

• Build, monitor, and optimize complex data workflows using orchestration tools like Azure Data Factory (ADF) or Apache Airflow.

• Leverage Databricks to write highly optimized, scalable PySpark and SQL queries, ensuring maximum performance and cost efficiency.

• Implement best practices for data warehousing, data engineering concepts, and schema design.

• Work with cross-functional teams to understand data requirements and deliver high-quality data products.

Technical Skills & Qualifications :

• Strong experience in Python, PySpark, and SQL.

• Proven experience working with Databricks.

• Solid track record of managing workflows using Azure Data Factory (ADF) or Apache Airflow.

• Deep understanding of ETL/ELT pipeline building, data engineering concepts, and workflow optimization.

• Familiarity with Delta Lake, Delta Tables, Parquet file formats, and DBT is highly preferred.