Senior Data Engineer - Python/Spark

Role: Data Engineer

About the Role:

We are looking for a Data Engineer who has strong experience in building and managing data platforms, data pipelines, and scalable data infrastructure, with a good understanding of data security, privacy, governance, and access controls.

Key Responsibilities:

• Design, build, and maintain scalable and reliable data platforms and data pipelines.

• Develop robust ETL/ELT pipelines to ingest, process, transform, and distribute data.

• Work with large and complex datasets across structured and unstructured data sources.

• Design data architectures that are secure, scalable, reliable, and highly available.

• Implement data security controls including authentication, authorization, encryption, access management, and data masking.

• Define and implement appropriate data access controls and role-based permissions.

• Ensure compliance with data privacy, security, and governance requirements.

• Work closely with Data Scientists, Analysts, Product Managers, Software Engineers, and Security teams.

• Build data quality, monitoring, lineage, and observability mechanisms.

• Optimize data pipelines and platforms for performance, reliability, and cost.

• Contribute to data governance practices, including data classification, ownership, retention, and lifecycle management.

• Identify and address security risks and vulnerabilities across data infrastructure.

• Participate in architecture discussions and help establish best practices for data engineering and platform security.

Tech Stack & Experience:

• 3+ years of experience in Data Engineering or a related field.

• Strong hands-on experience building data platforms and scalable data pipelines.

• Strong knowledge of SQL and experience with Python, Java, or Scala.

• Experience with modern data platforms, data warehouses, data lakes, or lakehouse architectures.

• Experience with technologies such as Spark, Kafka, Airflow, Databricks, Snowflake, BigQuery, Redshift, or similar platforms.

• Strong understanding of data security and privacy principles.

• Experience with IAM, RBAC, encryption, secrets management, and secure data access.

• Experience with CI/CD, version control, monitoring, and infrastructure automation is preferred.

• Strong problem-solving and analytical skills.

What We're Looking For:

We are particularly interested in candidates who have built or significantly contributed to data platforms, rather than candidates whose experience is limited to developing individual data pipelines. The ideal candidate combines strong Data Engineering fundamentals with a security-first mindset and understands how to build data platforms that are scalable, reliable, governed, and secure.