Staff Data Platform Engineer

This is a remote position.

About the Team/Role

The Data Platform Engineering Team acts as the backbone of our enterprise data architecture, bridging the gap between Data Engineering, Infrastructure, and Operations. Responsible for architecting, scaling, and maintaining multi-cloud infrastructure across AWS and Azure, the team takes direct ownership of core Apache Airflow orchestration, big data frameworks, and containerized environments to ensure a robust, production-grade platform.

We are seeking a Staff Software Engineer (Semantic Foundationa) with 6+ years of hands-on experience to join this team as a core platform engineer and DevOps specialist. In this role, you will take direct ownership of our orchestration and containerization stack while driving key initiatives in CI/CD automation, observability, data governance, and cloud cost optimization. You will bridge the gap between Data Engineering, Infrastructure, and Operations.

How you’ll make an impact

Infrastructure, Orchestration & DevOps

• Airflow Infrastructure Ownership: Design, deploy, scale, and maintain highly available Apache Airflow clusters (using Helm, Kubernetes, and Terraform) to support critical enterprise ETL/ELT workflows.

• Infrastructure-as-Code & GitOps: Drive DevOps practices using Terraform, Helm Charts, and ArgoCD to automate platform deployments, manage self-hosted GitHub runners, and enforce GitOps workflows.

• CI/CD & Automation: Architect and manage robust CI/CD pipelines utilizing GitHub Actions for seamless deployment of data pipelines, infrastructure components, and DAGs.

• Container & Cluster Management: Provision and manage scalable Kubernetes (EKS/AKS) clusters, Docker containers, and underlying cloud infrastructure across AWS (EC2, EMR, S3, VPC) and Azure (Synapse, ADLS).
• Observability, Telemetry & Alerting: Build and maintain end-to-end monitoring, logging, and alerting systems using Grafana, Prometheus, Loki, and centralized log management solutions to ensure high platform uptime and reliability.

Central Data Platform, Tools & Governance

• Big Data Platform Architecture: Build scalable data infrastructure supporting big data processing engines and data warehouses, including Apache Spark, AWS EMR, Snowflake, Azure Synapse, Apache Kafka, and dbt.

• Data Lineage & Governance: Deploy and maintain central data discovery and metadata tooling (e.g., DataHub) to facilitate data governance, schema management, and cataloging.

• Cost Optimization & FinOps: Actively monitor, audit, and optimize data infrastructure compute and storage costs across AWS and Azure (EC2, EMR, Snowflake queries, Kubernetes nodes).

• Developer Experience & Tooling: Build internal tools, CLI utilities, and dynamic workflow templates to improve developer productivity for data engineers, analytics engineers, and data scientists.