Big Data Platform Engineer
- Compensation
- Not disclosed
- Location
- India
- Work setup
- Remote
- Seniority
- Not specified
- Employment
- Not specified
- Experience
- 2+ years
- Category
- Data engineering
- Posted
Skills
Must have
- 1. Azure Kubernetes Service – AKS
- Strong Expertise
- ● AKS cluster deployment and administration ● Cluster upgrades and patch management ● Node pool management ● Ingress Controllers, including NGINX and Application Gateway ● RBAC implementation ● AKS se
- Experience
- ● Production support of AKS environments ● Multi-node and multi-environment AKS setups ● High availability and disaster recovery implementation
- 2. Kubernetes and Azure Networking
- Kubernetes Networking ● Kubernetes networking concepts ● Services, Ingress, and Load Balancers ● DNS resolution ● Pod-to-Pod communication ● Pod-to-Service communication ● Network Policies ● Service M
- Azure Networking
- ● Virtual Networks ● Subnets ● Private Endpoints ● Peering ● Network Security Groups ● User Defined Routes ● Azure Firewall ● VPN connectivity
- 3. Docker and Containerization
- ● Docker image creation and optimization ● Container lifecycle management ● Registry management ● Security scanning ● Multi-stage builds ● Container troubleshooting ● Azure Container Registry ● Image
- 4. Azure DevOps and CI/CD
- CI/CD Expertise ● Azure DevOps Pipelines ● YAML pipelines ● Build and Release pipelines ● Git repositories ● Branching strategies
- Infrastructure Deployment
- ● Infrastructure as Code ● ARM Templates ● Bicep ● Terraform
- Automation
- ● Deployment automation ● Environment promotion ● Release management
- 5. Monitoring and Observability
- Azure Monitor
- ● Monitoring strategy implementation ● Alert configuration ● Dashboard creation
Full description
Experience Requirements
● 8+ years of overall IT experience ● 5+ years of experience in Cloud Infrastructure Engineering ● 4+ years of experience working with Kubernetes platforms ● 3+ years of experience supporting enterprise-scale Data Platforms ● 3+ years of experience in DevOps/SRE practices ● 2+ years of experience working with Azure Data Services
Mandatory Skills
Candidates must possess strong hands-on experience in the following areas: ● Apache NiFi ● Azure Event Hub ● Azure Functions ● Enterprise-scale Data Platform Administration These are core and non-negotiable requirements. Profiles without hands-on experience in Apache NiFi and Azure Event Hub will not be considered.
Requirements
- Azure Kubernetes Service – AKS
Strong Expertise
● AKS cluster deployment and administration ● Cluster upgrades and patch management ● Node pool management ● Ingress Controllers, including NGINX and Application Gateway ● RBAC implementation ● AKS security hardening ● Autoscaling using HPA, VPA, and Cluster Autoscaler ● Persistent storage management ● Troubleshooting cluster issues ● Network policies
Experience
● Production support of AKS environments ● Multi-node and multi-environment AKS setups ● High availability and disaster recovery implementation
- Kubernetes and Azure Networking
Kubernetes Networking ● Kubernetes networking concepts ● Services, Ingress, and Load Balancers ● DNS resolution ● Pod-to-Pod communication ● Pod-to-Service communication ● Network Policies ● Service Mesh concepts, preferably Istio ● CNI Architecture
Azure Networking
● Virtual Networks ● Subnets ● Private Endpoints ● Peering ● Network Security Groups ● User Defined Routes ● Azure Firewall ● VPN connectivity
- Docker and Containerization
● Docker image creation and optimization ● Container lifecycle management ● Registry management ● Security scanning ● Multi-stage builds ● Container troubleshooting ● Azure Container Registry ● Image versioning and governance
- Azure DevOps and CI/CD
CI/CD Expertise ● Azure DevOps Pipelines ● YAML pipelines ● Build and Release pipelines ● Git repositories ● Branching strategies
Infrastructure Deployment
● Infrastructure as Code ● ARM Templates ● Bicep ● Terraform
Automation
● Deployment automation ● Environment promotion ● Release management
- Monitoring and Observability
Azure Monitor