Software Engineering Lead (GO & Python, AWS)

Optum is a global organization that delivers care, aided by technology to help millions of people live healthier lives. The work you do with our team will directly improve health outcomes by connecting people with the care, pharmacy benefits, data and resources they need to feel their best. Here, you will find a culture guided by inclusion, talented peers, comprehensive benefits and career development opportunities. Come make an impact on the communities we serve as you help us advance health optimization on a global scale. Join us to start Caring. Connecting. Growing together.

We are seeking a Senior Software Engineer to join the Enterprise Logging team and lead development of a next-generation log processing pipeline. This role is responsible for designing, building, and operating a high-throughput Lambda-based consumer that transforms and delivers ~500 TB/day of cloud log data from AWS, GCP, and Azure into Splunk Cloud. The ideal candidate has deep experience with streaming data systems, multi-cloud environments, and building production-grade data pipelines on a scale.

Primary Responsibilities:

• Design and develop a serverless event processing pipeline (AWS Lambda) that reads from Kinesis Data Streams, transforms events, and delivers CIM-compliant payloads to Splunk Cloud via HTTP Event Collector (HEC)
• Implement event parsing and transformation logic across multiple cloud log formats (CloudTrail, VPC Flow Logs, GCP Audit Logs, Azure Activity Logs, and others)
• Build and maintain source/source type mapping configurations supporting 59,000+ unique log type combinations
• Develop tiered data routing logic (Keep / Reduce / Skip) to optimize Splunk licensing costs
• Implement timestamp extraction and normalization across heterogeneous event formats
• Operate and monitor the pipeline in production - build observability dashboards, alerting, and error handling (dead-letter queues, retry logic)
• Collaborate with the Centralized Logging team on Kinesis stream integration and Enhanced Fan-Out configuration
• Participate in architecture decisions, code reviews, and CI/CD pipeline development
• Write infrastructure as code (Terraform or CloudFormation) for all deployed components
• Contribute data standards and onboarding documentation for new log sources
• Comply with all applicable Company policies, procedures, and business directives, changes including those relating to work location, team assignments, work schedules, and flexible work arrangements

Required Qualifications:

• Cloud Platforms (Multi-Cloud)
• AWS (primary): Solid hands-on experience with Lambda, Kinesis Data Streams, S3, IAM, CloudWatch, PrivateLink, DynamoDB, and SQS
• GCP: Working knowledge of Cloud Logging, Pub/Sub, IAM, and GCP audit log formats