CACI
Senior DevOps Engineer, Observability and FinOps
- Compensation
- Not disclosed
- Location
- Reston, VA, US
- Work setup
- Not specified
- Seniority
- Senior
- Employment
- Contract
- Experience
- 2+ years
- Category
- Devops
- Posted
Skills
Must have
- TS/SCI with polygraph
- Bachelor's degree in Computer Science, Information Technology, Software Engineering, or a related field.
- 3+ years deploying and operating at least one enterprise stack: Grafana/Prometheus ecosystem, Elastic (ELK), Dynatrace, or Splunk
- Metrics, logs, and traces instrumentation; working knowledge of OpenTelemetry or Prometheus exporters
- Built operational dashboards and alert rules used by real operators; understands SLI/SLO concepts
- 3+ years with Kubernetes in production; Red Hat OpenShift strongly aligned
- 2+ years with AWS (EC2, S3, EKS, IAM, CloudWatch, Cost and Usage Reports)
- Demonstrated experience analyzing cloud spend, applying tagging strategies, and delivering at least one cost optimization outcome
- Terraform and Ansible; Helm charts or Operators
- Built and maintained pipelines in GitLab CI, Jenkins, or equivalent; Git workflows
- Proficient in Python or Go plus scripting (Bash); able to write and query SQL and PromQL or equivalent
- RHEL administration and troubleshooting
- Hardening to DISA STIGs; familiarity with RMF and NIST SP 800-53 controls
- Worked in air-gapped or disconnected networks, including offline package and image management
Nice to have
- Hands-on with Grafana Alloy, Mimir, Loki, and Tempo at scale
- Production time on two or more of Grafana, Elastic, and Dynatrace, enabling side-by-side judgment
- Store-and-forward pipelines, edge collectors, Kafka or similar buffering over intermittent links
- Moved data or software through cross-domain solutions (CDS) or approved transfer processes
- OpenCost or Kubecost, CloudHealth, Apptio Cloudability, or AWS Cost Explorer and Cost and Usage Report analytics
- Istio or a comparable service mesh, including mesh-level metrics and tracing
- Cluster monitoring and user workload monitoring, Operators, OpenShift GitOps and Pipelines
Full description
Job Title: Senior DevOps Engineer, Observability and FinOps
Job Category: Engineering
Time Type: Full time
Minimum Clearance Required to Start: TS/SCI with Polygraph
Employee Type: Regular
Percentage of Travel Required: Up to 10%
Type of Travel: Continental US
Anticipated Posting End: 11/13/2026
The Opportunity:
This engineer makes mission systems visible and their costs accountable, on networks that cannot call out to the internet. The role blends platform engineering (Kubernetes/OpenShift, infrastructure as code, CI/CD) with enough full stack development to build the dashboards, APIs, and small internal tools that operators and financial stakeholders use every day.
Responsibilities:
- Deploy, configure, and operate the observability stack (for example Grafana Alloy, Mimir, Loki, and Tempo on OpenShift, or Elastic or Dynatrace where selected) on each network
- Instrument applications and infrastructure with OpenTelemetry and Prometheus exporters; standardize labels, tags, and naming
- Build real-time mission status dashboards and service health views for operators, engineers, and leadership
- Implement cost visibility and allocation across AWS, OpenShift, and on-premises resources using tagging standards and showback or chargeback models
- Build cost and utilization dashboards that join billing exports with observability metrics (unit cost per mission application, per namespace, per program)
- Identify and execute optimization: rightsizing, idle resource cleanup, storage lifecycle policies, Reserved Instance and Savings Plan coverage analysis
- Automate infrastructure and stack deployment with Terraform, Ansible, and Helm or Operators; manage configuration through GitOps (Argo CD or similar)
- Build and maintain CI/CD pipelines (GitLab CI, Jenkins, or Tekton) including image scanning, signing, and promotion into air-gapped registries
- Package software and dependencies for offline transfer and cross-domain delivery
- Develop lightweight web apps, APIs, and Grafana plugins or panels for custom status views and cost reports
- Write data integration scripts (Python, Go) that pull from billing exports, CMDBs, and ticketing systems
- Document runbooks, architecture diagrams, and user guides; train operators and mentor junior engineers
Qualifications:
Required:
- TS/SCI with polygraph
- Bachelor's degree in Computer Science, Information Technology, Software Engineering, or a related field.
- 3+ years deploying and operating at least one enterprise stack: Grafana/Prometheus ecosystem, Elastic (ELK), Dynatrace, or Splunk