Extensive experience in site reliability engineering, infrastructure engineering, or platform engineering, with a proven track record of operating high-traffic, business-critical production services.
Deep hands-on experience with Kubernetes, a major cloud provider, and infrastructure as code tools such as Terraform.
Strong understanding of SLOs, observability, incident management, and reliability engineering principles, with the ability to prioritize improvements based on business impact.
Proficiency in a programming language such as Rust or Python, along with the ability to read, troubleshoot, and contribute to production codebases.
Experience with API gateways, reverse proxies, or traffic management systems, including load balancing, rate limiting, failover, and caching.
Practical experience with Consul and Vault for service discovery, configuration management, and secrets management.
Knowledge of authentication and authorization mechanisms for internal platforms and APIs, including OAuth 2.0/OIDC, JWT, mutual TLS (mTLS), and API keys, particularly in multi-tenant environments.
Experience with model evaluation systems, including building or operating evaluation pipelines to compare models, prompts, and agents and identify quality regressions before and after deployment.
Familiarity with coding agents and agent harnesses, such as OpenHands, Claude Code, or comparable tools, including safe execution in sandboxed environments.
Experience with agent memory systems and supporting data stores, such as vector databases, search systems, relational databases, and key-value stores, with an emphasis on reliable operations.
Staff-level technical leadership, including the ability to establish direction across teams, write effective design documents, and build alignment around technical decisions.
Strong judgment when balancing reliability, infrastructure costs, and developer productivity.
Full description
This position is listed on behalf of a partner company, which manages all applications and next steps. Our partner is looking for a Staff Site Reliability Engineer – Dev Platform based in Brazil.
This is an opportunity to shape the reliability and scalability of a critical internal developer platform within a global financial technology environment. You will take ownership of an AI Gateway that provides engineering teams with governed, self-service access to leading large language models (LLMs), helping optimize performance, resilience, security, and cost. As a senior technical leader, you will define reliability standards, guide architectural decisions, and ensure the platform can support growing AI workloads across the organization. You will work across engineering, security, finance, and AI governance to build robust infrastructure and effective operational practices. The role combines hands-on site reliability engineering with strategic technical leadership, incident management, and continuous improvement. It is ideal for an experienced infrastructure professional passionate about cloud-native systems, AI platform engineering, and enabling developers to build and deliver software more efficiently.
Accountabilities
- Own the reliability, availability, and performance of the production AI Gateway, defining and maintaining service-level objectives (SLOs), error budgets, and on-call practices.
- Design and implement resilience mechanisms, including multi-provider model routing, failover, retries, rate limiting, and graceful degradation when external model providers experience disruptions.
- Set technical direction for gateway architecture, capacity planning, and scaling to accommodate increasing LLM traffic and expanding workloads.
- Develop comprehensive observability for AI traffic, including latency, errors, token consumption, and cost attribution by team and use case.
- Collaborate with Security, Finance, and AI governance stakeholders to implement access controls, usage budgets, audit logging, and appropriate data-handling policies.
- Lead incident response and conduct blameless post-incident reviews, translating findings into long-term reliability improvements.
- Automate infrastructure and operational processes through infrastructure as code, safe deployment strategies, and progressive rollouts.
- Support secure and reliable integrations with LLM providers and internal AI services.
- Mentor engineers, establish engineering best practices, and raise operational standards across the developer platform organization.
Staff Site Reliability Engineer - Dev Platform at Jobgether (Remote) — aijobsearch
- Influence technical decisions across teams through clear design documentation, architectural guidance, and collaborative leadership.
Requirements
- Extensive experience in site reliability engineering, infrastructure engineering, or platform engineering, with a proven track record of operating high-traffic, business-critical production services.
- Deep hands-on experience with Kubernetes, a major cloud provider, and infrastructure as code tools such as Terraform.
- Strong understanding of SLOs, observability, incident management, and reliability engineering principles, with the ability to prioritize improvements based on business impact.
- Proficiency in a programming language such as Rust or Python, along with the ability to read, troubleshoot, and contribute to production codebases.
- Experience with API gateways, reverse proxies, or traffic management systems, including load balancing, rate limiting, failover, and caching.
- Practical experience with Consul and Vault for service discovery, configuration management, and secrets management.
- Knowledge of authentication and authorization mechanisms for internal platforms and APIs, including OAuth 2.0/OIDC, JWT, mutual TLS (mTLS), and API keys, particularly in multi-tenant environments.
- Experience with model evaluation systems, including building or operating evaluation pipelines to compare models, prompts, and agents and identify quality regressions before and after deployment.
- Familiarity with coding agents and agent harnesses, such as OpenHands, Claude Code, or comparable tools, including safe execution in sandboxed environments.
- Experience with agent memory systems and supporting data stores, such as vector databases, search systems, relational databases, and key-value stores, with an emphasis on reliable operations.
- Staff-level technical leadership, including the ability to establish direction across teams, write effective design documents, and build alignment around technical decisions.
- Strong judgment when balancing reliability, infrastructure costs, and developer productivity.
Nice to have:
- Experience operating LLM gateways or integrating LLM provider APIs at scale, including streaming, token-based rate limiting, model routing, and caching.
- Experience hosting and serving models using technologies such as vLLM, or operating distributed inference and training workloads with Ray, including GPU scheduling, batching, autoscaling, and cost-per-token optimization.
- Experience designing, developing, and maintaining Kubernetes controllers and operators using tools such as controller-runtime or Kubebuilder, including custom resource definitions, reconciliation loops, and safe upgrades.
- Familiarity with cost attribution, budgeting, or FinOps practices for usage-based services.
- Experience working in security-sensitive or regulated environments, such as financial services or cryptocurrency.
- Contributions to open-source projects involving gateways, proxies, or observability.
Benefits
- Remote work: Opportunity to work remotely from Brazil as part of an internationally distributed organization.
- High-impact engineering: Help build and operate critical infrastructure supporting AI-enabled software development.
- Technical ownership: Influence platform architecture, reliability strategy, and engineering standards at the staff level.
- Cross-functional collaboration: Work with engineering, security, finance, and AI governance teams on complex technical challenges.
- Professional growth: Deepen expertise in cloud-native infrastructure, LLM platforms, distributed systems, and AI operations.
- Technical leadership: Mentor engineers and help shape best practices across a growing developer platform.
- Global environment: Collaborate with colleagues and stakeholders across international teams.
How Jobgether works:
We use an AI-powered matching process to ensure your application is reviewed quickly, objectively, and fairly against the role's core requirements. Our system identifies the top-fitting candidates, and this shortlist is then shared directly with the hiring company. The final decision and next steps (interviews, assessments) are managed by their internal team.
We appreciate your interest and wish you the best!
Why Apply Through Jobgether?
Data Privacy Notice: By submitting your application, you acknowledge that Jobgether will process your personal data to evaluate your candidacy and share relevant information with the hiring employer. This processing is based on legitimate interest and pre-contractual measures under applicable data protection laws (including GDPR). You may exercise your rights (access, rectification, erasure, objection) at any time.