Sr Software Engineer, Platform
Senior Software Engineer building the Kafka-compatible service on StreamNative Cloud, a cloud-native real-time data streaming platform blending Apache Kafka compatibility with modern storage (URSA engine). Day to day: designing distributed streaming/storage components, profiling and optimizing performance, debugging large-scale production issues, and contributing to open source, primarily in Java/
- Compensation
- Not disclosed
- Location
- Remote, West Coast
- Work setup
- Remote
- Seniority
- Senior
- Employment
- Full time
- Experience
- 5+ years
- Category
- Software engineering
- Posted
Skills
Must have
- 5+ years building large-scale distributed systems, backend infrastructure, or cloud services
- Strong understanding of Apache Kafka, Apache Pulsar, or other distributed messaging systems
- Experience with distributed systems concepts: replication, consistency, fault tolerance, scalability, distributed storage
- Experience with Kafka internals: protocol, broker architecture, KRaft, tiered storage, log storage, compaction
- Experience building cloud-native streaming or storage systems
- Strong programming skills in Java, Scala, or other systems programming languages
- Experience designing and operating reliable production systems
- Strong debugging and performance optimization skills
- Ability to independently own projects from design through production deployment
- Good communication and cross-functional collaboration skills
Nice to have
- Familiarity with object storage such as S3, GCS, or Azure Blob Storage
- Experience with lakehouse technologies such as Apache Iceberg or Parquet
- Experience with Kubernetes and cloud-native infrastructure
Full description
StreamNative, founded by the creators of Apache Pulsar, is redefining real-time data streaming. Our platform empowers organizations to process and analyze massive data streams at scale with unparalleled efficiency. At its core is the URSA engine, which seamlessly integrates Apache Pulsar and Apache Kafka, delivering unmatched compatibility and performance. By simplifying deployments and enabling direct integration with modern lakehouse architectures, URSA helps businesses innovate faster and more effectively. With our cost-efficient solutions, companies can build cutting-edge, real-time applications that drive measurable outcomes. Join us in revolutionizing data streaming and enabling organizations to thrive in a fast-paced, data-driven world.
About the Role
We are looking for a Senior Software Engineer to help build the Kafka service on StreamNative Cloud, a next-generation cloud-native streaming platform that combines Apache Kafka compatibility with modern storage technologies.
You will work on challenging distributed systems problems, including streaming infrastructure, storage scalability, reliability, and performance optimization. You will design and implement critical components of our streaming platform and help build reliable infrastructure that powers real-time data applications for developers and enterprises worldwide.
This is a high-impact engineering role for engineers who are passionate about Apache Kafka, distributed systems, and building scalable data infrastructure. You will join the Platform Engineering team, which builds the messaging and streaming infrastructure by combining Kafka compatibility with modern cloud-native storage technologies.
Responsibilities
- Design, implement, and maintain core components of the streaming platform — including Kafka-compatible APIs, storage systems, metadata management, and distributed services.
- Solve complex engineering problems involving distributed systems, storage, networking, and performance optimization, and continuously improve reliability through profiling, benchmarking, and testing.
- Debug and resolve production issues in large-scale distributed environments.
- Work closely with customers and field teams to understand real-world streaming use cases and help deliver effective technical solutions.
- Contribute to open-source projects and improve the developer experience of streaming platforms.
- On-call: one week per month, rotating, supporting customer tickets.
Qualifications
- 5+ years of experience building large-scale distributed systems, backend infrastructure, or cloud services.
- Strong understanding of Apache Kafka, Apache Pulsar, or other distributed messaging systems.