Software Engineer - Search & Vector Database Infrastructure
- Compensation
- Not disclosed
- Location
- San Jose
- Work setup
- Not specified
- Seniority
- Not specified
- Employment
- Not specified
- Experience
- Not specified
- Category
- Software engineering
- Posted
Skills
Must have
- B.S in Computer Science, AI, Machine Learning, or a related field.
- Strong software engineering skills in C++, Golang, Java, or Python.
- Experience with search engines, databases, storage systems, or distributed systems.
- Hands-on experience with Elasticsearch/OpenSearch or Milvus in production or system development.
- Strong understanding of distributed systems and performance optimization.
Nice to have
- Deep experience with Elasticsearch/OpenSearch, Lucene, Milvus/Knowhere, or similar systems.
- Strong knowledge of full-text search and (multi)-vector indexing, including HNSW, IVF, DiskANN, or related ANN algorithms.
- Experience optimizing billion-scale or larger vector retrieval for latency, throughput, recall, and cost.
- Experience with cloud-native storage, compute-storage separation, object storage, or lakehouse/lakebase architectures.
Full description
About the Team:
The Search team is dedicated to building a fully managed, one-stop platform for information retrieval and analytics. Deeply compatible with Elasticsearch, OpenSearch, and Milvus, the platform serves four core scenarios—GenAI applications, search, security, and observability—and provides comprehensive capabilities across full-text search, vector search, geospatial search, and hybrid search. We are also building next-generation Agentic Search products, including RAG, AI-powered search, and multimodal retrieval, continuously pushing the boundaries of intelligent search technology.
We are deeply involved in the open-source ecosystem and have contributed hundreds of PRs across the OpenSearch, Elasticsearch, Lucene, and Milvus communities. Several members of our team hold key leadership roles in these communities, including Governing Board Member, Technical Steering Committee Member, PMC Member, and Maintainer, helping shape the evolution of the broader search and database ecosystem through sustained technical contributions.
On the business side, we not only provide reliable infrastructure for some of ByteDance’s largest-scale vector and full-text hybrid search workloads, but also deliver enterprise-grade cloud search services to external customers through Volcano Engine, helping businesses unlock growth through data. We embrace a pragmatic yet ambitious engineering culture: while continuously publishing innovative research at top-tier conferences such as VLDB and ICDE, we remain strongly focused on real-world impact and commercial adoption, building a comprehensive product portfolio that serves a wide range of search and AI use cases.
Our team brings together leading database researchers and exceptional engineers from around the world. We are looking for talented individuals with expertise in search or vector database engine development, distributed systems architecture, or AI model innovation to join us. Together, we will explore uncharted technical territory, define the data foundation for the AI era, and empower both ByteDance and enterprises across industries to achieve AI-driven transformation.
Responsibilities
- Develop and evolve our Elasticsearch/OpenSearch and Milvus products, building high-performance and scalable search and vector retrieval capabilities.
- Develop high-performance full-text and vector search, including hybrid search, filtered search, multi-vector search, and large-scale ANN retrieval.
- Optimize vector indexing and distributed retrieval across HNSW, IVF, DiskANN, RaBitQ, storage, caching, and I/O.
- Explore cloud-native Vector Lakebase architectures for scalable search over data lakes and object storage.
Minimum Qualifications
- B.S in Computer Science, AI, Machine Learning, or a related field.
- Strong software engineering skills in C++, Golang, Java, or Python.