Join Sarvam's core API Platform team as a Software Engineer (SDE 2) and shape the future of AI product development in India. You will be instrumental in building and enhancing our high-performance APIs that serve foundational models across LLM, ASR, TTS, Vision, and Translate. Our platform, currently being rewritten in Go, handles everything from request routing and streaming to failover, metering, and billing, ensuring developers can seamlessly ship innovative products.

This is an individual contributor role focused on ownership, reliability, and performance for a multi-tenant platform. You'll work with cutting-edge technologies and contribute to AI solutions with population-scale impact.

Key responsibilities include evolving the LLM API platform from request inception to response delivery. You will contribute to the Python to Go rewrite, developing new services in Go while maintaining the stability of the existing Python stack. Building robust streaming infrastructure for SSE, managing long-lived connections, and ensuring clean handling of cancellations are crucial.

You'll also focus on developing sophisticated routing and failover mechanisms, ensuring protocol compatibility with the OpenAI spec, and engineering the commercial layer for metering, billing, and rate-limiting. Maintaining high performance and reliability for complex agentic and coding workloads, implementing comprehensive observability solutions, and conducting integration and load testing are also key aspects of this role. Close collaboration with Inference and MLOps teams for new model launches and adhering to a design-first, documentation-first engineering culture are expected.

We are seeking candidates with at least 3 years of experience building production backend systems. Proficiency in Python, specifically with asyncio and frameworks like FastAPI, and experience operating services under significant load are essential. Familiarity with Go, or a strong eagerness to learn it, is required as you will work with both languages during the platform rewrite.

Experience in developing low-latency, multi-tenant backend services and hands-on experience with streaming systems such as SSE or WebSockets are necessary. Strong fundamentals in PostgreSQL and Redis, including schema design and query optimization, are important. Comfort operating services on Kubernetes in production is expected. An ownership mindset, including robust testing practices, production debugging using telemetry, and post-incident analysis, is highly valued.

Bonus points include experience serving or integrating LLMs, familiarity with inference servers, and building metering or billing infrastructure. Experience in fast-paced startup environments, comfortable with ambiguity and rapid development, is also advantageous.