Join NVIDIA's LocalAI team as a Senior Systems Software Engineer, pioneering efficient on-device AI for RTX and DGX systems. This role is central to delivering high-performance, low-latency AI inference, optimizing memory usage, and building robust infrastructure for resource-constrained platforms. You'll be at the forefront of the shift towards client-side AI, enhancing real-time processing and privacy.

This position offers a unique opportunity to shape the future of AI on personal devices, working with cutting-edge hardware and software to accelerate AI adoption and innovation.

Collaborate with NVIDIA's cross-functional teams to align technical strategies and foster the AI ecosystem on RTX and DGX PCs.

Develop and optimize the local AI inference stack for various NVIDIA GPUs, prioritizing performance, stability, and scalability.

Design and implement modern inference runtimes and execution stacks using frameworks like llama.cpp, vLLM, PyTorch, and TensorRT to support diverse AI workloads.

Execute end-to-end optimization of AI models, data pipelines, and inference runtimes for next-generation GPU architectures.

Conduct system-level debugging, performance tuning, and accuracy analysis, establishing guidelines for efficient model bring-up and production readiness.

A Bachelor's, Master's, or PhD in Computer Science, Software Engineering, Mathematics, or a related field, coupled with 5+ years of relevant experience, is required.

Demonstrate exceptional C++ programming and debugging expertise, with a solid understanding of fundamental data structures, algorithms, and machine learning principles.

Possess proven experience in developing and optimizing AI inference pipelines using ML/DL frameworks such as Llama.cpp, vLLM, PyTorch, and TensorRT.

Exhibit a deep comprehension of inference backend internals, including scheduling, memory management, and hardware-aware optimization techniques.

Strong analytical abilities, effective problem-solving skills, and the capacity to manage multiple priorities in a dynamic environment are essential. Excellent communication skills are vital for seamless collaboration across teams.