Senior AI/LLM Inference Engineer
Persistent
Persistent
Join our dynamic team as a Senior AI/LLM Inference Engineer and drive the optimization of high-performance Large Language Models (LLM) and multimodal inference. You will be instrumental in delivering accurate, portable inference solutions across diverse environments, including CPU, GPU, and edge devices, meeting stringent latency, throughput, memory, power, and quality targets.
We are an AI-led, platform-driven Digital Engineering and Enterprise Modernization partner with a strong track record of helping industry leaders innovate and grow. Our commitment to client success and a culture of innovation empowers our global team to tackle complex challenges and deliver exceptional results. Be part of a company recognized for its leadership in AI and digital transformation.
Evaluate and benchmark foundation models, establishing robust LLM validation workflows. Implement advanced inference techniques like quantization, mixed-precision, calibration, and weight packing. Develop optimized inference paths for various architectures, including ARM NEON, x86 AVX2/SSE, and GPU compute. Integrate custom operators, backend adapters, and runtime dispatch mechanisms. Ensure system integrity through profiling and regression testing.
A strong foundation in Python and modern C++ programming is essential. Deep expertise in LLM inference, Transformers, PyTorch, Hugging Face, quantization, and Retrieval-Augmented Generation (RAG) is required. Proven experience with SIMD, CUDA or GPU compute, Linux, CMake, Ninja, Git, and CI/CD processes is mandatory. Familiarity with performance profiling, debugging in HPC environments, and machine learning frameworks beyond PyTorch is preferred. Knowledge of edge computing considerations for LLM deployment is a plus.
Persistent Systems
IT Consulting