Software Engineering Manager, AI/ML Infrastructure and Performance Engineering

Google

8+ yrs Bengaluru Full Time Hybrid (office + remote)
Google logo
Posted : today
Actively hiring

Job description

Lead AI/ML Infrastructure and Performance Engineering initiatives at Google. This role involves not only technical leadership in major projects but also managing a team of engineers. You will optimize code, empower your team to optimize theirs, contribute to product strategy, and foster team development. Our teams operate across diverse areas including AI, NLP, distributed computing, and large-scale system design.

As a manager, you will guide engineers across multiple teams and locations, oversee significant project budgets, and manage international deployments. Your expertise will drive continuous improvements in machine learning software and hardware stacks by providing insightful performance debugging for workloads and custom kernels. Gain a deep understanding of data collection, analysis, and visualization workflows across frameworks, XLA, and the runtime stack.

Responsibilities

Provide insightful performance debugging for machine learning software/hardware stacks, summarizing profile data from trace timelines, memory usage, compiler profiles, and ML graph summaries. Contribute to the end-to-end stack and analysis tools to support new ML paradigms like horizontal scaling for upcoming TPU chips. Partner with ML Stack leads to understand model optimization use cases and integrate debugging into 3P environments such as VSCode, Cursor, and Grafana. Collaborate with OSS ML inference frameworks like vLLM and SGLang to identify performance improvement opportunities. Work with teams owning various ML stack components to understand performance optimization needs, and utilize frameworks like TorchTPU to identify performance bottlenecks.

Qualifications

Requires a Bachelor's degree or equivalent practical experience, along with 8 years of experience in software engineering, machine learning infrastructure, computer architecture, distributed computing, people management, and debugging tools. Essential experience includes people management and building infrastructure to enhance the performance of ML systems and applications.

Preferred qualifications include expertise in Performance Optimization, GPU Programming, High Performance Computing, Large Language Models, and open-source contribution. Familiarity with ML infra, frameworks, and GPU/TPU performance analysis is advantageous. Experience in building agentic workflows for performance debugging and optimization, alongside generating innovative ideas and resolving ambiguity, is highly valued. Proficiency with ML frameworks like TensorFlow, JAX, PyTorch, and Keras, as well as ML inference frameworks such as vLLM, SG Lang, and Pathways, is a plus. Experience in open-source software development, including releasing and supporting projects, is also beneficial.

Essential Skills

Software EngineeringMachine Learning InfrastructureComputer ArchitectureDistributed ComputingPeople ManagementDebugging ToolsCommunicationPerformance DebuggingML Stack AnalysisFrameworksAccelerated Linear Algebra (XLA)Runtime Stack AnalysisML Inference Frameworks (vLLMSGLang)TorchTPU

Good to Have

Performance OptimizationGPU ProgrammingHigh Performance ComputingLarge Language ModelsOpen Source ContributionML InfrastructureGPU Performance AnalysisTPU Performance AnalysisAgentic WorkflowsTensorFlowJAXPyTorchKerasSG LangPathwaysOpen Source Software Development

Highlights

  • Actively hiring

More Details

RoleSoftware Engineering Manager, AI/ML Infrastructure and Performance Engineering
DepartmentAI / Machine Learning
Employment TypeFull Time, Hybrid (office + remote)

About the Company

Google logo

Google

IT Consulting

Software Engineering Manager, AI/ML Infrastructure and Performance Engineering at Google | SkillMX | SkillMX