Databricks AI Architect

Persistent

11–16 yrs Pune Full Time Hybrid (office + remote)
Persistent logo
Posted : today
Actively hiring

Job description

Join Persistent as a Databricks AI Architect, where you'll spearhead the creation of enterprise-scale Artificial Intelligence, Machine Learning, and Generative AI solutions. This role demands profound expertise in Databricks, Python, PySpark, Machine Learning, and contemporary AI architectures. You will architect robust AI platforms, drive innovation with Generative AI and Agentic AI, and deliver impactful AI solutions tailored to business needs, utilizing vast data ecosystems.

This is a full-time position within our Data Intelligence and Consumer department, offering a chance to work with industry leaders and contribute to cutting-edge digital engineering and modernization initiatives.

Responsibilities

Architect, develop, and deploy end-to-end ML and Generative AI solutions on Databricks using Python and PySpark. Perform EDA to uncover data patterns, trends, and quality insights. Build, train, evaluate, and deploy ML models for predictive and prescriptive analytics. Design scalable feature engineering, data preparation, and model training pipelines. Implement and manage ML lifecycle processes with MLflow, including tracking, registry, versioning, and deployment. Architect RAG solutions integrating enterprise data, vector databases, embeddings, and LLMs. Develop Generative AI applications like intelligent assistants and chatbots. Design scalable, secure, reliable, and operationally excellent enterprise AI architectures. Lead architecture discussions and establish best practices for AI/ML and data platforms. Evaluate AI models for performance, explainability, and business impact. Collaborate with stakeholders to translate requirements into technical AI solutions. Drive AI governance, model management, responsible AI, and deployment standards. Support data engineering initiatives within the Databricks ecosystem. Contribute to Agentic AI initiatives for autonomous workflows. Mentor engineering and data science teams on AI architecture and best practices. Stay abreast of advancements in AI, ML, LLMs, MLOps, and cloud-native AI.

Qualifications

Possess 11+ years of experience in Data Science, ML, AI Engineering, or Data Platform Architecture. Demonstrate strong hands-on expertise with Databricks and its Lakehouse Platform. Exhibit deep proficiency in Python and PySpark development. Show a solid understanding of ML algorithms, model evaluation, and statistical analysis. Proven experience building and deploying enterprise-grade ML solutions. Hands-on experience with EDA and feature engineering on large datasets. Strong grasp of data engineering concepts and distributed data processing. Experience with MLflow for experiment and lifecycle management. Familiarity with MLOps principles and production AI operations. Experience designing and implementing Data Lakehouse architectures. Proficient in SQL and data manipulation. Hands-on experience designing and developing Generative AI solutions. Solid knowledge of LLMs and enterprise AI architectures. Experience implementing RAG frameworks and knowledge retrieval systems. Understanding of vector databases, embeddings, and semantic search. Experience integrating AI services (e.g., OpenAI, Azure OpenAI). Understanding of Agentic AI concepts and autonomous workflows. Practical implementation experience or strong theoretical knowledge is valued. Experience with prompt engineering, optimization, and LLM evaluation frameworks. Ability to define technical roadmaps, architecture standards, and strategies. Strong stakeholder management and solution design capabilities. Experience leading cross-functional teams. Excellent communication, presentation, and mentoring skills.

Essential Skills

DatabricksPythonPySparkMachine LearningGenerative AIAI ArchitectMLOpsData EngineeringLarge Language Models (LLMs)Retrieval-Augmented Generation (RAG)Vector DatabasesEmbeddingsPrompt EngineeringSQL

Highlights

  • Actively hiring

More Details

RoleDatabricks AI Architect
IndustryInformation Technology & Services
DepartmentEngineering, Information Technology
Employment TypeFull Time, Hybrid (office + remote)

About the Company

Persistent Systems logo

Persistent Systems

Information Technology & Services

Databricks AI Architect at Persistent | SkillMX | SkillMX