Technical Lead-Data Engg
Birlasoft
Birlasoft
This role focuses on designing and implementing advanced Generative AI and Agentic AI solutions. You will leverage cutting-edge Databricks technologies and techniques like prompt engineering and Retrieval-Augmented Generation (RAG) to build innovative AI-powered use cases.
The position requires expertise in creating robust and scalable data pipelines, ensuring data quality, and supporting AI/ML workloads with high-quality datasets. You will also be instrumental in developing enterprise ontologies and semantic data models.
Design and deploy GenAI and Agentic AI solutions utilizing Databricks Genie, Agent Bricks, and Mosaic AI. Build and deploy LLM-powered applications and use cases. Develop and optimize scalable data pipelines using Databricks, PySpark, and SQL. Implement Delta Lake, Unity Catalog, and the medallion architecture for data management. Enhance Spark job performance, cost-efficiency, and scalability. Manage data ingestion, transformation, and enrichment from diverse sources. Maintain data quality, reliability, and observability throughout data pipelines. Architect enterprise ontologies, taxonomies, and semantic data models. Prepare feature-ready datasets to support AI/ML operations. Develop and fine-tune LLMs and ML models with MLflow and Mosaic AI. Implement prompt engineering, model evaluation, and safety guardrails for GenAI initiatives. Integrate AI solutions seamlessly with existing enterprise data and business processes.
A strong background in designing and implementing GenAI and Agentic AI solutions is essential. Proven experience in building LLM-powered use cases and applications. Proficiency in developing scalable data pipelines with Databricks, PySpark, and SQL. Solid understanding of Delta Lake, Unity Catalog, and medallion architecture. Expertise in optimizing Spark jobs for performance, cost, and scalability. Experience with data ingestion, transformation, and enrichment from multiple sources. Commitment to ensuring data quality, reliability, and observability. Skills in designing enterprise ontologies, taxonomies, vocabularies, and semantic data models. Ability to prepare high-quality, feature-ready datasets for AI/ML workloads. Experience in developing and fine-tuning LLMs/ML models using Databricks MLflow and Mosaic AI. Proficiency in prompt engineering, model evaluation, and implementing guardrails for enterprise GenAI use cases. Capability to integrate AI solutions with enterprise data and business workflows.
Birlasoft
IT Consulting