Data Scientist- Databricks Genie
Birlasoft
Birlasoft
This role focuses on designing and implementing cutting-edge Generative AI and Agentic AI solutions, leveraging the Databricks platform. You will be instrumental in building LLM-powered use cases and ensuring scalable, high-quality data pipelines for advanced analytics and AI workloads.
The position requires a deep understanding of AI/ML model development, data engineering best practices, and the ability to integrate complex AI solutions into enterprise environments. You'll play a key part in shaping the future of AI within the organization.
- Develop and deploy GenAI and Agentic AI solutions using Databricks Genie and Mosaic AI, including model training, prompt engineering, and RAG. - Build LLM-powered applications and create robust, scalable data pipelines with PySpark and SQL on Databricks. - Implement data governance using Delta Lake, Unity Catalog, and Medallion architecture, while optimizing Spark jobs for efficiency. - Manage data ingestion, transformation, and enrichment from diverse sources, ensuring data quality and reliability. - Design enterprise ontologies and semantic data models to support AI/ML initiatives and prepare high-quality datasets for feature engineering. - Fine-tune LLMs and ML models using Databricks MLflow and Mosaic AI, implementing prompt engineering, evaluation, and guardrails.
Proven experience in designing and implementing GenAI and Agentic AI solutions, with expertise in Databricks Genie, Mosaic AI, and RAG techniques. Proficiency in building LLM-powered use cases and developing scalable data pipelines using Databricks, PySpark, and SQL. Strong knowledge of Delta Lake, Unity Catalog, and Medallion architecture, along with experience in optimizing Spark jobs. Ability to perform comprehensive data ingestion, transformation, and enrichment, ensuring data quality and reliability. Experience in designing enterprise ontologies, taxonomies, and semantic data models, and supporting AI/ML workloads with feature-ready datasets. Expertise in developing and fine-tuning LLMs/ML models with Databricks MLflow and Mosaic AI, including prompt engineering and model evaluation.
Birlasoft
Data Analytics