Databricks -Technical Specialist-Data Engg
Birlasoft
Birlasoft
Advance your career as a Technical Specialist focusing on Data Engineering at Databricks. This role involves designing and implementing cutting-edge GenAI and Agentic AI solutions.
You will leverage Databricks Genie, Agent Bricks, and Mosaic AI for model training, evaluation, prompt engineering, and Retrieval-Augmented Generation (RAG).
This is an exciting opportunity to build LLM-powered use cases and contribute to the future of AI.
Key responsibilities include designing and building scalable data pipelines using Databricks, PySpark, and SQL, while implementing Delta Lake, Unity Catalog, and the medallion architecture. You will optimize Spark jobs for performance, cost, and scalability, and handle data ingestion, transformation, and enrichment from diverse sources.
Further duties involve ensuring data quality, reliability, and observability across all pipelines. You will also design and develop enterprise ontologies, taxonomies, vocabularies, and semantic data models to support AI/ML workloads with high-quality, feature-ready datasets.
This role requires hands-on experience in developing and fine-tuning LLMs/ML models using Databricks MLflow and Mosaic AI. You will implement prompt engineering, model evaluation, and guardrails for enterprise GenAI use cases, and integrate AI solutions seamlessly with enterprise data and business workflows.
While specific degree requirements are not listed, a strong background in data engineering principles and practical experience with the listed technologies is essential.
Birlasoft
Technology