Technical Lead-Data Engg

Birlasoft

Fresher Noida Full Time Work from office
Birlasoft logo
Posted : 1 week ago
Actively hiring

Job description

This role focuses on designing and implementing advanced Generative AI and Agentic AI solutions. You will leverage cutting-edge Databricks technologies and techniques like prompt engineering and Retrieval-Augmented Generation (RAG) to build innovative AI-powered use cases.

The position requires expertise in creating robust and scalable data pipelines, ensuring data quality, and supporting AI/ML workloads with high-quality datasets. You will also be instrumental in developing enterprise ontologies and semantic data models.

Responsibilities

Design and deploy GenAI and Agentic AI solutions utilizing Databricks Genie, Agent Bricks, and Mosaic AI. Build and deploy LLM-powered applications and use cases. Develop and optimize scalable data pipelines using Databricks, PySpark, and SQL. Implement Delta Lake, Unity Catalog, and the medallion architecture for data management. Enhance Spark job performance, cost-efficiency, and scalability. Manage data ingestion, transformation, and enrichment from diverse sources. Maintain data quality, reliability, and observability throughout data pipelines. Architect enterprise ontologies, taxonomies, and semantic data models. Prepare feature-ready datasets to support AI/ML operations. Develop and fine-tune LLMs and ML models with MLflow and Mosaic AI. Implement prompt engineering, model evaluation, and safety guardrails for GenAI initiatives. Integrate AI solutions seamlessly with existing enterprise data and business processes.

Qualifications

A strong background in designing and implementing GenAI and Agentic AI solutions is essential. Proven experience in building LLM-powered use cases and applications. Proficiency in developing scalable data pipelines with Databricks, PySpark, and SQL. Solid understanding of Delta Lake, Unity Catalog, and medallion architecture. Expertise in optimizing Spark jobs for performance, cost, and scalability. Experience with data ingestion, transformation, and enrichment from multiple sources. Commitment to ensuring data quality, reliability, and observability. Skills in designing enterprise ontologies, taxonomies, vocabularies, and semantic data models. Ability to prepare high-quality, feature-ready datasets for AI/ML workloads. Experience in developing and fine-tuning LLMs/ML models using Databricks MLflow and Mosaic AI. Proficiency in prompt engineering, model evaluation, and implementing guardrails for enterprise GenAI use cases. Capability to integrate AI solutions with enterprise data and business workflows.

Essential Skills

DatabricksGenAIAgentic AILLMPySparkSQLDelta LakeUnity CatalogMedallion ArchitectureSparkData IngestionData TransformationData QualityOntologiesTaxonomiesSemantic Data ModelsAI/ML WorkloadsMLflowPrompt EngineeringModel EvaluationGuardrailsRAG

Highlights

  • Actively hiring

More Details

RoleTechnical Lead-Data Engg
DepartmentData Engineering, AI / Machine Learning
Employment TypeFull Time, Work from office

About the Company

Birlasoft logo

Birlasoft

IT Consulting