Databricks AI & Data Engineer

NTT DATA

8+ yrs Hyderabad Full Time Hybrid (office + remote)
NTT DATA logo
Posted : today
Actively hiring

Job description

This role focuses on designing, developing, and deploying Agentic AI solutions to automate critical data management and governance processes across enterprise platforms. You will leverage cutting-edge technologies like LLMs and RAG to build scalable AI agents that enhance data quality, enforce policies, and streamline stewardship workflows. Collaboration with various engineering teams and business stakeholders is key to identifying governance use cases and aligning solutions with enterprise data strategy.

Key responsibilities include building autonomous and human-in-the-loop agents for data cataloging, glossary management, policy compliance, and issue remediation. You will also integrate these agents with diverse enterprise platforms and governance technologies. The role involves creating proof-of-value demonstrations, managing development backlogs, and ensuring continuous refinement of governance controls. Additionally, implementing monitoring and guardrails for AI solutions to ensure responsible AI practices is essential.

Responsibilities

Design and implement Agentic AI solutions for data governance automation, including data quality, lineage, and policy enforcement. Build scalable AI agents using LLMs, RAG, and orchestration frameworks. Collaborate with stakeholders to define agent capabilities and align solutions with enterprise data strategy. Develop and maintain autonomous and human-in-the-loop agents for various data management tasks. Integrate AI agents with enterprise platforms and governance tools. Create and present proof-of-value demonstrations and pilot implementations. Manage governance backlogs and prioritize agent development initiatives. Implement monitoring, observability, and guardrails for AI solutions. Support proposal development and client presentations for data governance modernization. Contribute to reusable frameworks and best practices for Agentic AI implementations.

Qualifications

Requires a minimum of 8 years of experience in Data Management, Data Engineering, Data Governance, Metadata Management, or Enterprise Data Architecture. A minimum of 5 years is needed for designing and implementing data governance, MDM, metadata management, data quality, or regulatory compliance solutions. Additionally, 3+ years of experience in developing AI/ML, Generative AI, or Agentic AI solutions using LLMs and orchestration frameworks is necessary.

Proficiency in Python and modern AI development frameworks such as LangChain, LangGraph, Semantic Kernel, AutoGen, CrewAI, OpenAI, Azure AI Foundry, or Amazon Bedrock is required. You should also have 3+ years of hands-on experience with cloud platforms like Azure, AWS, or Google Cloud Platform. Experience integrating governance platforms like Collibra or Databricks Unity Catalog (2+ years), and designing APIs, workflow orchestration, or event-driven architectures (2+ years) are also essential.

Experience defining governance policies, lineage workflows, data quality controls, stewardship processes, and compliance monitoring capabilities is expected. Working in Agile delivery environments with backlog-driven development is crucial. Strong client-facing communication skills are vital for leading workshops, requirements gathering, and executive demonstrations.

Essential Skills

Data ManagementData EngineeringData GovernanceMetadata ManagementEnterprise Data ArchitectureMaster Data Management (MDM)Data QualityRegulatory ComplianceAI/MLGenerative AIAgentic AILLMsOrchestration FrameworksPythonLangChainLangGraphSemantic KernelAutoGenCrewAIOpenAIAzure AI FoundryAmazon BedrockAzureAWSGoogle Cloud PlatformCollibraDatabricks Unity CatalogAPIsWorkflow OrchestrationEvent-Driven ArchitecturesEnterprise System IntegrationsAgile DeliveryClient-Facing Communication

Good to Have

AtaccamaInformaticaMicrosoft PurviewSnowflake

Highlights

  • Actively hiring

More Details

RoleDatabricks AI & Data Engineer
DepartmentData Engineering, AI / Machine Learning
Employment TypeFull Time, Hybrid (office + remote)

About the Company

NTT DATA logo

NTT DATA

IT Consulting

Databricks AI & Data Engineer at NTT DATA | SkillMX | SkillMX