Databricks AI & Data Engineer

NTT DATA

8+ yrs Hyderabad Full Time Hybrid (office + remote)
NTT DATA logo
Posted : today
Actively hiring

Job description

Join our innovative team as an AI & Data Engineer, focusing on the design and deployment of advanced Agentic AI solutions. This role is pivotal in automating critical data governance processes across enterprise data platforms. You will leverage cutting-edge technologies like LLMs and RAG to build scalable AI agents that enhance data quality, metadata management, and regulatory compliance.

Collaborate closely with cross-functional teams, including Data Governance Engineers, Data Engineers, and Architects, to identify governance challenges and architect effective AI-driven solutions. Your work will directly impact the efficiency and robustness of our data management strategies, ensuring alignment with enterprise data goals.

Responsibilities

Architect and implement Agentic AI solutions for automated data governance, quality, and metadata management.

Develop scalable AI agents utilizing LLMs, RAG, and orchestration frameworks for data management and compliance.

Partner with stakeholders to define agent capabilities and align solutions with enterprise data strategy.

Build and maintain autonomous and human-in-the-loop agents for data cataloging, glossary management, and issue remediation.

Integrate AI agents with diverse enterprise platforms including data catalogs, MDM, and cloud data solutions.

Create proof-of-value demonstrations and pilot implementations to showcase AI governance automation.

Continuously refine governance backlogs and prioritize agent development initiatives.

Implement monitoring, observability, and guardrails for AI solutions to ensure responsible AI practices.

Support proposal development and client presentations for data and AI governance modernization.

Qualifications

A strong foundation in Data Management, Data Engineering, Data Governance, or Enterprise Data Architecture with a minimum of 8 years of experience is essential.

Demonstrated expertise in designing and implementing enterprise data governance, MDM, metadata management, or data quality solutions over 5+ years is required.

Develop AI/ML, Generative AI, or Agentic AI solutions using LLMs and orchestration frameworks for at least 3 years.

Possess 3+ years of hands-on experience with Python and modern AI development frameworks such as LangChain, LangGraph, or OpenAI.

Significant experience (3+ years) working with major cloud platforms like Azure, AWS, or Google Cloud Platform.

Familiarity with integrating governance platforms like Collibra or Databricks Unity Catalog, with a minimum of 2 years of experience.

Experience in designing APIs, workflow orchestration, or event-driven architectures (2+ years).

Proficiency in defining governance policies, lineage workflows, and data quality controls.

Experience in Agile delivery environments and strong client-facing communication skills are vital for this role.

Essential Skills

Data GovernanceData QualityMetadata ManagementLineage AnalysisPolicy EnforcementStewardship WorkflowsLarge Language Models (LLMs)Retrieval Augmented Generation (RAG)Orchestration FrameworksEnterprise Governance ToolsData ManagementRegulatory ComplianceData CatalogingBusiness Glossary ManagementData Ownership AssignmentIssue RemediationAtaccamaCollibraInformaticaMicrosoft PurviewDatabricksSnowflakeAWSAzurePythonLangChainLangGraphSemantic KernelAutoGenCrewAIOpenAIAzure AI FoundryAmazon BedrockGoogle Cloud PlatformDatabricks Unity CatalogAPIsWorkflow OrchestrationEvent-Driven ArchitecturesEnterprise System IntegrationsAgile DeliveryClient-Facing CommunicationWorkshop FacilitationRequirements GatheringArchitecture ReviewsExecutive DemonstrationsAI/MLGenerative AIAgentic AI

Highlights

  • Actively hiring

More Details

RoleDatabricks AI & Data Engineer
DepartmentData Engineering, AI / Machine Learning
Employment TypeFull Time, Hybrid (office + remote)

About the Company

NTT DATA logo

NTT DATA

IT Consulting

Databricks AI & Data Engineer at NTT DATA | SkillMX | SkillMX