Data Engineer Senior Consultant

NTT DATA

5 yrs Bengaluru Full Time Hybrid (office + remote)
NTT DATA logo
Posted : today
Actively hiring

Job description

Join NTT DATA as a Data Engineer Senior Consultant and contribute to transforming large-scale data migrations. This role focuses on designing and implementing modern data stack solutions, leveraging Python, dbt, and Airflow to achieve functional equivalence with existing Informatica PowerCenter workflows.

As a senior technical contributor, you will be instrumental in delivering production-quality pipelines at an enterprise scale. Your expertise will guide the conversion of complex data processes into efficient, modern equivalents, ensuring seamless data flow and transformation.

NTT DATA is a global leader in business and technology services, committed to driving client success and positive societal impact through responsible innovation. We are building the future of AI and digital infrastructure, offering unparalleled capabilities in enterprise-scale AI, cloud, security, and data services.

Responsibilities

Translate complex Informatica PowerCenter workflows into Python EL scripts and dbt transformation models. Develop and maintain Python EL pipelines for high-volume data loading, utilizing tools like SQLAlchemy and Oracle Data Pump. Design dbt transformation models following best practices, including layering, macros, and incremental strategies. Implement dbt-native and custom Python unit tests for data quality validation.

Orchestrate end-to-end data pipelines using Airflow DAGs, ensuring functional equivalence with source workflows. Contribute to GitLab CI/CD pipelines for dbt and Airflow code, incorporating automated testing and deployment. Conduct peer code reviews and provide technical feedback to engineering teams. Troubleshoot performance issues and data discrepancies during testing phases.

Prepare technical documentation, runbooks, and handover materials. Collaborate with cross-functional teams to ensure successful data migration outcomes. Actively participate in knowledge sharing and continuous improvement initiatives within the team.

Qualifications

A minimum of 5 years of experience with Informatica PowerCenter, including mappings, sessions, workflows, and parameterization. At least 3 years of hands-on experience with dbt Core, covering model layering, macros, tests, and incremental materializations. Proficiency in Airflow for DAG authoring and task dependency management is essential.

Requires 5+ years of robust Python experience, with a focus on production use of parallelism and batch data processing. Strong SQL skills are necessary, including complex joins, window functions, and aggregations on large datasets. Experience with Oracle Data Pump or SQL Server BCP/BULK INSERT for high-volume data movement is also a must-have.

Preferred qualifications include experience with SQLAlchemy, cx_Oracle, or pyodbc for Python-based database connectivity. Experience mentoring junior and mid-level engineers in offshore delivery models is advantageous. This role is suited for experienced professionals ready to tackle challenging data migration projects.

Essential Skills

Informatica PowerCenterPythonSQLdbt CoreAirflowOracle Data PumpSQL Server BCP

Good to Have

SQLAlchemycx_Oraclepyodbc

Highlights

  • Actively hiring

More Details

RoleData Engineer Senior Consultant
Employment TypeFull Time, Hybrid (office + remote)

About the Company

NTT DATA logo

NTT DATA

IT Consulting

Data Engineer Senior Consultant at NTT DATA | SkillMX | SkillMX