Data Migration Engineer - Nov Expanded Team

NTT DATA

Fresher Bengaluru Full Time Work from office
NTT DATA logo
Posted : today
Actively hiring

Job description

Join our expanding team as a Data Migration Engineer, a pivotal role focused on transforming complex data workflows. You will be instrumental in migrating intricate Informatica processes into modern Python EL scripts and dbt transformation models, ensuring seamless data movement and robust data integrity.

This position offers an exciting opportunity to work with high-volume datasets and cutting-edge technologies in data warehousing and orchestration. You'll contribute directly to the efficiency and scalability of our data infrastructure.

Responsibilities

Translate complex Informatica workflows, including sessions, mappings, and parameterizations, into equivalent Python EL scripts and dbt transformation models. Develop and maintain robust Python EL pipelines for landing high-volume data, utilizing libraries like SQLAlchemy, cx_Oracle, and pyodbc, along with bulk export tools such as Oracle Data Pump and SQL Server BCP. Design and implement dbt transformation models, adhering to best practices for layering, macros, incremental strategies, and snapshot patterns. Create dbt-native tests and custom Python unit tests to ensure transformation accuracy and data quality. Orchestrate end-to-end Python EL and dbt scripts using Airflow DAGs, ensuring functional equivalence to original Informatica workflows. Contribute to GitLab CI/CD pipelines for dbt and Airflow code, encompassing lint gates, automated testing, and deployment. Conduct peer code reviews, offering constructive technical feedback. Troubleshoot performance bottlenecks and data discrepancies during SIT and UAT phases, including row-count reconciliation. Develop comprehensive technical documentation, runbooks, and handover materials.

Qualifications

Proven experience in assessing and translating complex Informatica workflows. Proficiency in developing Python EL pipelines for high-volume data loading using SQLAlchemy, cx_Oracle, pyodbc, Oracle Data Pump, and SQL Server BCP. Strong understanding of dbt transformation modeling best practices, including layering, macros, incremental strategies, and snapshot patterns. Experience with dbt-native testing and custom Python unit testing. Skilled in orchestrating data pipelines with Airflow. Familiarity with GitLab CI/CD for dbt and Airflow code. Ability to troubleshoot performance issues and perform data reconciliation. Excellent communication and collaboration skills for peer reviews and documentation. Experience working in an onsite environment in Bangalore.

Essential Skills

PythondbtSQLAlchemycx_OraclepyodbcOracle Data PumpSQL Server BCPAirflowGitLab CI/CD

Highlights

  • Actively hiring

More Details

RoleData Migration Engineer - Nov Expanded Team
DepartmentData Engineering
Employment TypeFull Time, Work from office

About the Company

NTT DATA logo

NTT DATA

IT Consulting

Data Migration Engineer - Nov Expanded Team at NTT DATA | SkillMX | SkillMX