Data Migration Engineer - Nov Expanded Team
NTT DATA
NTT DATA
Join our expanding team as a Data Migration Engineer, a pivotal role focused on transforming complex data workflows. This position involves translating existing Informatica processes into modern Python EL and dbt models, ensuring efficient and accurate data handling.
We are seeking an engineer adept at building robust data pipelines capable of managing high-volume datasets and ensuring data integrity through rigorous testing and documentation.
Key responsibilities include assessing Informatica workflows and converting them into Python EL scripts and dbt transformation models.
You will develop and maintain Python EL pipelines to load large datasets into target warehouses, utilizing tools like SQLAlchemy and bulk export utilities.
Design and build dbt transformation models, adhering to best practices, and implement dbt-native and Python unit tests for validation.
Orchestrate end-to-end data pipelines using Airflow DAGs and contribute to GitLab CI/CD for code deployment and testing.
Peer code reviews, performance troubleshooting, and contributing to technical documentation are also integral parts of this role.
Candidates should possess strong experience in assessing and translating Informatica workflows into Python EL scripts and dbt models.
Proficiency in developing Python EL pipelines using SQLAlchemy, cx_Oracle, pyodbc, and bulk export tools is essential.
A solid understanding of dbt, including model layering, macros, and incremental strategies, is required.
Experience with Airflow for pipeline orchestration and GitLab CI/CD for DevOps practices is necessary.
Ability to troubleshoot data discrepancies and performance issues, along with contributing to technical documentation, is expected.
NTT DATA
IT Consulting