NTT DATA is seeking a seasoned Data Engineer Senior Consultant to join our dynamic team in Bangalore, India. We are looking for innovative and passionate individuals eager to contribute to our forward-thinking organization. This role offers an excellent opportunity for growth within a global leader in business and technology services.

As part of our commitment to accelerating client success and positively impacting society, you will leverage your expertise in data engineering to drive digital transformation. Join a collaborative environment where your contributions are valued, and you can develop your career alongside industry experts.

Your primary responsibilities will include analyzing existing ETL workflows (Informatica, AWS Glue) and migrating them to Databricks Lakehouse using Databricks notebooks and pipelines. You will develop data transformations using Python, PySpark, and SQL, implementing Bronze, Silver, and Gold layers. This involves creating batch and incremental processing pipelines with Delta Lake, managing Databricks Jobs/Workflows, and reusing common project frameworks for logging and error handling.

You will be instrumental in converting Informatica logic to Databricks, supporting the migration of AWS Glue jobs, and performing unit testing and data reconciliation. Investigating and resolving pipeline execution issues, supporting various testing phases (integration, UAT, production), and collaborating with senior developers are key aspects of this role. Adherence to coding, performance, and security standards, participating in code reviews, and preparing technical documentation are also essential.

The ideal candidate possesses 3-6 years of experience in data engineering or ETL development, with hands-on proficiency in Databricks and Apache Spark. Strong development skills in Python/PySpark and SQL are mandatory, alongside experience with Databricks notebooks, Spark SQL, Delta Lake, and Databricks Jobs/Workflows. Familiarity with ETL technologies like Informatica PowerCenter or AWS Glue, relational databases, and data transformation concepts is required. A Bachelor's degree in Computer Science, IT, Engineering, or equivalent experience is necessary. Excellent analytical, problem-solving, and communication skills are essential for success in this role.