NTT DATA seeks talented individuals eager to innovate and grow within a forward-thinking organization. Join our inclusive and adaptable team as a Data Engineer Senior Consultant in Bangalore, India.

This role focuses on transforming existing data pipelines and migrating ETL workloads to the Databricks Lakehouse. You will leverage Databricks, Spark, and Python to develop robust data solutions, implement layered data architectures, and ensure efficient data processing. Collaboration with senior team members and adherence to development standards are key to success.

We are looking for professionals with a solid foundation in data engineering and ETL, proficient in Databricks and relevant technologies. A degree in Computer Science, IT, Engineering, or equivalent experience is required.

Analyze current Informatica and AWS Glue processes to understand data flows and transformations. Develop Databricks notebooks and pipelines to migrate existing ETL workloads. Create data transformations using Python, PySpark, Spark SQL, and Databricks SQL. Implement Bronze, Silver, and Gold data layers following project architecture and coding standards. Build batch and incremental processing pipelines utilizing Delta Lake and control mechanisms. Develop and maintain Databricks Jobs/Workflows and Lakeflow Declarative Pipelines. Integrate common project frameworks for logging, auditing, data quality, error handling, and notifications. Convert Informatica transformations and business logic into equivalent Databricks implementations. Support the migration of AWS Glue jobs and database processing into Databricks. Conduct unit testing and source-to-target data reconciliation for migrated pipelines. Investigate and resolve data, SQL, PySpark, and pipeline execution issues. Assist with integration testing, UAT, and production validation activities.

Requires 3-6 years of experience in data engineering, ETL development, or application/data platform development. Proficiency with Databricks and Apache Spark is essential. Strong development skills in Python/PySpark and SQL are required. Experience with Databricks notebooks, Spark SQL, Databricks SQL, Delta Lake, and Databricks Jobs/Workflows. Proven experience developing batch or incremental ETL/ELT pipelines. Familiarity with ETL technologies such as Informatica PowerCenter, AWS Glue, or SSIS. Experience working with relational databases and SQL-based data processing. Understanding of data transformation, source-to-target mapping, and ETL development principles. Experience with unit testing, data validation, and troubleshooting data issues. Basic knowledge of layered data architectures (Bronze/Silver/Gold). Ability to adhere to development frameworks and coding standards. Possess strong analytical, problem-solving, and communication skills. A Bachelor's degree in Computer Science, Information Technology, Engineering, or equivalent experience is necessary.