Data Platform Modernization Engineer - Junior (Databricks)

NTT DATA

3–6 yrs Bengaluru Full Time Hybrid (office + remote)
NTT DATA logo
Posted : yesterday
Actively hiring

Job description

NTT DATA is seeking an entry-level Data Platform Modernization Engineer specializing in Databricks to join our dynamic team in Bangalore. This role is ideal for passionate individuals eager to innovate and grow within a forward-thinking organization. You will play a key role in transforming existing data workflows into a modern Databricks Lakehouse architecture.

Responsibilities

Your primary responsibilities will include analyzing existing Informatica and AWS Glue ETL processes, developing Databricks notebooks and pipelines using Python, PySpark, and SQL to migrate workloads. You will implement layered data processing (Bronze, Silver, Gold), build batch and incremental pipelines, and manage Databricks Jobs/Workflows. Converting existing transformations, supporting migration of AWS Glue jobs, performing unit testing and data reconciliation, and resolving pipeline execution issues are also crucial. Collaboration with senior developers, adherence to coding standards, participating in code reviews, and preparing technical documentation will be essential.

Qualifications

We are looking for candidates with 3-6 years of data engineering or ETL development experience, including hands-on expertise with Databricks and Apache Spark. Strong proficiency in Python/PySpark and SQL is required, along with experience in Databricks notebooks, Spark SQL, Delta Lake, and Databricks Jobs/Workflows. Familiarity with ETL technologies like Informatica PowerCenter or AWS Glue, relational databases, and data transformation concepts is necessary. A Bachelor's degree in Computer Science, Information Technology, Engineering, or equivalent experience is expected. Excellent analytical, problem-solving, and communication skills are vital for success in this role.

Essential Skills

DatabricksApache SparkPythonPySparkSQLSpark SQLDatabricks SQLDelta LakeDatabricks NotebooksDatabricks Jobs/WorkflowsETL DevelopmentInformatica PowerCenterAWS GlueSSISRelational DatabasesData TransformationSource-to-Target MappingUnit TestingData ValidationTroubleshooting

Good to Have

Databricks CertificationLakeflow Declarative PipelinesDelta Live TablesUnity CatalogDatabricks Asset BundlesCI/CDAWS S3AWS GlueAWS RedshiftAuto LoaderStreaming IngestionCDCSCD Type 1SCD Type 2Data Quality FrameworksAudit FrameworksError Handling FrameworksInformatica PowerCenter MappingsInformatica WorkflowsAWS LambdaSpark Performance OptimizationDelta Performance OptimizationCloud Data ModernizationData Migration

Highlights

  • Actively hiring

More Details

RoleData Platform Modernization Engineer - Junior (Databricks)
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

NTT DATA logo

NTT DATA

IT Consulting

Data Platform Modernization Engineer - Junior (Databricks) at NTT DATA | SkillMX | SkillMX