Data Platform Modernization Engineer - Junior (Databricks)

NTT DATA

3–6 yrs Bengaluru Full Time Hybrid (office + remote)
NTT DATA logo
Posted : yesterday
Actively hiring

Job description

Join NTT DATA as a Junior Data Platform Modernization Engineer specializing in Databricks. This role focuses on migrating existing data workloads to the Databricks Lakehouse. You will analyze current ETL processes, develop new pipelines using Databricks technologies, and ensure seamless data transformation and processing. If you are passionate about innovation and want to grow in a dynamic, inclusive organization, this is an excellent opportunity.

Responsibilities

Analyze Informatica and AWS Glue jobs to understand data flows and transformations. Develop Databricks notebooks and data pipelines using Python, PySpark, and SQL for ETL modernization. Implement layered data architectures (Bronze, Silver, Gold) with Delta Lake. Create batch and incremental processing pipelines, and manage Databricks Jobs/Workflows. Convert existing ETL logic into Databricks, ensuring business rules are preserved. Support migration of AWS Glue jobs and database processing. Conduct unit testing, data reconciliation, and resolve pipeline execution issues. Collaborate with senior developers on technical challenges and adhere to development standards. Participate in code reviews and prepare technical documentation.

Qualifications

This role requires 3-6 years of experience in data engineering or ETL development, with hands-on experience in Databricks and Apache Spark. Proficiency in Python/PySpark and SQL is essential. Experience with Databricks notebooks, Spark SQL, Delta Lake, and Databricks Jobs/Workflows is crucial. Familiarity with ETL/ELT pipeline development, relational databases, and data transformation concepts is needed. A Bachelor's degree in Computer Science, IT, Engineering, or equivalent experience is required. Strong analytical, problem-solving, and communication skills are a must.

Essential Skills

DatabricksApache SparkPythonPySparkSQLSpark SQLDatabricks SQLDelta LakeDatabricks Jobs/WorkflowsETLELTRelational DatabasesData TransformationSource-to-Target MappingUnit TestingData ValidationTroubleshootingBronze/Silver/Gold Architecture

Good to Have

Databricks CertificationLakeflow Declarative PipelinesDelta Live TablesUnity CatalogDatabricks Asset BundlesCI/CDAWS S3AWS GlueAWS RedshiftAuto LoaderStreaming IngestionCDCSCD Type 1/2Data Quality FrameworksAudit FrameworksError Handling FrameworksInformatica PowerCenterAWS LambdaSpark Performance OptimizationDelta Performance OptimizationCloud Data ModernizationCloud Migration

Highlights

  • Actively hiring

More Details

RoleData Platform Modernization Engineer - Junior (Databricks)
IndustryInformation Technology & Services
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

NTT DATA logo

NTT DATA

Information Technology & Services

Data Platform Modernization Engineer - Junior (Databricks) at NTT DATA | SkillMX | SkillMX