Azure Databricks Data Engineer

Infosys

2–3 yrs Bengaluru Full Time Hybrid (office + remote)
Infosys logo
Posted : today
Actively hiring

Job description

Join our Data & Analytics practice to modernize data landscapes through cloud-native architectures, advanced analytics, AI, and intelligent data platforms. As an Associate Consultant specializing in Databricks Data Engineering, you will contribute to enterprise-scale data modernization programs. This includes Lakehouse implementations, cloud migrations, real-time analytics, and building AI-ready data ecosystems.

This role offers an opportunity to work with cutting-edge technologies and drive significant digital transformation initiatives.

Responsibilities

Key responsibilities include designing, developing, and maintaining robust data pipelines using Databricks and Apache Spark. You will build and optimize ETL/ELT workflows for extensive datasets, and develop data ingestion frameworks to integrate data from diverse sources.

Implementing and managing the Lakehouse architecture with Databricks and Delta Lake is crucial. Optimizing Spark jobs for performance and cost efficiency, and collaborating with analytics and business teams to deliver data assets are also core duties. You will implement data quality frameworks, monitor solutions, and troubleshoot production issues.

Furthermore, you will participate in architecture and code reviews, promote best practices, and create essential technical documentation and reusable components.

Qualifications

We are looking for professionals with 2-3 years of dedicated experience in Data Engineering and Big Data technologies, with strong hands-on expertise in Databricks. Proficiency in PySpark or Spark SQL for distributed data processing is essential.

Candidates should possess strong SQL skills, including query optimization and performance tuning. Experience in designing and developing ETL/ELT pipelines, along with hands-on work with Delta Lake, DataFrames, and Spark architecture, is required. Familiarity with cloud platforms like Azure, AWS, or GCP, and Git-based source code management and DevOps practices are also necessary.

A Bachelor of Engineering degree is required, alongside strong analytical, debugging, and problem-solving capabilities. Experience with data warehousing concepts, dimensional modeling, and data lake architectures is beneficial.

Essential Skills

DatabricksApache SparkPySparkSpark SQLSQLETLELTDelta LakeDataFramesAzureAWSGCPGitDevOps

Good to Have

Data WarehousingDimensional ModelingData Lake Architectures

Highlights

  • Actively hiring

More Details

RoleAzure Databricks Data Engineer
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

Infosys logo

Infosys

IT Consulting

Azure Databricks Data Engineer at Infosys | SkillMX | SkillMX