Azure Databricks Data Engineer

Infosys

7–9 yrs Bengaluru Full Time Hybrid (office + remote)
Infosys logo
Posted : today
Actively hiring

Job description

Join our Data & Analytics practice as a Databricks Data Engineer, instrumental in modernizing data landscapes. You will architect and implement cloud-native solutions, advanced analytics, and intelligent data platforms.

This role is pivotal in driving enterprise-scale data modernization programs. You'll focus on Lakehouse implementations, cloud migrations, and building AI-ready data ecosystems, transforming how organizations leverage their data.

Responsibilities

Design, develop, and maintain robust data pipelines using Databricks and Apache Spark, ensuring scalability and efficiency. Build and optimize ETL/ELT workflows for processing vast amounts of structured and unstructured data. Develop adaptable data ingestion frameworks to seamlessly integrate data from diverse sources like databases, APIs, and cloud storage. Implement and manage a cutting-edge Lakehouse architecture with Databricks and Delta Lake. Optimize Spark jobs and Databricks workloads for peak performance, cost-effectiveness, and scalability. Collaborate closely with analytics, BI, and business teams to deliver valuable, reusable data assets that meet evolving requirements. Establish comprehensive data quality frameworks, monitoring solutions, and governance controls throughout the data pipelines. Proactively troubleshoot production issues, ensuring swift and effective resolution to maintain business continuity. Engage in architecture and code reviews, contributing to technical discussions and championing best practices. Create and maintain essential technical documentation, reusable components, and deployment standards.

Qualifications

We seek professionals with 7-9 years of extensive hands-on experience in Databricks. Demonstrated proficiency in PySpark or Spark SQL for distributed data processing is essential. Strong SQL skills, including expertise in query optimization and performance tuning, are required. Proven experience in designing and developing efficient ETL/ELT pipelines. Practical experience with Delta Lake, DataFrames, and understanding Spark architecture. Solid knowledge of data warehousing concepts, dimensional modeling, and data lake architectures. Experience working with major cloud platforms such as Azure, AWS, or GCP. Familiarity with Git-based source code management and DevOps practices is expected. Exceptional analytical, debugging, and problem-solving capabilities are crucial for this role.

Essential Skills

DatabricksApache SparkPySparkSpark SQLSQLETLELTDelta LakeDataFramesData WarehousingDimensional ModelingData Lake ArchitecturesAzureGitDevOps

Good to Have

AWSGCP

Highlights

  • Actively hiring

More Details

RoleAzure Databricks Data Engineer
IndustryInformation Technology & Services, Data Analytics
DepartmentData Engineering, Software Development
Employment TypeFull Time, Hybrid (office + remote)

About the Company

Infosys logo

Infosys

Information Technology & Services

Azure Databricks Data Engineer at Infosys | SkillMX | SkillMX