Azure Databricks Data Engineer

Infosys

3–5 yrs Bengaluru Full Time Work from office
Infosys logo
Posted : today
Actively hiring

Job description

Join our team as a Databricks Data Engineer and take a lead role in designing, developing, and optimizing cutting-edge data platforms. This position focuses on leveraging Databricks and cloud technologies to build robust, large-scale data pipelines and implement modern Lakehouse architectures. You'll be instrumental in enabling advanced analytics and driving enterprise-wide digital transformation initiatives.

This role offers an exciting opportunity to collaborate with cross-functional teams, including data architects, analysts, and business stakeholders, to deliver high-performance data solutions. You will be part of a dynamic practice that helps organizations modernize their data landscapes through cloud-native architectures, advanced analytics, AI, and intelligent data platforms. Experience enterprise-scale data modernization programs, including Lakehouse implementations, cloud migrations, and real-time analytics.

Responsibilities

Key responsibilities include designing, developing, and maintaining scalable data pipelines using Databricks and Apache Spark. You will build and optimize ETL/ELT workflows for processing massive datasets, and develop data ingestion frameworks to integrate data from diverse sources. Implementing and managing Lakehouse architectures with Databricks and Delta Lake, alongside optimizing Spark jobs for performance and cost-efficiency, are core functions. Collaboration with analytics, BI, and business teams to understand data needs and deliver reusable data assets is essential. You will also implement data quality frameworks, monitoring solutions, and governance controls, troubleshoot production issues, and contribute to architecture and code reviews, fostering best practices and creating comprehensive technical documentation.

Qualifications

We are looking for candidates with 3-5 years of experience in Data Engineering and Big Data technologies. Essential qualifications include strong hands-on experience with Databricks, proficiency in PySpark or Spark SQL for distributed data processing, and robust SQL skills with a focus on query optimization. Experience in designing and developing ETL/ELT pipelines, along with hands-on expertise in Delta Lake, DataFrames, and Spark architecture, is required. A solid understanding of data warehousing concepts, dimensional modeling, and data lake architectures is necessary. Prior experience working with cloud platforms such as Azure, AWS, or GCP is beneficial. Familiarity with Git-based source code management and DevOps practices is expected. Exceptional analytical, debugging, and problem-solving skills are crucial for success in this role.

Essential Skills

DatabricksApache SparkPySparkSpark SQLETLELTDelta LakeSQLData WarehousingDimensional ModelingData LakeAzureAWSGCPGitDevOps

Highlights

  • Actively hiring

More Details

RoleAzure Databricks Data Engineer
IndustryInformation Technology & Services
DepartmentData Engineering
Employment TypeFull Time, Work from office

About the Company

Infosys logo

Infosys

Information Technology & Services

Azure Databricks Data Engineer at Infosys | SkillMX | SkillMX