Azure Databricks Data Engineer
Infosys
Infosys
Join Infosys' Data & Analytics practice as a Databricks Data Engineer and drive enterprise-scale data modernization programs. You will be instrumental in implementing cloud-native architectures, advanced analytics, AI, and intelligent data platforms.
This role offers the opportunity to work on cutting-edge projects involving Lakehouse implementations, cloud migrations, and real-time analytics, contributing to the creation of AI-ready data ecosystems.
Design, develop, and maintain robust data pipelines using Databricks and Apache Spark. Build and optimize ETL/ELT workflows for processing large datasets, and develop ingestion frameworks for integrating data from diverse sources.
Implement and manage Lakehouse architecture with Databricks and Delta Lake. Optimize Spark jobs for performance, scalability, and cost efficiency. Collaborate with analytics and business teams to deliver reusable data assets and ensure data quality and governance.
A minimum of 5-7 years of hands-on experience with Databricks is essential, alongside proficiency in PySpark or Spark SQL. Strong SQL skills for query optimization and experience in designing ETL/ELT pipelines are required.
Candidates should have experience with Delta Lake, DataFrames, and Spark architecture. Familiarity with data warehousing, data lake concepts, cloud platforms (Azure, AWS, GCP), Git, and DevOps practices is necessary. Excellent analytical and problem-solving abilities are a must.
Infosys
Information Technology & Services