Azure Databricks Data Engineer

Infosys

7–9 yrs Bengaluru Full Time Hybrid (office + remote)
Infosys logo
Posted : today
Actively hiring

Job description

Join our Data & Analytics practice to modernize data landscapes using cloud-native architectures, advanced analytics, AI, and intelligent data platforms. As a Senior Consultant, you will drive enterprise-scale data modernization programs, focusing on Lakehouse implementations, cloud migrations, real-time analytics, and building AI-ready data ecosystems.

This role is integral to leveraging Databricks and cloud technologies to design, develop, and optimize cutting-edge data solutions. You will contribute to building scalable, high-performance data platforms that empower enterprise-wide digital transformation.

Responsibilities

Key responsibilities include designing, developing, and maintaining robust data pipelines using Databricks and Apache Spark. You will build and optimize ETL/ELT workflows for large datasets, develop data ingestion frameworks from diverse sources, and implement Lakehouse architecture with Delta Lake.

Further duties involve optimizing Spark jobs for performance and cost efficiency, collaborating with analytics and business teams to deliver data assets, and implementing data quality and governance controls. Troubleshooting production issues and participating in technical reviews to promote best practices are also essential.

This role requires creating technical documentation and reusable components, ensuring adherence to deployment standards.

Qualifications

We are seeking individuals with 7-9 years of hands-on experience with Databricks and proficiency in PySpark or Spark SQL for distributed data processing. Strong SQL skills, including query optimization and performance tuning, are essential.

Candidates should have experience designing and developing ETL/ELT pipelines, along with practical knowledge of Delta Lake, DataFrames, and Spark architecture. Familiarity with data warehousing concepts, dimensional modeling, and data lake architectures is required.

Experience with cloud platforms like Azure, AWS, or GCP, along with Git-based source code management and DevOps practices, is crucial. Excellent analytical, debugging, and problem-solving skills are necessary for success in this role.

Essential Skills

DatabricksApache SparkPySparkSpark SQLSQLETLELTDelta LakeDataFramesData WarehousingDimensional ModelingData Lake ArchitectureAzureGitDevOpsProblem SolvingData IngestionData Quality

Good to Have

AWSGCP

Highlights

  • Actively hiring

More Details

RoleAzure Databricks Data Engineer
IndustryInformation Technology & Services, Data Analytics
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

Infosys logo

Infosys

Information Technology & Services

Azure Databricks Data Engineer at Infosys | SkillMX | SkillMX