Azure Databricks Data Engineer

Infosys

9–11 yrs Bengaluru Full Time Hybrid (office + remote)
Infosys logo
Posted : today
Actively hiring

Job description

Join Infosys' Data & Analytics practice as a Lead Consultant and drive enterprise-scale data modernization programs. This role focuses on leveraging Databricks for implementing cloud-native architectures, advanced analytics, AI, and intelligent data platforms. You will be instrumental in modernizing data landscapes through Lakehouse implementations, cloud migrations, real-time analytics, and building AI-ready data ecosystems.

This opportunity is ideal for experienced professionals passionate about shaping the future of data within a leading technology organization. Contribute to cutting-edge projects and help clients unlock the full potential of their data.

Responsibilities

As a Databricks Data Engineer, your core responsibilities include designing, developing, and optimizing modern data platforms. You will build and maintain scalable data pipelines using Databricks and Apache Spark, alongside creating robust ETL/ELT workflows for extensive data processing. Developing data ingestion frameworks from various sources and implementing Lakehouse architectures with Delta Lake are key. Optimizing Spark jobs for peak performance, scalability, and cost-efficiency is also critical.

Further duties involve collaborating with analytics and business teams to deliver reusable data assets, establishing data quality and governance controls, and troubleshooting production issues. You'll also participate in code and architecture reviews, promoting best practices, and creating essential technical documentation.

Qualifications

We are seeking candidates with 9-11 years of hands-on experience with Databricks. Essential skills include proficiency in PySpark or Spark SQL for distributed data processing, strong SQL expertise with query optimization, and proven experience in designing and developing ETL/ELT pipelines. Familiarity with Delta Lake, DataFrames, Spark architecture, and cloud platforms like Azure, AWS, or GCP is required.

Knowledge of data warehousing concepts, dimensional modeling, and data lake architectures is beneficial. Experience with Git-based source code management and DevOps practices is also important. Strong analytical, debugging, and problem-solving abilities are crucial for success in this role. A Bachelor of Engineering degree is a prerequisite.

Essential Skills

DatabricksApache SparkPySparkSpark SQLSQLETLELTDelta LakeDataFramesSpark ArchitectureData WarehousingDimensional ModelingData Lake ArchitectureAzureAWSGCPGitDevOpsData PipelinesData IngestionData QualityData Governance

Highlights

  • Actively hiring

More Details

RoleAzure Databricks Data Engineer
IndustryInformation Technology & Services, Data Analytics
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

Infosys logo

Infosys

Information Technology & Services

Azure Databricks Data Engineer at Infosys | SkillMX | SkillMX