Databricks Data Engineer

Infosys

7–9 yrs Bengaluru Full Time Hybrid (office + remote)
Infosys logo
Posted : today
Actively hiring

Job description

Join Infosys' Data & Analytics practice as a Senior Consultant, specializing in Databricks Data Engineering. You will be instrumental in modernizing data landscapes through cloud-native architectures, advanced analytics, and AI-driven platforms. This role offers the opportunity to work on enterprise-scale data modernization programs, including Lakehouse implementations, cloud migrations, and the development of real-time analytics and AI-ready data ecosystems.

This position is within the Data & Analytics Unit, focusing on critical functional areas that drive innovation and business value through sophisticated data solutions. As part of Infosys Limited, you will contribute to shaping the future of data for major organizations.

Responsibilities

Design, develop, and maintain robust, scalable data pipelines utilizing Databricks and Apache Spark. Build and optimize complex ETL/ELT workflows to efficiently process large volumes of structured and unstructured data. Develop advanced data ingestion frameworks for seamless integration from diverse sources like databases, APIs, and cloud storage. Implement and manage cutting-edge Lakehouse architectures leveraging Databricks and Delta Lake. Enhance Spark job performance, scalability, and cost efficiency within Databricks environments. Collaborate effectively with analytics, BI, and business teams to fulfill data requirements and deliver reusable data assets. Establish comprehensive data quality frameworks, monitoring solutions, and governance controls for all data pipelines. Address and resolve production issues promptly to ensure uninterrupted business operations. Participate actively in architecture and code reviews, promoting industry best practices and technical excellence. Create and maintain essential technical documentation, reusable components, and deployment standards.

Qualifications

Demonstrate strong hands-on expertise with Databricks platform. Possess proficiency in PySpark or Spark SQL for efficient distributed data processing. Exhibit strong SQL skills, including query optimization and performance tuning. Have proven experience in designing and developing comprehensive ETL/ELT pipelines. Showcase practical experience with Delta Lake, DataFrames, and Spark architecture. Maintain a solid understanding of data warehousing concepts, dimensional modeling, and data lake architectures. Acquire hands-on experience with major cloud platforms such as Azure, AWS, or GCP. Familiarity with Git-based source code management and essential DevOps practices is expected. Possess exceptional analytical, debugging, and problem-solving capabilities. Requires 7-9 years of relevant professional experience.

Essential Skills

DatabricksApache SparkPySparkSpark SQLSQLETLELTDelta LakeDataFramesAzureAWSGCPGitDevOps

Highlights

  • Actively hiring

More Details

RoleDatabricks Data Engineer
IndustryInformation Technology & Services
DepartmentData Science
Employment TypeFull Time, Hybrid (office + remote)

About the Company

Infosys logo

Infosys

Information Technology & Services

Databricks Data Engineer at Infosys | SkillMX | SkillMX