Databricks

Infosys

2–3 yrs Bengaluru Full Time Hybrid (office + remote)
Infosys logo
Posted : today
Actively hiring

Job description

Join a dynamic team where data engineering drives real-world impact. This role focuses on designing and deploying robust data solutions utilizing Databricks and PySpark. You will empower teams to transform raw data into reliable, analytics-ready assets. Close collaboration with consultants, data engineers, and stakeholders is key to understanding business needs, building dependable pipelines, and ensuring high-quality releases. This is an ideal opportunity for professionals with 2-3 years of experience passionate about solving data challenges, enhancing performance, and embracing modern lakehouse practices. If you are driven by clean engineering, continuous improvement, and contributing in a collaborative environment where your efforts are recognized and valued, this position offers a balanced path for ownership, mentorship, and professional growth.

Responsibilities

- Develop and maintain robust data pipelines and transformations using Databricks and PySpark. - Implement scalable ETL/ELT workflows for data ingestion, cleansing, and curation supporting downstream analytics and reporting. - Optimize Spark jobs for peak performance and cost-efficiency through tuning partitions, caching, joins, and cluster configurations. - Create reusable notebooks and modular code to foster consistent development and simplify maintenance. - Conduct thorough data validation, reconciliation, and quality checks to guarantee dataset accuracy and reliability. - Collaborate with cross-functional teams to gather requirements, clarify data definitions, and deliver cohesive solutions. - Provide support for deployments and production operations by troubleshooting failures, analyzing logs, and resolving incidents. - Contribute to documentation, coding standards, and best practices for Databricks-based development.

Qualifications

- Hold a Bachelor’s or Master’s degree in B.TECH, M.TECH, MCA, or MSC, or an equivalent qualification. - Possess 2-3 years of practical experience in data engineering or analytics engineering projects, specifically with Databricks. - Demonstrate strong proficiency in PySpark for building transformations and managing distributed data processing. - Exhibit a solid understanding of data pipeline concepts, fundamental data modeling, and handling structured/semi-structured data. - Be adept at debugging and troubleshooting Spark jobs, with a proven ability to collaborate effectively within delivery teams. - Experience with Spark optimization techniques and practical performance tuning in Databricks environments is highly desirable. - Familiarity with Delta Lake concepts, including ACID tables, schema evolution, and incremental processing patterns, is beneficial. - Exposure to orchestrating workflows and managing dependencies for end-to-end pipeline execution is a plus. - Prior experience in agile delivery models, taking ownership of tasks, timelines, and quality outcomes, is valued. - Possess strong communication skills to translate requirements into actionable data solutions and clearly document outcomes.

Essential Skills

DatabricksPySparkETLDelta LakeSpark SQLData ModelingWorkflow OrchestrationPerformance Tuning

Good to Have

Spark optimization techniquesDelta Lake conceptsorchestrating workflowsagile delivery modelsStrong communication skills

Highlights

  • Actively hiring

More Details

RoleDatabricks
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

Infosys logo

Infosys

IT Consulting

Databricks at Infosys | SkillMX | SkillMX