Databricks
Infosys
Infosys
Join a dynamic team where data engineering drives real-world impact. This role focuses on designing and deploying robust data solutions utilizing Databricks and PySpark. You will empower teams to transform raw data into reliable, analytics-ready assets. Close collaboration with consultants, data engineers, and stakeholders is key to understanding business needs, building dependable pipelines, and ensuring high-quality releases. This is an ideal opportunity for professionals with 2-3 years of experience passionate about solving data challenges, enhancing performance, and embracing modern lakehouse practices. If you are driven by clean engineering, continuous improvement, and contributing in a collaborative environment where your efforts are recognized and valued, this position offers a balanced path for ownership, mentorship, and professional growth.
- Develop and maintain robust data pipelines and transformations using Databricks and PySpark. - Implement scalable ETL/ELT workflows for data ingestion, cleansing, and curation supporting downstream analytics and reporting. - Optimize Spark jobs for peak performance and cost-efficiency through tuning partitions, caching, joins, and cluster configurations. - Create reusable notebooks and modular code to foster consistent development and simplify maintenance. - Conduct thorough data validation, reconciliation, and quality checks to guarantee dataset accuracy and reliability. - Collaborate with cross-functional teams to gather requirements, clarify data definitions, and deliver cohesive solutions. - Provide support for deployments and production operations by troubleshooting failures, analyzing logs, and resolving incidents. - Contribute to documentation, coding standards, and best practices for Databricks-based development.
- Hold a Bachelor’s or Master’s degree in B.TECH, M.TECH, MCA, or MSC, or an equivalent qualification. - Possess 2-3 years of practical experience in data engineering or analytics engineering projects, specifically with Databricks. - Demonstrate strong proficiency in PySpark for building transformations and managing distributed data processing. - Exhibit a solid understanding of data pipeline concepts, fundamental data modeling, and handling structured/semi-structured data. - Be adept at debugging and troubleshooting Spark jobs, with a proven ability to collaborate effectively within delivery teams. - Experience with Spark optimization techniques and practical performance tuning in Databricks environments is highly desirable. - Familiarity with Delta Lake concepts, including ACID tables, schema evolution, and incremental processing patterns, is beneficial. - Exposure to orchestrating workflows and managing dependencies for end-to-end pipeline execution is a plus. - Prior experience in agile delivery models, taking ownership of tasks, timelines, and quality outcomes, is valued. - Possess strong communication skills to translate requirements into actionable data solutions and clearly document outcomes.
Infosys
IT Consulting