Databricks Engineer

Infosys

5–8 yrs Bengaluru Full Time Work from office
Infosys logo
Posted : today
Actively hiring

Job description

Join our Data & Analytics Unit as an experienced Databricks Engineer. This role involves designing, building, and optimizing robust, large-scale data solutions. If you possess strong expertise in Databricks, PySpark, Spark, SQL, and major cloud platforms, and are adept at modern data engineering, we encourage you to apply.

This position is with Infosys Limited and is located in Bangalore, India. We are seeking professionals with 5-8 years of dedicated experience in data engineering and big data technologies to contribute to our innovative projects.

Responsibilities

Key responsibilities include designing and maintaining data pipelines using Databricks, developing efficient ETL/ELT workflows, and optimizing data transformation logic with PySpark and Spark SQL. You will implement and manage Delta Lake for data storage, work with both batch and real-time data ingestion, and monitor pipelines for performance and scalability.

Furthermore, this role requires collaboration with data analysts, scientists, and business stakeholders to ensure data quality, security, and governance. Troubleshooting production issues and providing ongoing support are also crucial aspects of this position.

Qualifications

Candidates should have 5-8 years of experience in Data Engineering and Big Data. Essential skills include hands-on experience with Databricks, PySpark, Spark SQL, Python, and SQL. Familiarity with Delta Lake, Databricks Workflows, Unity Catalog, and cloud platforms like Azure, AWS, or GCP is expected.

Experience with orchestration tools such as Azure Data Factory or Apache Airflow is beneficial, alongside a solid understanding of data warehousing and ETL/ELT methodologies. Preferred qualifications include experience with Kafka/Event Hub for streaming data and familiarity with Git, CI/CD, and DevOps practices. Strong problem-solving, analytical, and communication abilities are a must.

Essential Skills

DatabricksPySparkSpark SQLDelta LakePythonSQLAWSAzureGCPData WarehousingETLELTGitCI/CDDevOps

Good to Have

Databricks WorkflowsUnity CatalogAzure Data FactoryApache AirflowKafkaEvent Hub

Highlights

  • Actively hiring

More Details

RoleDatabricks Engineer
DepartmentData Engineering
Employment TypeFull Time, Work from office

About the Company

Infosys logo

Infosys

IT Consulting

Databricks Engineer at Infosys | SkillMX | SkillMX