Databricks Engineer
Infosys
Infosys
Join our Data & Analytics Unit as an experienced Databricks Engineer. This role involves designing, building, and optimizing robust, large-scale data solutions. If you possess strong expertise in Databricks, PySpark, Spark, SQL, and major cloud platforms, and are adept at modern data engineering, we encourage you to apply.
This position is with Infosys Limited and is located in Bangalore, India. We are seeking professionals with 5-8 years of dedicated experience in data engineering and big data technologies to contribute to our innovative projects.
Key responsibilities include designing and maintaining data pipelines using Databricks, developing efficient ETL/ELT workflows, and optimizing data transformation logic with PySpark and Spark SQL. You will implement and manage Delta Lake for data storage, work with both batch and real-time data ingestion, and monitor pipelines for performance and scalability.
Furthermore, this role requires collaboration with data analysts, scientists, and business stakeholders to ensure data quality, security, and governance. Troubleshooting production issues and providing ongoing support are also crucial aspects of this position.
Candidates should have 5-8 years of experience in Data Engineering and Big Data. Essential skills include hands-on experience with Databricks, PySpark, Spark SQL, Python, and SQL. Familiarity with Delta Lake, Databricks Workflows, Unity Catalog, and cloud platforms like Azure, AWS, or GCP is expected.
Experience with orchestration tools such as Azure Data Factory or Apache Airflow is beneficial, alongside a solid understanding of data warehousing and ETL/ELT methodologies. Preferred qualifications include experience with Kafka/Event Hub for streaming data and familiarity with Git, CI/CD, and DevOps practices. Strong problem-solving, analytical, and communication abilities are a must.
Infosys
IT Consulting