Join Deloitte's Regulatory & Financial Risk team as a Data Engineer in Hyderabad. You will be instrumental in designing, building, and maintaining robust data processing solutions. This role offers a chance to work with cutting-edge big data technologies and contribute to critical risk mitigation strategies.

We are seeking a skilled professional with extensive PySpark expertise and a strong grasp of the data engineering landscape. If you thrive on optimizing distributed computing and have a proven track record in SDLC-based project development, this is an exciting opportunity to make a significant impact.

Your core responsibilities will involve designing and optimizing large-scale data pipelines using PySpark, building and maintaining ETL/ELT workflows, and working with big data technologies like Hadoop, Spark, and Impala. You will collaborate with cross-functional teams, ensuring code quality through rigorous reviews and version control. Performance tuning of Spark jobs, data modeling, and pipeline orchestration are key aspects of this role. Furthermore, you will actively participate in the entire SDLC, troubleshoot data processing issues, and contribute to automation and monitoring frameworks.

A mandatory requirement for this role is strong hands-on experience with Python and PySpark. Proficiency in SQL, including advanced querying and performance tuning, is essential. You should have experience with distributed data systems such as Hadoop and the Spark ecosystem, along with familiarity with Impala, Hive, and relational databases like SQL Server. A solid understanding of data structures, algorithms, performance optimization, ETL pipeline development, and data warehousing concepts is expected. Experience with version control systems like Git and knowledge of SDLC methodologies are also required. Cloud platform experience (Azure, AWS, GCP) and exposure to workflow orchestration tools like Airflow are highly desirable.