Big Data Lead
Hexaware
Hexaware
Join our team as a Big Data Lead and spearhead a critical data warehouse migration project. This role focuses on transitioning an existing Oracle-based data warehouse to Databricks on the Allianz Data Platform (ADP). We are seeking a seasoned Data Engineer with a strong background in modern data engineering practices and AI-enabling platforms, making this an exciting opportunity for innovation and leadership.
Key responsibilities include analyzing the current Oracle data warehouse and PostgreSQL source feeds, designing the target model in Databricks, and building/optimizing ETL/ELT pipelines using Spark SQL and PySpark. You will be instrumental in translating Oracle SQL/PL-SQL logic to Databricks, ensuring functional equivalence and performance. Ensuring data quality, performance, and reliability for downstream analytics and AI/ML use cases is paramount. Collaboration with Enterprise Architects, the ADP platform team, and business stakeholders is essential to maintain alignment with architecture and best practices. Documenting migration patterns, data models, and operational procedures will support long-term maintainability.
This position requires a Graduate or Postgraduate degree in Engineering, or a Postgraduate degree in non-engineering disciplines. A minimum of 5 to 9 years of experience in ITIL process, application support, or deployment activities is essential. Candidates should possess a positive attitude and a proactive approach to providing solutions. Experience with Databricks, SQL, data warehousing, ETL/ELT processes, Spark SQL, PySpark, Oracle, PostgreSQL, and SAS ETL is highly desirable for success in this role.
Hexaware Technologies
IT Consulting