Join our dynamic team as a Deputy Manager, specializing in Engineering as a Service/Operate. This pivotal role focuses on leading Databricks initiatives and driving innovation in data solutions. You will be instrumental in shaping how we leverage data across multiple portfolios and guiding critical migrations, including the transition from Hadoop to Azure.

This position offers a unique opportunity to make a significant impact on our data infrastructure and strategy. We are seeking a motivated individual who can provide strong technical leadership and contribute to the evolution of our data services.

Design and build robust, scalable data pipelines for efficient ETL processes, integrating data from diverse sources into our data lake or warehouse. Develop sophisticated data models to optimize storage, retrieval, and analysis, considering data volume, variety, and velocity.

Ensure seamless data integration across various systems, including databases, APIs, and streaming platforms, maintaining data consistency. Implement advanced data transformation and cleansing techniques to ensure data quality and readiness for analysis.

Manage and maintain data warehouses or data lakes for effective storage of structured and unstructured data. Establish strong data governance and security practices to protect sensitive information and ensure regulatory compliance.

Continuously monitor and optimize data processing performance, identifying and resolving bottlenecks. Collaborate closely with data analysts and scientists to deliver reliable and well-organized datasets tailored to their analytical and modeling needs.

Possess strong programming and scripting skills in languages such as Python, SQL, or Scala, complemented by expertise in scripting and automation tools. Demonstrate hands-on experience with the Azure cloud environment, Azure Databricks, SQL, Spark, Scala, or Python. Familiarity with Jira, Github, and DevOps tools is essential.

Knowledge of data integration and ETL tools like Apache Kafka, Apache Spark, Apache Airflow, or Talend is required. A solid understanding of data modeling concepts, dimensional modeling, and schema design is necessary. Experience with big data technologies such as Apache Hadoop, Hive, HBase, or Cassandra is expected.

Proficiency in cloud platforms for data storage, processing, and analytics is crucial. A strong grasp of data governance principles, data security best practices, and regulatory compliance is mandatory. Excellent problem-solving, analytical, and communication skills are vital for effective collaboration with cross-functional teams and stakeholders.