Data Engineer-Data Platforms-Azure
IBM
IBM
Join IBM Consulting and shape the future of hybrid cloud and AI for leading companies worldwide. This role offers the opportunity to collaborate globally, leverage robust IBM technology and Red Hat, and drive impactful change. We foster a culture of curiosity and innovation, encouraging you to challenge norms and develop creative solutions. Your long-term career development and unique skills are highly valued in our growth-focused, empathetic environment.
As a Data Engineer specializing in Azure Data Platforms, you will design, develop, and maintain data engineering solutions. This includes building and operating batch and real-time data pipelines using Azure services like Synapse Analytics, Data Factory, and Databricks.
You will also develop and manage the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB. Implementation and maintenance of Azure Data Platform components such as ADLS2, Blob Storage, and Azure Purview are key. Proficiency in open-source technologies like Apache Airflow and dbt, alongside Spark/Python or Spark/Scala, is essential for supporting these solutions and ensuring seamless data processing.
Demonstrated experience in data engineering on the Azure Cloud ecosystem is required, focusing on designing, building, and operating batch and real-time data pipelines. You should possess extensive knowledge of Azure services including Azure Synapse Analytics, Azure Data Factory, Azure Databricks, and Event Hub.
Experience with Azure Data Platform components like ADLS2, Blob Storage, SQLDW, Synapse Analytics (with Spark and SQL), Azure Functions (with Python), Azure Purview, and Cosmos DB is crucial. Familiarity with open-source technologies such as Apache Airflow and dbt, combined with proficiency in Spark/Python or Spark/Scala, is also a strong requirement for this role.
IBM
Technology