Data Engineer-Data Platforms-Azure
IBM
IBM
Join IBM Consulting to shape the future of hybrid cloud and AI for leading companies worldwide. We foster a collaborative environment where curiosity drives innovation and client success. This role focuses on leveraging robust IBM technology and Red Hat, empowering you to deliver impactful solutions.
Our culture prioritizes long-term career development and values individual skills and experiences. As a Data Engineer on the Azure Data Platforms team, you will be instrumental in designing, developing, and maintaining cutting-edge data engineering solutions within the Azure Cloud ecosystem.
You will be responsible for designing, building, and operating robust batch and real-time data pipelines utilizing a variety of Azure services. Key tasks include developing the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB, and applying deep expertise in Azure Data Platform components like ADLS2, Blob Storage, and Azure Purview.
Implementation of open-source technologies such as Apache Airflow and dbt, alongside Spark with Python or Scala, will be crucial for supporting data engineering solutions. You will also operate essential Azure services including Event Hub, Streaming Analytics, Managed Streaming for Apache Kafka, and Azure Databricks.
This role requires proficiency in Azure services like Azure Synapse Analytics, Azure Data Factory, Azure Databricks, and Event Hub for pipeline development. You should possess strong knowledge of the Azure Data Platform, including ADLS2, Blob Storage, SQLDW, Synapse Analytics (Spark/SQL), Azure Functions (Python), Azure Purview, and Cosmos DB.
Experience with open-source technologies like Apache Airflow, dbt, Spark (Python/Scala), and operating Azure services such as Event Hub, Streaming Analytics, Managed Streaming for Apache Kafka, and Azure Databricks with Spark is essential. Familiarity with designing, building, and operating batch and real-time data pipelines is a must.
IBM
Data Analytics