Data Engineer-Data Platforms-Azure
IBM
IBM
Join IBM Consulting and help leading companies navigate their hybrid cloud and AI journeys. We foster a collaborative global environment where curiosity drives innovation and success. Our culture emphasizes continuous growth, empathy, and developing your unique skills for long-term career advancement.
This role focuses on designing, developing, and maintaining robust data engineering solutions within the Azure Cloud ecosystem, specifically for data platforms.
Key responsibilities include designing and building scalable batch and real-time data pipelines using Azure services like Synapse Analytics, Data Factory, Databricks, and Event Hub. You will also develop and operate the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB, while implementing and maintaining core Azure Data Platform components.
Leverage open-source technologies such as Apache Airflow and dbt, alongside Spark with Python or Scala, to enhance data engineering solutions. Ensure the seamless operation and maintenance of these solutions for reliable data processing.
We are seeking individuals with proven experience in data engineering on the Azure Cloud platform. This includes hands-on work with Azure Data Platform components such as ADLS2, Blob Storage, SQLDW, Synapse Analytics (Spark/SQL), Azure Functions (Python), Azure Purview, and Cosmos DB.
Demonstrated proficiency in open-source technologies like Apache Airflow and dbt, and Spark (Python/Scala) is essential. Experience in designing, building, and operating both batch and real-time data pipelines, as well as the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB, is required.
IBM
Information Technology & Services