Data Engineer-Data Platforms-Azure
IBM
IBM
Join IBM Consulting and shape the future of hybrid cloud and AI for leading companies worldwide. As a Data Engineer on our Data Platforms team, you will leverage robust IBM technology and strategic partnerships to drive meaningful change. We foster a culture of curiosity, innovation, and continuous growth, valuing your unique contributions to accelerate client impact.
This role focuses on developing and maintaining advanced data engineering solutions within the Azure Cloud ecosystem. You will be instrumental in designing, building, and operating efficient batch and real-time data pipelines, utilizing a comprehensive suite of Azure services to deliver high-impact data strategies.
Your key responsibilities will include designing and constructing robust batch and real-time data pipelines using Azure services like Azure Synapse Analytics, Azure Data Factory, Azure DataBricks, and Event Hub. You will develop and manage the data layer utilizing Azure Synapse Analytics, SQL DW, and Cosmos DB. Furthermore, you will apply your expertise in Azure Data Platform components, implement open-source technologies such as Apache Airflow and dbt, and operate critical Azure services for data streaming and processing.
We are seeking individuals proficient in Azure Data Platform components including ADLS2, Blob Storage, SQLDW, Synapse Analytics (Spark & SQL), Azure Functions with Python, Azure Purview, and Cosmos DB. Experience in developing and operating batch and real-time data pipelines using Azure services is essential. Familiarity with open-source technologies like Apache Airflow, dbt, Spark/Python, or Spark/Scala is highly valued. A strong understanding of operating Azure services such as Event Hub, Streaming Analytics, Managed Streaming for Apache Kafka, and Azure DataBricks with Spark is also required.
IBM
Data Analytics