Data Engineer-Data Platforms-Azure
IBM
IBM
Join IBM Consulting to shape hybrid cloud and AI journeys for leading companies worldwide. Leverage robust IBM technology, Red Hat, and strategic partnerships to drive meaningful change. Embrace a culture of curiosity, innovation, and continuous growth, where your unique skills and long-term career development are prioritized. This role focuses on designing, developing, and maintaining advanced data engineering solutions within the Azure Cloud ecosystem.
As a Data Engineer on the Data Platforms team, you will be instrumental in developing and maintaining data engineering solutions on Azure.
Key responsibilities include: - Architecting and building robust batch and real-time data pipelines using Azure services like Azure Synapse Analytics, Azure Data Factory, and Azure DataBricks. - Designing and implementing the data layer across Azure Synapse Analytics, SQL DW, and Cosmos DB. - Managing and integrating Azure Data Platform components such as ADLS2, Blob Storage, SQLDW, Synapse Analytics, Azure Functions, Azure Purview, and Cosmos DB. - Utilizing open-source technologies like Apache Airflow and dbt, alongside Spark with Python or Scala, to enhance data engineering workflows. - Ensuring the seamless operation and maintenance of data engineering solutions for efficient data processing.
We are seeking a Data Engineer with proven experience in Azure cloud environments and a strong understanding of data platform components.
Essential qualifications: - Expertise in Azure services including Azure Synapse Analytics, Azure Data Factory, Azure DataBricks, and Event Hub for pipeline development. - Hands-on experience with Azure Data Platform components like ADLS2, Blob Storage, SQLDW, Synapse Analytics (Spark/SQL), Azure Functions (Python), Azure Purview, and Cosmos DB. - Proficiency in open-source technologies such as Apache Airflow, dbt, Spark (Python/Scala) for data engineering solutions. - Demonstrated experience in designing, building, and operating batch and real-time data pipelines on Azure. - Familiarity with developing the data layer within Azure Synapse Analytics, SQL DW, and Cosmos DB.
IBM
Information Technology & Services