Data Engineer-Data Platforms-Azure
IBM
IBM
Join IBM Consulting to shape hybrid cloud and AI journeys for leading companies. Our collaborative environment, supported by IBM technology and Red Hat, empowers you to drive significant client impact. Curiosity and innovation are at our core, fostering a culture of growth and empathy for your career development. You will leverage your unique skills to create impactful solutions.
This role focuses on advising on, developing, and maintaining data engineering solutions within the Azure Cloud ecosystem. You'll be instrumental in designing, building, and operating robust batch and real-time data pipelines, utilizing a range of advanced Azure services.
Key responsibilities include designing and building resilient data pipelines using Azure services like Synapse Analytics, Data Factory, Databricks, and Event Hub. You will develop and manage the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB. Implementation and maintenance of Azure Data Platform components such as ADLS2, Blob Storage, SQLDW, Synapse Analytics (with Spark and SQL), Azure Functions (Python), Azure Purview, and Cosmos DB are crucial.
Utilize open-source technologies like Apache Airflow and dbt, along with Spark/Python or Spark/Scala, to enhance data engineering solutions. Ensure the seamless operation and maintenance of data engineering solutions for efficient batch and real-time data processing. Maintain effective data pipelines and platforms.
Proficiency in Azure services including Azure Synapse Analytics, Azure Data Factory, Azure Databricks, and Event Hub for designing and operating data pipelines is essential. Experience with Azure Data Platform components like ADLS2, Blob Storage, SQLDW, Synapse Analytics (Spark/SQL), Azure Functions (Python), Azure Purview, and Cosmos DB is required.
Familiarity with open-source technologies such as Apache Airflow and dbt, alongside Spark/Python or Spark/Scala, is necessary for supporting data engineering solutions. Experience in developing and maintaining data engineering solutions on Azure Cloud, including designing and operating batch and real-time data pipelines, is a key requirement. Demonstrated experience in designing, building, and operating the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB is also crucial for this role.
IBM
Information Technology & Services