Data Engineer-Data Platforms-Azure
IBM
IBM
Join IBM Consulting and contribute to long-term client relationships, driving hybrid cloud and AI transformations worldwide. You'll collaborate with leading companies, leveraging IBM technology and Red Hat expertise to deliver impactful solutions. Our culture fosters curiosity, innovation, and continuous growth, supporting your career development and valuing your unique contributions.
This role focuses on Data Engineering within Data Platforms on Azure, requiring expertise in designing, developing, and maintaining robust data engineering solutions on the Azure Cloud ecosystem.
As a Data Engineer on Azure Data Platforms, you will be instrumental in designing, building, and operating efficient batch and real-time data pipelines. This includes developing the data layer using Azure Synapse Analytics, SQL DW, and Cosmos DB, and implementing key Azure Data Platform components like ADLS2, Blob Storage, Synapse Analytics with Spark and SQL, Azure Functions with Python, Azure Purview, and Cosmos DB.
Your responsibilities extend to utilizing open-source technologies such as Apache Airflow and dbt, alongside Spark with Python or Scala, to enhance data engineering solutions. Ensuring seamless data processing operations for both batch and real-time workloads is paramount. You will leverage your proficiency in the Azure toolset to maintain and optimize data engineering infrastructure.
We are looking for a Data Engineer with demonstrated experience in Azure cloud services, specifically within data platforms. Essential expertise includes designing, building, and operating batch and real-time data pipelines using Azure services like Azure Synapse Analytics, Azure Data Factory, Azure DataBricks, and Event Hub.
Proficiency in developing the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB is crucial. You should have hands-on experience with Azure Data Platform components, including ADLS2, Blob Storage, SQLDW, Synapse Analytics with Spark and SQL, Azure Functions with Python, Azure Purview, and Cosmos DB. Familiarity with open-source technologies like Apache Airflow and dbt, coupled with Spark/Python or Spark/Scala, is highly valued for supporting advanced data engineering solutions.
IBM
Information Technology & Services