Data Engineer-Data Platforms-Azure
IBM
IBM
Join IBM Consulting and embark on a career focused on building lasting client relationships and driving global collaboration. You will partner with leading organizations across various industries, guiding them through their hybrid cloud and AI transformations. Leveraging robust IBM technology, Red Hat, and support from our strategic partners, you'll possess the resources to enact significant change and accelerate client success. At IBM Consulting, curiosity is key to achievement. We foster an environment that encourages questioning the status quo, exploring novel concepts, and developing innovative solutions that yield tangible outcomes. Our growth-oriented and empathetic culture prioritizes your long-term career advancement while celebrating your distinct skills and experiences.
As a Data Engineer within our Data Platforms team on Azure, you will be instrumental in advising on, developing, and maintaining sophisticated data engineering solutions within the Azure Cloud ecosystem.
Your core responsibilities will include:
- Designing and constructing robust batch and real-time data pipelines utilizing key Azure services like Azure Synapse Analytics, Azure Data Factory, Azure Databricks, and Event Hub. - Developing and managing the data layer on platforms such as Azure Synapse Analytics, SQL DW, and Cosmos DB. - Implementing and overseeing Azure Data Platform components, including ADLS2, Blob Storage, SQLDW, Synapse Analytics (with Spark and SQL), Azure Functions (with Python), Azure Purview, and Cosmos DB. - Applying expertise in open-source technologies such as Apache Airflow and dbt, alongside Spark/Python or Spark/Scala, to enhance data engineering solutions. - Ensuring the seamless operation and maintenance of data engineering solutions for efficient batch and real-time data processing.
We are seeking individuals with proven experience in:
- Designing, building, and operating batch and real-time data pipelines using Azure services like Azure Synapse Analytics, Azure Data Factory, Azure Databricks, and Event Hub. - Developing and managing the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB. - Working with Azure Data Platform components, including ADLS2, Blob Storage, SQLDW, Synapse Analytics with Spark and SQL, Azure Functions with Python, Azure Purview, and Cosmos DB. - Leveraging open-source technologies such as Apache Airflow and dbt, combined with proficiency in Spark/Python or Spark/Scala for data engineering solutions. - Experience in data engineering within the Azure Cloud ecosystem, encompassing the full lifecycle of pipeline development and maintenance.
IBM
Technology