Data Engineer-Data Platforms-Azure
IBM
IBM
Embark on a rewarding career with IBM Consulting, where long-term client relationships and global collaboration are paramount. You'll partner with industry leaders to guide their hybrid cloud and AI strategies. Leveraging robust IBM technology, Red Hat, and support from our strategic partners, you'll possess the tools to drive impactful change and accelerate client success. Curiosity is the engine of achievement at IBM Consulting; we encourage challenging the status quo, exploring novel ideas, and crafting innovative solutions for tangible results. Our growth-oriented and empathetic culture fosters your career development while celebrating your unique contributions.
As a specialized Data Engineer focusing on Data Platforms within the Azure ecosystem, your role will involve advising on, developing, and maintaining sophisticated data engineering solutions. You will be instrumental in designing, constructing, and managing both batch and real-time data pipelines, utilizing a comprehensive suite of Azure services.
Key responsibilities include designing and constructing robust batch and real-time data pipelines using essential Azure services such as Azure Synapse Analytics, Azure Data Factory, Azure DataBricks, and Event Hub. You will also develop and maintain the data layer within Azure Synapse Analytics, SQL DW, and Cosmos DB.
Furthermore, you will implement and manage Azure Data Platform components, including ADLS2, Blob Storage, SQLDW, Synapse Analytics (with Spark and SQL), Azure Functions (with Python), Azure Purview, and Cosmos DB. Proficiency in open-source technologies like Apache Airflow and dbt, alongside Spark with Python or Scala, will be applied to support data engineering solutions. Ensuring the seamless operation and maintenance of data engineering solutions for efficient batch and real-time data processing is crucial.
Demonstrated proficiency in Azure services like Azure Synapse Analytics, Azure Data Factory, Azure DataBricks, and Event Hub for building and operating data pipelines. Practical experience with Azure Data Platform components such as ADLS2, Blob Storage, SQLDW, Synapse Analytics (Spark/SQL), Azure Functions (Python), Azure Purview, and Cosmos DB is expected.
Exposure to and application of open-source technologies including Apache Airflow and dbt, coupled with Spark (Python/Scala), is vital for supporting data engineering initiatives. Proven experience in developing and maintaining data engineering solutions within the Azure Cloud ecosystem, encompassing the design, construction, and operation of batch and real-time data pipelines, is required. Experience in designing, building, and operating the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB is also essential.
IBM
Software