Data Engineer-Data Platforms-Azure
IBM
IBM
Join IBM Consulting and shape the future of hybrid cloud and AI for leading global companies. We foster a collaborative environment where curiosity drives innovation and delivers tangible results. Grow your career in a supportive culture that values your unique contributions and long-term development.
This role focuses on Data Engineering within Azure Data Platforms, where you'll be instrumental in advising on, developing, and maintaining robust data engineering solutions.
If you thrive on tackling complex data challenges and building cutting-edge solutions, we encourage you to explore this opportunity.
Design, construct, and manage real-time and batch data pipelines utilizing key Azure services like Azure Synapse Analytics, Azure Data Factory, Azure Databricks, and Event Hub. You will also develop and maintain the data layer across Azure Synapse Analytics, SQL DW, and Cosmos DB, ensuring efficient data management.
Implement and oversee Azure Data Platform components, including ADLS2, Blob Storage, SQLDW, Synapse Analytics (with Spark and SQL), Azure Functions (Python), Azure Purview, and Cosmos DB. Leverage open-source technologies such as Apache Airflow and dbt, along with Spark (Python or Scala) to enhance data engineering workflows.
Ensure the continuous operation and maintenance of data engineering solutions, guaranteeing seamless processing of both batch and real-time data.
Demonstrate proficiency in Azure services such as Azure Synapse Analytics, Azure Data Factory, Azure Databricks, and Event Hub for building and operating data pipelines. Solid experience with Azure Data Platform components like ADLS2, Blob Storage, SQLDW, Synapse Analytics (Spark/SQL), Azure Functions (Python), Azure Purview, and Cosmos DB is essential.
Possess expertise in open-source technologies, including Apache Airflow and dbt, alongside Spark with Python or Scala, to support data engineering initiatives. Proven experience in developing and maintaining data engineering solutions within the Azure Cloud ecosystem, covering the design, construction, and operation of batch and real-time data pipelines is required.
Experience in designing, building, and operating the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB is a key requirement. You should also be well-versed in developing and maintaining data engineering solutions on Azure, with a focus on pipeline creation and operational excellence.
IBM
Data Analytics