Data Engineer-Data Platforms-Azure
IBM
IBM
Join IBM Consulting and be part of a team that builds lasting client relationships worldwide. You'll collaborate with leading companies to guide their hybrid cloud and AI strategies. Leveraging IBM's technology, Red Hat, and support from strategic partners, you'll drive significant impact and positive change. At IBM Consulting, curiosity is key to success. We foster an environment where challenging the status quo, exploring novel ideas, and developing innovative solutions are encouraged to achieve tangible results. Our growth-oriented and empathetic culture prioritizes your long-term career development while celebrating your unique talents and experiences.
As a Data Engineer focused on Data Platforms within Azure, your role involves advising on, developing, and maintaining robust data engineering solutions across the Azure Cloud ecosystem. You will be instrumental in designing, building, and operating both batch and real-time data pipelines leveraging a variety of Azure services.
Key responsibilities include: - Designing and building comprehensive data pipelines utilizing Azure services like Azure Synapse Analytics, Azure Data Factory, Azure Databricks, and Event Hub. - Developing and managing the data layer on platforms such as Azure Synapse Analytics, SQL DW, and Cosmos DB. - Implementing and maintaining critical Azure Data Platform components, including ADLS2, Blob Storage, SQLDW, Synapse Analytics (with Spark and SQL), Azure Functions (with Python), Azure Purview, and Cosmos DB. - Applying expertise in open-source technologies like Apache Airflow and dbt, alongside Spark/Python or Spark/Scala, to enhance data engineering solutions. - Ensuring the seamless operation and maintenance of data engineering solutions for efficient batch and real-time data processing.
We are seeking experienced professionals with a strong background in data engineering on the Azure platform. Your expertise should encompass:
- Proven experience designing, building, and operating batch and real-time data pipelines within the Azure Cloud ecosystem. - In-depth knowledge of Azure Data Platform components such as ADLS2, Blob Storage, SQLDW, Synapse Analytics (including Spark and SQL capabilities), Azure Functions (with Python), Azure Purview, and Cosmos DB. - Proficiency in utilizing Azure services like Azure Synapse Analytics, Azure Data Factory, Azure Databricks, and Event Hub for pipeline development. - Familiarity with open-source technologies including Apache Airflow and dbt, as well as Spark/Python or Spark/Scala, to support data engineering initiatives. - Demonstrated experience in designing, building, and operating the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB.
IBM
IT Services and Consulting