Data Engineer-Data Platforms-Azure
IBM
IBM
Embark on a fulfilling career journey with IBM Consulting, where innovation thrives on collaborative, long-term client partnerships. You will empower leading organizations across diverse sectors to navigate their hybrid cloud and AI transformations. Leveraging robust IBM technology, Red Hat, and support from our strategic partners, you will possess the essential tools to drive significant impact and accelerate client success. At IBM Consulting, curiosity is the cornerstone of achievement. We foster an environment that encourages questioning the status quo, exploring novel concepts, and developing groundbreaking solutions that yield tangible results. Our growth-oriented and empathetic culture prioritizes your continuous career development while celebrating your unique proficiencies and perspectives.
As a Data Engineer focused on Data Platforms within the Azure ecosystem, your role involves advising on, developing, and maintaining advanced data engineering solutions. You will be instrumental in designing, building, and managing both batch and real-time data pipelines utilizing a range of Azure services.
Key contributions include: - Crafting and deploying robust batch and real-time data pipelines using Azure Synapse Analytics, Azure Data Factory, Azure Databricks, and Event Hub. - Developing and optimizing the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB. - Implementing and overseeing Azure Data Platform components such as ADLS2, Blob Storage, SQLDW, Synapse Analytics (with Spark and SQL), Azure Functions (with Python), Azure Purview, and Cosmos DB. - Applying expertise in open-source technologies like Apache Airflow and dbt, alongside Spark/Python or Spark/Scala, to enhance data engineering initiatives. - Ensuring the smooth operation and maintenance of data engineering solutions for uninterrupted batch and real-time data processing.
We are seeking individuals with a strong command of the Azure toolset, including proficiency in Azure Synapse Analytics, Azure Data Factory, Azure Databricks, and Event Hub for designing, building, and operating data pipelines. Essential experience includes working with Azure Data Platform components like ADLS2, Blob Storage, SQLDW, Synapse Analytics (Spark/SQL), Azure Functions (Python), Azure Purview, and Cosmos DB.
Familiarity with open-source technologies such as Apache Airflow and dbt, coupled with Spark/Python or Spark/Scala, is crucial for supporting data engineering solutions. Demonstrated experience in developing and maintaining data engineering solutions on the Azure Cloud ecosystem, covering the design, build, and operation of batch and real-time data pipelines, is required. Experience in designing, building, and operating the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB is also a key requirement.
IBM
Information Technology & Services