Data Engineer-Data Platforms-Azure

IBM

Fresher Bengaluru Full Time Hybrid (office + remote)
IBM logo
Posted : today
Actively hiring

Job description

Join IBM Consulting and shape the future of hybrid cloud and AI for leading companies worldwide. You'll collaborate closely with clients, leveraging IBM technology and Red Hat to drive impactful change. Our culture fosters curiosity, innovation, and continuous growth, supporting your long-term career development while valuing your unique contributions.

As a Data Engineer on the Data Platforms team, you will be instrumental in developing and managing sophisticated data engineering solutions within the Azure cloud environment.

Responsibilities

Your core responsibilities will involve designing, constructing, and managing robust batch and real-time data pipelines utilizing a suite of Azure services. This includes building and operating the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB. You'll also implement and maintain key Azure Data Platform components such as ADLS2, Blob Storage, Synapse Analytics with Spark and SQL, Azure Functions with Python, Azure Purview, and Cosmos DB.

Furthermore, you will apply your expertise in open-source technologies like Apache Airflow and dbt, alongside Spark with Python or Scala, to enhance data engineering solutions. Ensuring the seamless operation and maintenance of these data engineering solutions for both batch and real-time processing is crucial.

Qualifications

We are seeking individuals with proven experience in data engineering on the Azure Cloud ecosystem, including the design, development, and operation of batch and real-time data pipelines. A strong understanding of Azure Data Platform components is essential, encompassing ADLS2, Blob Storage, SQLDW, Synapse Analytics (Spark and SQL), Azure Functions (Python), Azure Purview, and Cosmos DB.

Proficiency in open-source technologies such as Apache Airflow and dbt, coupled with Spark (Python or Scala), is required. Experience in designing, building, and operating the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB is also a key requirement. Familiarity with Azure services like Azure Synapse Analytics, Azure Data Factory, Azure DataBricks, and Event Hub for pipeline development is expected.

Essential Skills

AzureData EngineeringData PipelinesPythonSparkAzure Data FactoryAzure DataBricksAzure Synapse AnalyticsAzure Cosmos DBApache AirflowdbtSQL DWADLS2Blob StorageAzure FunctionsAzure PurviewReal-time DataStructured Query LanguageScala Programming Language

Highlights

  • Actively hiring

More Details

RoleData Engineer-Data Platforms-Azure
IndustryInformation Technology & Services
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

IBM logo

IBM

Information Technology & Services

Data Engineer-Data Platforms-Azure at IBM | SkillMX | SkillMX