Data Engineer-Data Platforms-Azure

IBM

Fresher Bengaluru Full Time Hybrid (office + remote)
IBM logo
Posted : today
Actively hiring

Job description

Join IBM Consulting and become a key player in shaping the future of hybrid cloud and AI for leading companies worldwide. As a Data Engineer specializing in Azure Data Platforms, you will be instrumental in designing, developing, and maintaining advanced data engineering solutions within the Azure cloud ecosystem. This role offers a dynamic environment where curiosity, innovation, and collaboration drive success, fostering your long-term career growth.

Your expertise will be crucial in building and operating robust batch and real-time data pipelines. You'll leverage cutting-edge Azure services and open-source technologies to create impactful data solutions. We encourage a culture of challenging the status quo and exploring new ideas to deliver tangible results for our clients.

Responsibilities

Design, build, and maintain batch and real-time data pipelines utilizing Azure services like Azure Synapse Analytics, Azure Data Factory, and Azure Databricks. Develop and manage the data layer on Azure Synapse Analytics, SQL DW, and Cosmos DB. Implement and administer core Azure Data Platform components, including ADLS2, Blob Storage, Synapse Analytics (Spark & SQL), Azure Functions (Python), Azure Purview, and Cosmos DB. Apply proficiency in open-source technologies such as Apache Airflow and dbt, alongside Spark with Python or Scala, to support data engineering initiatives. Ensure the continuous and seamless operation of data engineering solutions for effective batch and real-time data processing.

Qualifications

Demonstrated experience in data engineering on the Azure Cloud, encompassing the design, construction, and operation of batch and real-time data pipelines. Proven ability to work with Azure Data Platform components, including ADLS2, Blob Storage, SQLDW, Synapse Analytics (Spark & SQL), Azure Functions (Python), Azure Purview, and Cosmos DB. Familiarity with open-source technologies like Apache Airflow and dbt, coupled with expertise in Spark/Python or Spark/Scala. Experience designing, building, and operating data layers within Azure Synapse Analytics, SQL DW, and Cosmos DB. Proficiency with Azure services such as Azure Synapse Analytics, Azure Data Factory, Azure Databricks, and Event Hub for pipeline development.

Essential Skills

AzureData EngineeringData PipelinesPythonSparkSQLApache AirflowAzure Data FactoryAzure Synapse AnalyticsDatabricksAzure Cosmos DBAzure FunctionsScala

Good to Have

dbtAzure PurviewBlob StorageADLS2

Highlights

  • Actively hiring

More Details

RoleData Engineer-Data Platforms-Azure
IndustryInformation Technology & Services, Software Development
DepartmentData Engineering, Software Development
Employment TypeFull Time, Hybrid (office + remote)

About the Company

IBM logo

IBM

Information Technology & Services

Data Engineer-Data Platforms-Azure at IBM | SkillMX | SkillMX