Data Engineer-Data Platforms-Google

IBM

Fresher Bengaluru Full Time Hybrid (office + remote)
IBM logo
Posted : today
Actively hiring

Job description

Join IBM Consulting and shape the future of hybrid cloud and AI for leading companies worldwide. Our collaborative environment, fueled by curiosity and innovation, empowers you to challenge norms and create impactful solutions. We prioritize your long-term career growth, valuing your unique contributions.

As a Data Engineer specializing in Google's data platforms, you'll be instrumental in designing, building, and maintaining robust data engineering solutions within the Google Cloud ecosystem. This role demands expertise in leveraging various Google services for both batch and real-time data pipelines, data migration strategies, and effective data layer design.

Responsibilities

Key responsibilities include designing and developing batch and real-time data pipelines using Google services like DataProc, DataFlow, PubSub, BigQuery, and BigTable. You will build and maintain data engineering solutions utilizing Google Cloud Storage, BigTable, BigQuery DataProc with Spark and Hadoop, and Google DataFlow with Apache Beam or Python.

You'll also schedule and manage the data platform via Google Cloud Scheduler and Cloud Composer (Airflow), implement data migration solutions for seamless system transfers, and optimize the data layer using BigQuery, Big Table, and Cloud Spanner for efficient data handling.

Qualifications

This role requires demonstrated experience in designing, building, and maintaining data engineering solutions on Google's Cloud ecosystem, including services like Google DataProc, DataFlow, PubSub, BigQuery, Big Table, Cloud Spanner, CloudSQL, and AlloyDB.

Candidates should have experience developing and managing batch/real-time data pipelines for Data Warehouses and Datelakes using Google services and open-source technologies like Apache Airflow, dbt, Spark/Python, or Spark/Scala. Proficiency with Google Cloud Storage, BigTable, BigQuery DataProc (Spark/Hadoop), and Google DataFlow (Apache Beam/Python) is essential. Experience with data layer design using BigQuery, Big Table, and Cloud Spanner, along with data migration solutions, is also expected.

Essential Skills

Google CloudData PipelinesBigQueryData EngineeringPythonSpark

Good to Have

HadoopScalaApache AirflowData MigrationData Platform ManagementSchedulingOpen SourceApache BeamBigTableReal-time DataCloud StoragedbtCloud Composer

Highlights

  • Actively hiring

More Details

RoleData Engineer-Data Platforms-Google
IndustryData Analytics
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

IBM logo

IBM

Data Analytics

Data Engineer-Data Platforms-Google at IBM | SkillMX | SkillMX