Data Engineer-Data Platforms-Google

IBM

Fresher Bengaluru Full Time Hybrid (office + remote)
IBM logo
Posted : today
Actively hiring

Job description

Embark on a career with IBM Consulting, where you'll cultivate lasting client relationships and collaborate globally. You will assist leading organizations across various sectors in their hybrid cloud and AI transformations. Leveraging IBM's robust technology, Red Hat, and support from strategic partners, you'll be equipped to drive significant change and accelerate client success. Curiosity is key at IBM Consulting; we encourage challenging the status quo, exploring novel ideas, and crafting innovative solutions that deliver tangible results. Our culture fosters growth and empathy, prioritizing your long-term career development while celebrating your unique talents and experiences.

Responsibilities

As a Data Engineer focused on Google's data platforms, your core role involves designing, building, and maintaining robust data engineering solutions within the Google Cloud ecosystem. You will expertly utilize Google's suite of services to create efficient batch and real-time data pipelines, manage data migrations, and architect sophisticated data layers. Key responsibilities include:

- Designing and developing batch and real-time data pipelines for Data Warehouses and Datelakes using Google services like DataProc, DataFlow, PubSub, BigQuery, and BigTable. - Building and maintaining data engineering solutions with Google Cloud Storage, BigTable, BigQuery DataProc (Spark/Hadoop), and Google DataFlow (Apache Beam/Python). - Managing data platforms through Google Cloud Scheduler and Cloud Composer (Airflow) for optimized pipeline operations. - Developing and implementing effective data migration strategies. - Optimizing data layers using BigQuery, BigTable, and Cloud Spanner for efficient data handling.

Qualifications

This role demands extensive experience within Google's cloud environment, focusing on designing, building, and maintaining data engineering solutions. A strong understanding of services such as Google DataProc, DataFlow, PubSub, BigQuery, BigTable, Cloud Spanner, CloudSQL, and AlloyDB is essential.

You should possess proven experience in developing and managing batch and real-time data pipelines for Data Warehouses and Datelakes, utilizing Google services alongside open-source technologies like Apache Airflow, dbt, Spark (Python/Scala). Proficiency in Google Cloud Storage, BigTable, BigQuery DataProc (Spark/Hadoop), and Google DataFlow (Apache Beam/Python) is critical.

Familiarity with scheduling and managing data platforms using Google Cloud Scheduler and Cloud Composer (Airflow) is expected. Experience in data layer design with BigQuery, BigTable, and Cloud Spanner, as well as developing data migration solutions using Google services, will be highly valued. Expertise in open-source technologies such as Apache Airflow, dbt, Spark/Python, or Spark/Scala is also beneficial.

Essential Skills

Google CloudData EngineeringData PipelinesBigQueryDataFlowPubSubBigTableSparkHadoopPythonApache BeamCloud SchedulerCloud ComposerApache AirflowCloud StorageData MigrationCloud SpannerCloud SQLAlloyDBScaladbt

Highlights

  • Actively hiring

More Details

RoleData Engineer-Data Platforms-Google
IndustryIT Services and Consulting
DepartmentData Engineering, Software Development
Employment TypeFull Time, Hybrid (office + remote)

About the Company

IBM logo

IBM

IT Services and Consulting

Data Engineer-Data Platforms-Google at IBM | SkillMX | SkillMX