Data Engineer-Data Platforms-Google

IBM

Fresher Bengaluru Full Time Hybrid (office + remote)
IBM logo
Posted : today
Actively hiring

Job description

Join IBM Consulting and shape the future of hybrid cloud and AI for leading companies. This role focuses on designing, building, and maintaining robust data engineering solutions within Google's Cloud ecosystem. You'll leverage cutting-edge Google services to create efficient batch and real-time data pipelines, manage data migration, and optimize data layer design. We foster a culture of curiosity and innovation, encouraging you to challenge norms and create impactful solutions.

As a Data Engineer, you will be instrumental in developing and implementing data strategies that drive significant client impact. Your expertise will be key in transforming how organizations utilize their data, contributing to their digital transformation journeys.

Responsibilities

Design and develop scalable batch and real-time data pipelines using Google services like DataProc, DataFlow, PubSub, BigQuery, and BigTable.

Build and maintain data engineering solutions leveraging Google Cloud Storage, BigTable, BigQuery DataProc with Spark/Hadoop, and Google DataFlow with Apache Beam or Python.

Manage and schedule data platform operations using Google Cloud Scheduler and Cloud Composer (Airflow).

Implement effective data migration strategies to ensure seamless data transfer.

Optimize data layer design using BigQuery, BigTable, and Cloud Spanner for efficient storage and retrieval.

Qualifications

Proven experience designing, building, and maintaining data engineering solutions on Google's Cloud ecosystem.

Expertise in developing and managing batch and real-time data pipelines for Data Warehouse and Datalake using Google services and open-source technologies (e.g., Apache Airflow, dbt, Spark/Python, Spark/Scala).

Proficiency with Google Cloud services including Cloud Storage, BigTable, BigQuery, DataProc, and DataFlow.

Familiarity with data platform management, including scheduling and orchestration tools like Cloud Scheduler and Cloud Composer (Airflow).

Experience in data layer design and optimization using services like BigQuery, BigTable, and Cloud Spanner.

Essential Skills

Data EngineeringData PipelinesBigQueryDataFlowPubSubBigTableCloud StorageSparkPythonHadoopApache BeamCloud ComposerApache AirflowdbtScalaData MigrationCloud SchedulerCloud SpannerCloudSQLAlloyDB

Highlights

  • Actively hiring

More Details

RoleData Engineer-Data Platforms-Google
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

IBM logo

IBM

IT Consulting

Data Engineer-Data Platforms-Google at IBM | SkillMX | SkillMX