Data Engineer-Data Platforms-Google

IBM

Fresher Bengaluru Full Time Hybrid (office + remote)
IBM logo
Posted : today
Actively hiring

Job description

Join our dynamic team and contribute to shaping the future of hybrid cloud and AI journeys for leading companies worldwide. As a Data Engineer focused on Google's data platforms, you will play a crucial role in designing, building, and maintaining robust data engineering solutions within the Google Cloud ecosystem. This position requires a deep understanding of Google services for managing both batch and real-time data pipelines, facilitating data migration, and architecting efficient data layers.

At IBM Consulting, we foster a culture of curiosity and innovation, empowering you to challenge conventions and develop groundbreaking solutions that drive tangible client impact. We are committed to your long-term career development, valuing your unique contributions within an empathetic and growth-oriented environment.

Responsibilities

Design and develop sophisticated batch and real-time data pipelines for Data Warehouses and Datalakes utilizing Google services like DataProc, DataFlow, PubSub, BigQuery, and Big Table.

Construct and maintain data engineering solutions leveraging Google Cloud Storage, BigTable, BigQuery DataProc (with Spark and Hadoop), and Google DataFlow (with Apache Beam or Python).

Manage the data platform efficiently through scheduling and orchestration tools such as Google Cloud Scheduler and Cloud Composer (Airflow).

Implement and refine data migration strategies using Google services to ensure seamless data transfers between various systems.

Design and optimize the data layer using services like BigQuery, Big Table, and Cloud Spanner for superior data storage and retrieval performance.

Qualifications

Demonstrate expertise in designing, building, and maintaining data engineering solutions within Google's Cloud ecosystem, including services such as Google DataProc, DataFlow, PubSub, BigQuery, Big Table, Cloud Spanner, CloudSQL, and AlloyDB.

Possess proven experience in developing and managing batch and real-time data pipelines for Data Warehouses and Datalakes, incorporating Google services and open-source technologies like Apache Airflow, dbt, Spark/Python, or Spark/Scala.

Show proficiency with Google Cloud Storage, BigTable, BigQuery DataProc (with Spark and Hadoop), and Google DataFlow (with Apache Beam or Python) for building and maintaining data engineering solutions.

Exhibit knowledge of data platform management, including scheduling and orchestrating operations via Google Cloud Scheduler and Cloud Composer (Airflow) for efficient pipeline execution.

Understand data layer design principles and have experience with services such as BigQuery, Big Table, and Cloud Spanner for optimized data storage and retrieval.

Be familiar with open-source technologies like Apache Airflow, dbt, Spark/Python, or Spark/Scala for data pipeline development and management.

Have practical experience in developing and implementing data migration solutions using Google services for seamless data transfer.

Show experience with Cloud Composer (Airflow) for effective data platform scheduling and management.

Essential Skills

Google CloudData EngineeringData PipelinesBigQueryDataFlowPubSubBigTableSparkPythonHadoopApache BeamCloud SchedulerCloud ComposerAirflowCloud SpannerCloudSQLAlloyDBdbtScalaData MigrationData WarehouseDatalake

Highlights

  • Actively hiring

More Details

RoleData Engineer-Data Platforms-Google
Employment TypeFull Time, Hybrid (office + remote)

About the Company

IBM logo

IBM

IT Consulting

Data Engineer-Data Platforms-Google at IBM | SkillMX | SkillMX