Data Engineer-Data Platforms-Google

IBM

Fresher Bengaluru Full Time Hybrid (office + remote)
IBM logo
Posted : today
Actively hiring

Job description

Join IBM Consulting and drive transformative change for leading companies by shaping their hybrid cloud and AI strategies. We foster a culture of curiosity and growth, empowering you to develop innovative solutions and accelerate client impact. This role focuses on designing, building, and maintaining robust data engineering solutions within Google's Cloud ecosystem.

As a specialized Data Engineer, you'll leverage Google's powerful services for both batch and real-time data processing, ensuring efficient data migration and optimal data layer design. Your contributions will be key to our clients' data platform success and innovation.

Responsibilities

Architect and develop efficient batch and real-time data pipelines for data warehouses and data lakes using Google Cloud services like DataProc, DataFlow, Pub/Sub, BigQuery, and BigTable.

Build and maintain sophisticated data engineering solutions leveraging Google Cloud Storage, BigTable, BigQuery DataProc with Spark/Hadoop, and Google DataFlow with Apache Beam or Python.

Manage and schedule data platform operations using Google Cloud Scheduler and Cloud Composer (Airflow) to ensure seamless pipeline execution.

Design and implement effective data migration strategies using Google services for secure and efficient data transfer.

Optimize data layers using BigQuery, BigTable, and Cloud Spanner for enhanced data storage and retrieval performance.

Qualifications

Demonstrated experience in designing, building, and maintaining data engineering solutions within the Google Cloud ecosystem, utilizing services such as DataProc, DataFlow, Pub/Sub, BigQuery, BigTable, Cloud Spanner, CloudSQL, and AlloyDB.

Proven ability to develop and manage batch and real-time data pipelines for Data Warehousing and Data Lakes, incorporating open-source technologies like Apache Airflow, dbt, Spark/Python, or Spark/Scala.

Proficiency with Google Cloud Storage, BigTable, BigQuery DataProc with Spark/Hadoop, and Google DataFlow with Apache Beam or Python.

Understanding of data platform management, including scheduling and orchestration using Google Cloud Scheduler and Cloud Composer (Airflow).

Experience in data layer design using BigQuery, BigTable, and Cloud Spanner, focusing on efficient storage and retrieval.

Familiarity with open-source technologies for data pipeline development and management.

Essential Skills

Google Cloud PlatformData EngineeringData PipelinesBigQueryDataFlowPub/SubBigTableDataProcCloud StorageCloud ComposerApache AirflowSparkPythonHadoopApache BeamScaladbtCloud SpannerCloud SQLAlloyDBData MigrationScheduling

Highlights

  • Actively hiring

More Details

RoleData Engineer-Data Platforms-Google
DepartmentData Engineering
Employment TypeFull Time, Hybrid (office + remote)

About the Company

IBM logo

IBM

IT Consulting

Data Engineer-Data Platforms-Google at IBM | SkillMX | SkillMX