Data Engineer-Data Platforms-Google
IBM
IBM
Join IBM Consulting and drive transformative change for leading companies by shaping their hybrid cloud and AI strategies. We foster a culture of curiosity and growth, empowering you to develop innovative solutions and accelerate client impact. This role focuses on designing, building, and maintaining robust data engineering solutions within Google's Cloud ecosystem.
As a specialized Data Engineer, you'll leverage Google's powerful services for both batch and real-time data processing, ensuring efficient data migration and optimal data layer design. Your contributions will be key to our clients' data platform success and innovation.
Architect and develop efficient batch and real-time data pipelines for data warehouses and data lakes using Google Cloud services like DataProc, DataFlow, Pub/Sub, BigQuery, and BigTable.
Build and maintain sophisticated data engineering solutions leveraging Google Cloud Storage, BigTable, BigQuery DataProc with Spark/Hadoop, and Google DataFlow with Apache Beam or Python.
Manage and schedule data platform operations using Google Cloud Scheduler and Cloud Composer (Airflow) to ensure seamless pipeline execution.
Design and implement effective data migration strategies using Google services for secure and efficient data transfer.
Optimize data layers using BigQuery, BigTable, and Cloud Spanner for enhanced data storage and retrieval performance.
Demonstrated experience in designing, building, and maintaining data engineering solutions within the Google Cloud ecosystem, utilizing services such as DataProc, DataFlow, Pub/Sub, BigQuery, BigTable, Cloud Spanner, CloudSQL, and AlloyDB.
Proven ability to develop and manage batch and real-time data pipelines for Data Warehousing and Data Lakes, incorporating open-source technologies like Apache Airflow, dbt, Spark/Python, or Spark/Scala.
Proficiency with Google Cloud Storage, BigTable, BigQuery DataProc with Spark/Hadoop, and Google DataFlow with Apache Beam or Python.
Understanding of data platform management, including scheduling and orchestration using Google Cloud Scheduler and Cloud Composer (Airflow).
Experience in data layer design using BigQuery, BigTable, and Cloud Spanner, focusing on efficient storage and retrieval.
Familiarity with open-source technologies for data pipeline development and management.
IBM
IT Consulting