Data Engineer-Data Platforms-Google
IBM
IBM
This role focuses on designing, constructing, and maintaining robust data engineering solutions within Google's comprehensive cloud ecosystem. You will leverage various Google services for both batch and real-time data pipelines, manage data migration processes, and architect efficient data layers. Curiosity and a drive for innovation are key to succeeding here, as you'll be encouraged to explore new ideas and create impactful solutions that deliver tangible results for leading companies navigating their hybrid cloud and AI journeys.
Design and develop scalable batch and real-time data pipelines for data warehouses and data lakes using Google Cloud services. Build and maintain data engineering solutions by utilizing Google Cloud Storage, BigTable, BigQuery, DataProc with Spark/Hadoop, and DataFlow with Apache Beam or Python. Ensure efficient data pipeline operations by scheduling and managing the data platform using Google Cloud Scheduler and Cloud Composer (Airflow). Develop and implement seamless data migration solutions between systems. Optimize the data layer for efficient storage and retrieval using services like BigQuery, Big Table, and Cloud Spanner.
Demonstrated experience in designing, building, and maintaining data engineering solutions on Google Cloud, encompassing services like DataProc, DataFlow, PubSub, BigQuery, Big Table, Cloud Spanner, CloudSQL, and AlloyDB. Proven track record in developing and managing batch and real-time data pipelines for Data Warehouses and Datalakes, employing Google services and open-source technologies such as Apache Airflow, dbt, Spark/Python, or Spark/Scala. Proficiency in Google Cloud Storage, BigTable, BigQuery DataProc with Spark and Hadoop, and Google DataFlow with Apache Beam or Python. Understanding of data platform management, including scheduling and orchestration with Google Cloud Scheduler and Cloud Composer (Airflow). Experience with data layer design using BigQuery, Big Table, and Cloud Spanner. Familiarity with open-source technologies is a plus.
IBM
Data Analytics