Data Engineer-Data Platforms-Google
IBM
IBM
IBM Consulting is seeking a skilled Data Engineer to join our FutureNow Centers, contributing to client success in hybrid cloud and AI transformations. This role focuses on leveraging Google's data platforms to design, build, and maintain robust data engineering solutions. You will collaborate with leading companies across various industries, driving innovation and delivering impactful results. Our culture fosters curiosity, innovation, and continuous learning, supporting your long-term career development and valuing your unique contributions.
Design and develop scalable batch and real-time data pipelines for data warehouses and data lakes using Google Cloud services like DataProc, DataFlow, Pub/Sub, BigQuery, and BigTable. Develop comprehensive data engineering solutions utilizing Google Cloud Storage, BigTable, BigQuery, DataProc (Spark/Hadoop), and DataFlow (Apache Beam/Python). Manage and schedule data platform operations using Google Cloud Scheduler and Cloud Composer (Airflow) for optimal pipeline efficiency. Implement effective data migration strategies using Google services to ensure seamless data transfer between systems. Design and optimize data layers with Google services such as BigQuery, BigTable, and Cloud Spanner for efficient data storage and retrieval.
Proven expertise in designing, building, and maintaining data engineering solutions within the Google Cloud ecosystem, including services like DataProc, DataFlow, Pub/Sub, BigQuery, BigTable, Cloud Spanner, Cloud SQL, and AlloyDB. Demonstrated experience in developing and managing batch and real-time data pipelines for Data Warehouses and Data Lakes, utilizing Google services and open-source technologies such as Apache Airflow, dbt, Spark/Python, or Spark/Scala. Proficiency in working with Google Cloud Storage, BigTable, BigQuery DataProc with Spark and Hadoop, and Google DataFlow with Apache Beam or Python for building data engineering solutions. Solid understanding of data platform management, including scheduling and managing operations with Google Cloud Scheduler and Cloud Composer (Airflow). Experience in data layer design and optimization using Google services like BigQuery, BigTable, and Cloud Spanner. Bachelor's Degree is required; a Master's Degree is preferred.
IBM
Management Consulting