GCP Senior Data Engineer (IN)
HCLTech
HCLTech
Join our dynamic team as a Senior Data Engineer, specializing in Google Cloud Platform (GCP). This role is pivotal in architecting and implementing robust, scalable data ecosystems that empower data-driven decisions across the enterprise. You will collaborate closely with data scientists, analysts, and business units to translate complex requirements into high-performance data solutions.
Advance your career by designing, building, and optimizing cutting-edge data pipelines and architectures. Ensure the reliability and efficiency of our data platforms while driving innovation in enterprise-wide analytics.
Lead the design, development, and maintenance of scalable, high-performance data pipelines within the GCP environment. Develop and manage modern data architectures for both batch and real-time processing needs, implementing reusable frameworks for standardized data ingestion and transformation.
Utilize key GCP services including Dataflow, BigQuery, Pub/Sub, Cloud Storage, and Cloud SQL/Firestore to build efficient and scalable data integration workflows. Optimize storage and query performance, focusing on cost-effectiveness and scalability.
Implement rigorous data quality checks, validation frameworks, and monitoring mechanisms, ensuring adherence to data governance, security, and compliance standards. Collaborate with stakeholders to understand needs and deliver actionable insights.
Monitor and troubleshoot data pipelines, ensuring high availability and performance with robust logging and alerting. Provide technical leadership and mentorship to junior engineers, driving best practices in coding and architecture. Continuously explore GCP innovations and recommend improvements for system performance.
We seek a Senior Data Engineer with 5-6 years of experience and deep expertise in GCP. A Bachelor's or Master's degree in Computer Science, IT, or a related field is required.
Essential skills include hands-on experience with Dataflow, BigQuery, Pub/Sub, Cloud Storage, Cloud SQL, and Firestore. Proficiency in ETL/ELT processes, data warehousing, and data modeling is crucial, alongside strong programming skills in Python and SQL.
Key competencies involve strong analytical and problem-solving abilities, meticulous attention to detail, and the capacity to work effectively both independently and within cross-functional teams. Excellent communication and stakeholder management skills are also essential.
Preferred qualifications include GCP certifications, experience with Big Data technologies like Hadoop and Spark, exposure to Machine Learning/AI workflows, and familiarity with DevOps practices and CI/CD pipelines.
HCLTech
IT Consulting