Subject Matter Expert (Support&Ops) (IN)
HCLTech
HCLTech
Join our team as a Subject Matter Expert in Support & Operations, focusing on Data Engineering. This role involves applying software engineering best practices to create and manage automated data transformation pipelines. You'll work with diverse storage and computation technologies, ensuring our data architecture is robust and scalable.
The ideal candidate will develop resilient, modular, and cost-effective data pipelines that adapt to evolving needs.
Design, develop, and maintain sophisticated data pipelines and ETL processes leveraging Microsoft Azure services like Azure Data Factory, Synapse, Databricks, and Fabric. You will utilize Azure data storage solutions, including Data Lake Storage Gen 2 and Blob storage, for efficient data organization. Proficiency in SQL DML for querying modern RDBMS such as SQL Server and PostgreSQL is essential.
Further responsibilities include collaborating with architects and stakeholders to deliver high-quality data solutions, optimizing Azure data pipelines for peak performance, and ensuring data integrity through validation. You will also monitor and troubleshoot pipeline issues, stay abreast of industry advancements, and manage the CI/CD process for data solutions. Familiarity with programming languages like C#, Java, JavaScript, or Python is beneficial.
We require a Bachelor’s degree in Computer Science, Engineering, or a related discipline, or equivalent practical experience. Candidates should possess 3-5 years of demonstrated experience as a Data Engineer or in a similar role focused on data and ETL processes. A strong command of Microsoft Azure services is crucial, including Azure Data Factory, Synapse, Databricks, Blob Storage, and Data Lake Gen 2.
Expertise in using SQL DML for efficient RDBMS querying and a solid understanding of software engineering principles applied to data engineering (e.g., CI/CD, version control, testing) are necessary. Proficiency in PySpark, strong problem-solving abilities, and excellent communication skills are also key requirements for this position.
HCLTech
IT Consulting