Azure Senior Data Engineer (IN)
HCLTech
HCLTech
Seeking a skilled Databricks Engineer to architect, build, and refine data solutions on the Microsoft Azure platform. This role is pivotal for driving data-informed strategies through the creation of robust data pipelines, seamless integration of diverse data sources, and delivery of impactful analytics solutions aligned with business goals.
We are looking for a professional to build scalable data solutions using Azure Databricks and the broader Azure ecosystem. The ideal candidate will be adept at developing and managing Azure Data Factory pipelines for efficient data ingestion and transformation, as well as integrating various data types into Azure Data Lake or Analytics platforms. Developing ETL processes to support comprehensive analytics and reporting needs is also a key aspect of this position.
Key responsibilities include designing and implementing scalable data solutions leveraging Azure Databricks and the Azure ecosystem. You will be responsible for building and managing Azure Data Factory (ADF) pipelines to ensure efficient data ingestion and transformation. Integrating structured and unstructured data from multiple sources into Azure Data Lake / Analytics platforms is crucial.
Develop robust ETL processes to support analytics and reporting requirements. Collaborate closely with cross-functional teams to deliver high-quality, impactful data solutions. Translate complex business requirements into clear technical designs and implement scalable, resilient architectures. Ensure optimal performance, data quality, and reliability across all data pipelines. Support deployment and version control utilizing Azure DevOps.
Mandatory skills encompass strong expertise in the Microsoft Azure Stack, including T-SQL, SSIS, SSAS, and SSRS. Hands-on experience is required with Azure Data Factory (ADF), Azure Databricks, Azure Synapse Analytics, Azure DevOps, and Azure Storage/Data Lake. A solid understanding of ETL processes and data integration principles is essential, alongside proficiency in SQL and relational databases. Strong analytical and problem-solving capabilities are vital for success.
Preferred skills include experience with Python scripting and Spark APIs, exposure to client-facing roles and stakeholder interactions, and familiarity with Agile/Scrum methodologies. A Bachelor's degree in Computer Science, Information Technology, or a related field is required. Azure Data Engineer Associate or Microsoft Certified Azure Data & AI Associate certification is considered a plus.
HCLTech
IT Consulting