Azure Data Architect (IN)
HCLTech
HCLTech
Join us as an Azure Databricks Architect, where you'll lead the design, development, and optimization of cutting-edge data platforms and analytics solutions on Microsoft Azure. This role is pivotal in architecting end-to-end data pipelines, ensuring robust data governance, and enabling advanced analytics and AI initiatives.
Your expertise will be crucial in shaping scalable data architectures, building efficient ETL/ELT processes with Azure Databricks and Spark, and designing comprehensive data lake and data warehouse solutions. You will also define data modeling strategies for both batch and real-time scenarios, upholding stringent data security, governance, and compliance standards.
Architect and implement scalable data solutions using Azure services and Azure Databricks. Develop and optimize ETL/ELT pipelines, leveraging Apache Spark for efficient data processing. Design data lake and data warehouse architectures, incorporating Azure Data Lake and Synapse Analytics. Define and implement robust data modeling strategies for batch and real-time data. Ensure adherence to data security, governance, and compliance mandates. Collaborate closely with stakeholders to translate business needs into technical designs. Optimize data workloads for performance and cost-effectiveness. Implement CI/CD practices for data engineering workflows. Provide technical leadership and mentorship to data engineering teams. Integrate advanced analytics, AI, and Machine Learning capabilities into data solutions.
Demonstrated experience with Azure Databricks and Apache Spark (PySpark/Scala). Proficiency in Azure services such as Azure Data Factory, Azure Data Lake Storage, Azure Synapse Analytics, and Azure Event Hubs. Deep understanding of big data architecture and distributed systems. Strong command of SQL, Python, and core data engineering principles. Experience with various data modeling techniques, including star and dimensional modeling. Familiarity with real-time streaming technologies like Kafka or Azure Event Hubs. Knowledge of DevOps principles and CI/CD practices in data environments. Expertise in data quality, testing, and security/governance frameworks. Ability to optimize for cost and performance in cloud data environments.
HCLTech
IT Consulting