Databricks - Senior Engineer
Iris Software
Iris Software
Join a leading IT services company recognized as one of India's Top 25 Best Workplaces. We are seeking a Senior Engineer to contribute to innovative technology solutions. This role offers a significant opportunity for professional growth within an award-winning culture that values talent and ambition.
Our vision is to be the most trusted technology partner for our enterprise clients, fostering an environment where top professionals can realize their full potential. With a global presence, we specialize in complex, mission-critical applications across diverse sectors using cutting-edge technologies like Data & Analytics, Cloud, and DevOps.
Architect and implement scalable data engineering solutions leveraging PySpark and modern distributed processing frameworks. Define robust data ingestion, transformation, and processing architectures to meet business and analytical needs. Design and optimize data platforms using Snowflake or Delta Lake on Databricks for enterprise-level data management.
Lead the development of high-performance batch and streaming data pipelines, including event-driven architectures with Apache Kafka or Amazon Kinesis. Establish data streaming standards and scalable processing patterns. Architect workflow orchestration solutions using Apache Airflow or Databricks Workflows, ensuring reliable pipeline execution through comprehensive monitoring and operational controls.
Expertise in PySpark, Apache Kafka, Databricks Workflows, and Delta Lake on Databricks is mandatory. Experience with Snowflake or Amazon Kinesis is also required. Design and implement robust data quality, validation, reconciliation, and governance practices. Drive the development of business-focused data products by enhancing data quality, discoverability, and usability.
Promote the responsible use of AI-assisted engineering for improved development, testing, and documentation. Review data pipeline designs and implementations to ensure adherence to engineering, scalability, and performance standards. Troubleshoot complex data processing and streaming platform issues with detailed root cause analysis. Mentor team members on best practices in data engineering and relevant technologies.
Iris Software
Information Technology & Services