Join Iris Software, recognized as one of India's Top 25 Best Workplaces in the IT industry. We are a rapidly growing IT services company committed to being a trusted technology partner and a platform for professionals to achieve their full potential. With a global presence across India, the U.S.A., and Canada, we empower enterprise clients in financial services, healthcare, transportation & logistics, and professional services through technology-enabled transformation.

Our expertise spans mission-critical applications, leveraging cutting-edge technologies like Application & Product Engineering, Data & Analytics, Cloud, DevOps, Data & MLOps, Quality Engineering, and Business Automation. At Iris, your career is a launchpad for growth, where you own your journey and shape your success.

Design and implement scalable data engineering solutions using PySpark and distributed processing frameworks. Architect data ingestion, transformation, and processing pipelines. Develop and optimize Snowflake or Delta Lake on Databricks for enterprise-scale data platforms. Lead the creation of high-performance batch and streaming data pipelines, including event-driven architectures with Apache Kafka or Amazon Kinesis. Define data streaming standards and processing patterns. Architect workflow orchestration using Apache Airflow or Databricks Workflows, ensuring reliable execution through monitoring and scheduling.

Drive data quality, validation, and governance initiatives. Implement Lakehouse architecture principles, data observability, and platform engineering standards for enhanced scalability and reliability. Develop business-focused data products by improving data quality, discoverability, and usability. Promote the responsible use of AI-assisted engineering to boost productivity and quality. Review designs and implementations for adherence to standards. Troubleshoot complex data issues with root cause analysis. Mentor team members on PySpark, Snowflake, Delta Lake, Kafka, Kinesis, and Airflow best practices. Collaborate with stakeholders for end-to-end data platform delivery.

This senior engineering role requires a strong command of PySpark, Amazon Kinesis, Delta Lake on Databricks, and Databricks Workflows. Expertise in designing scalable data engineering solutions, data ingestion, transformation, and processing architectures is essential. You should be proficient in designing and optimizing Snowflake or Delta Lake on Databricks, and in leading the implementation of high-performance batch and streaming data pipelines. Experience with event-driven architectures using Apache Kafka or Amazon Kinesis, along with defining data streaming standards, is expected. Architecting workflow orchestration with Apache Airflow or Databricks Workflows, and establishing robust monitoring and controls are key responsibilities. A strong understanding of data quality, validation, reconciliation, and governance practices is crucial.

Candidates should be adept at designing solutions based on modern Lakehouse architecture, data observability, and platform engineering standards. Driving the development of business-focused data products by enhancing data quality, discoverability, and usability is important. The ability to promote responsible AI-assisted engineering, review pipeline designs, and troubleshoot complex data processing issues through root cause analysis is required. Excellent collaboration and communication skills are necessary to work with various teams and stakeholders. Mentorship of junior team members on core technologies and best practices is also a key aspect of this role.