Join Iris Software, a rapidly expanding IT services company recognized among India's Top 25 Best Workplaces in the IT industry. We are seeking a Senior Data Engineer to contribute to cutting-edge projects, leveraging the latest technologies in data and analytics. Our vision is to be our clients' most trusted technology partner, and a preferred destination for top industry professionals. With a global presence, we empower enterprise clients across diverse sectors like financial services, healthcare, and logistics through technology-enabled transformation.

At Iris, we believe in fostering a culture where your potential is recognized and your career journey is shaped by you. We offer a launchpad for growth, providing opportunities for continuous learning, mentorship, and significant impact. Discover a work environment where your voice matters and your contributions create real change.

Design and implement scalable data engineering solutions using PySpark and modern distributed data processing frameworks. Define robust data ingestion, transformation, and processing architectures aligned with business objectives. Architect and optimize Snowflake or Delta Lake on Databricks solutions for enterprise-scale data platforms. Lead the development of high-performance batch and streaming data pipelines, including event-driven architectures using Apache Kafka or Amazon Kinesis. Establish comprehensive monitoring, scheduling, and operational controls for reliable pipeline execution. Drive data quality, validation, reconciliation, and governance practices across all data engineering solutions.

Develop business-focused data products by enhancing data quality, discoverability, and usability. Promote the responsible use of AI-assisted engineering capabilities to boost development productivity and quality. Review data pipeline designs and implementations to ensure adherence to engineering, scalability, and performance standards. Troubleshoot complex data processing, workflow, and streaming platform issues with detailed root cause analysis. Mentor team members on PySpark, Snowflake, Delta Lake, Kafka, Kinesis, Airflow, and data engineering best practices. Collaborate effectively with various teams and stakeholders for end-to-end data platform delivery.

We are looking for a Senior Data Engineer with mandatory expertise in PySpark, Amazon Kinesis, Delta Lake on Databricks, and Databricks Workflows. Proficiency in Snowflake, Apache Kafka, and Apache Airflow is also essential.

Key responsibilities include designing scalable data solutions, leading the implementation of high-performance pipelines, and architecting event-driven data architectures. You will be instrumental in driving data quality, governance, and adopting modern Lakehouse architecture principles. The role also involves mentoring team members and collaborating with stakeholders to ensure successful data platform delivery.

Ideal candidates will demonstrate strong ownership, promote quality-focused engineering, apply analytical thinking to complex challenges, and communicate effectively regarding delivery status and improvement opportunities. Adaptability to evolving technologies and business requirements is crucial.