Join Iris Software, recognized as one of India's Top 25 Best Workplaces in the IT industry. We are a rapidly growing IT services company committed to being a trusted technology partner and fostering the growth of top industry professionals. With a global presence across India, the U.S.A., and Canada, we empower enterprise clients in financial services, healthcare, transportation & logistics, and professional services through technology-enabled transformations. Our expertise spans mission-critical applications, leveraging cutting-edge technologies in Application & Product Engineering, Data & Analytics, Cloud, DevOps, Data & MLOps, Quality Engineering, and Business Automation.

As a Senior Engineer, you will architect and implement scalable data engineering solutions using PySpark and modern distributed data processing frameworks. Key responsibilities include designing data ingestion, transformation, and processing architectures, optimizing Snowflake or Delta Lake on Databricks, and leading the implementation of high-performance batch and streaming data pipelines. You will also design event-driven architectures using Apache Kafka or Amazon Kinesis, establish monitoring and operational controls, and drive data quality, validation, and governance practices. Promoting the responsible use of AI-assisted engineering capabilities to enhance development productivity is also a core aspect of this role. Additionally, you will review pipeline designs, troubleshoot complex issues, and mentor team members on best practices.

This role requires strong ownership and effective collaboration with various teams and business stakeholders. You should demonstrate quality-focused engineering through proactive validation and continuous improvement, applying strong analytical thinking to complex data engineering challenges. Adaptability in managing evolving technologies and business requirements is crucial. Excellent communication skills are essential for reporting delivery status, risks, dependencies, and improvement opportunities. A deep understanding of Databricks Workflows, Apache Spark, Data Quality & Validation, PySpark, SQL, Apache Kafka, and Snowflake is mandatory. Experience with Delta Lake, Amazon Kinesis, Apache Airflow, Data Observability, Platform Engineering, and AI-assisted Engineering will be highly beneficial.