Senior Data Engineer (Noida, UP, India)
Iris Software
Iris Software
Join a leading IT services company recognized as one of India’s Top 25 Best Workplaces in the IT industry. We offer an award-winning culture that values your talent and ambitions, providing a launchpad for your career growth. Our vision is to be the most trusted technology partner for our clients and the first choice for top industry professionals.
With a global presence across India, the U.S.A., and Canada, we empower enterprise clients through technology-enabled transformations. Our expertise spans financial services, healthcare, transportation & logistics, and professional services, focusing on complex, mission-critical applications utilizing cutting-edge technologies like Data & Analytics, Cloud, DevOps, and AI.
At Iris, we believe in "Build Your Future. Own Your Journey." You'll have ownership of your career path with ample opportunities for growth, continuous learning, and mentorship. We foster a culture where your potential is recognized, your voice is heard, and your work makes a significant impact.
Design and implement scalable data engineering solutions using PySpark and modern distributed data processing frameworks. Define robust data ingestion, transformation, and processing architectures to meet business objectives.
Develop and optimize data platforms on Snowflake or Delta Lake on Databricks, focusing on enterprise-scale solutions. Lead the implementation of high-performance batch and streaming data pipelines, ensuring reliability and efficiency.
Architect event-driven data architectures using Apache Kafka or Amazon Kinesis, and establish data streaming standards. Design workflow orchestration solutions with Apache Airflow or Databricks Workflows, ensuring reliable pipeline execution through robust monitoring and operational controls.
Drive data quality, validation, reconciliation, and governance across all data engineering solutions. Implement modern Lakehouse architecture principles, data observability practices, and platform engineering standards to enhance scalability, reliability, and operational visibility.
Champion the development of business-focused data products by improving data quality, discoverability, usability, and documentation. Promote the responsible use of AI-assisted engineering to boost development productivity and quality. Review data pipeline designs and implementations to ensure adherence to engineering, scalability, and performance standards.
Troubleshoot complex data processing, workflow, and streaming platform issues through detailed root cause analysis. Mentor team members on key technologies and data engineering best practices. Collaborate effectively with cross-functional teams and stakeholders to ensure seamless end-to-end data platform delivery.
This senior role requires 7-8 years of extensive experience in data engineering, with a strong command of mandatory skills including Amazon Kinesis, Apache Spark, Data Quality & Validation, PySpark, SQL, Apache Airflow, and Delta Lake on Databricks.
Key responsibilities involve designing scalable data solutions, defining ingestion and transformation architectures, and optimizing platforms like Snowflake or Delta Lake on Databricks. You will lead the implementation of batch and streaming data pipelines, design event-driven architectures, and architect workflow orchestration solutions.
Emphasis is placed on driving data quality, validation, and governance. Candidates should be proficient in implementing Lakehouse architecture principles, data observability, and platform engineering standards. Experience in developing business-focused data products and promoting AI-assisted engineering is highly desirable.
Strong analytical thinking, adaptability, and effective communication are essential. You will be responsible for troubleshooting complex data issues, mentoring team members, and collaborating with stakeholders for successful platform delivery. A commitment to quality-focused engineering and continuous improvement is expected.
Iris Software
Information Technology & Services