Iceberg, Doris, Trino
Infosys
Infosys
Embark on a pivotal role shaping modern analytics platforms driven by Iceberg, Doris, and Trino. This position sits at the forefront of data engineering and query performance, empowering teams to extract swift and dependable insights from extensive datasets. It's an ideal opportunity for individuals passionate about tackling intricate data architecture challenges, optimizing distributed systems, and guiding engineers towards robust, scalable solutions.
You will collaborate closely with data engineers, platform teams, and stakeholders to cultivate dependable data products, enhance developer experience, and ensure data accessibility, governance, and performance. If the prospect of working with open table formats, high-concurrency analytical workloads, and building systems that unlock true data usability across an organization excites you, this is a prime environment for growth and purposeful leadership.
Lead the architectural design and implementation of lakehouse and analytics solutions utilizing Iceberg, Doris, and Trino to enable scalable querying and reporting. Design and maintain Iceberg table structures, partitioning strategies, schema evolution patterns, and data lifecycle management, including compaction, snapshots, and retention policies. Build and optimize distributed query workflows within Trino, encompassing connector configuration, query performance tuning, resource governance, and workload management. Develop and refine analytical data models and ingestion patterns with Doris, focusing on high-performance OLAP workloads. Implement resilient batch and stream processing pipelines using Spark, prioritizing correctness, scalability, and cost-effectiveness. Establish performance benchmarks, monitor Service Level Agreements (SLAs), and resolve production issues across compute, storage, and query layers. Promote best practices in data quality, reliability, and operational excellence through automation, comprehensive documentation, and effective runbooks. Mentor engineering team members, conduct thorough design reviews, and drive technical decisions that align with long-term platform objectives.
A Bachelor's or Master's degree in BTECH, MTECH, MCA, MSC, or a related discipline is required. Possess 6-8 years of experience in data engineering, analytics engineering, or the development of distributed data platforms. Demonstrate strong hands-on expertise with Iceberg, covering table design, partitioning, schema evolution, and maintenance operations. Exhibit significant practical experience with Trino for federated/distributed querying, performance optimization, and operational troubleshooting. Showcase robust hands-on experience with Doris, specifically for OLAP use cases, data modeling, and query performance enhancement. Provide proven experience in building production-ready data pipelines using Spark. Maintain a solid understanding of distributed systems, query execution principles, and data storage formats tailored for analytical workloads.
Infosys
IT Consulting