Iceberg, Doris, Trino

Infosys

6–8 yrs Bengaluru Full Time Work from office
Infosys logo
Posted : today
Actively hiring

Job description

Embark on a pivotal role shaping modern analytics platforms driven by Iceberg, Doris, and Trino. This position sits at the forefront of data engineering and query performance, empowering teams to extract swift and dependable insights from extensive datasets. It's an ideal opportunity for individuals passionate about tackling intricate data architecture challenges, optimizing distributed systems, and guiding engineers towards robust, scalable solutions.

You will collaborate closely with data engineers, platform teams, and stakeholders to cultivate dependable data products, enhance developer experience, and ensure data accessibility, governance, and performance. If the prospect of working with open table formats, high-concurrency analytical workloads, and building systems that unlock true data usability across an organization excites you, this is a prime environment for growth and purposeful leadership.

Responsibilities

Lead the architectural design and implementation of lakehouse and analytics solutions utilizing Iceberg, Doris, and Trino to enable scalable querying and reporting. Design and maintain Iceberg table structures, partitioning strategies, schema evolution patterns, and data lifecycle management, including compaction, snapshots, and retention policies. Build and optimize distributed query workflows within Trino, encompassing connector configuration, query performance tuning, resource governance, and workload management. Develop and refine analytical data models and ingestion patterns with Doris, focusing on high-performance OLAP workloads. Implement resilient batch and stream processing pipelines using Spark, prioritizing correctness, scalability, and cost-effectiveness. Establish performance benchmarks, monitor Service Level Agreements (SLAs), and resolve production issues across compute, storage, and query layers. Promote best practices in data quality, reliability, and operational excellence through automation, comprehensive documentation, and effective runbooks. Mentor engineering team members, conduct thorough design reviews, and drive technical decisions that align with long-term platform objectives.

Qualifications

A Bachelor's or Master's degree in BTECH, MTECH, MCA, MSC, or a related discipline is required. Possess 6-8 years of experience in data engineering, analytics engineering, or the development of distributed data platforms. Demonstrate strong hands-on expertise with Iceberg, covering table design, partitioning, schema evolution, and maintenance operations. Exhibit significant practical experience with Trino for federated/distributed querying, performance optimization, and operational troubleshooting. Showcase robust hands-on experience with Doris, specifically for OLAP use cases, data modeling, and query performance enhancement. Provide proven experience in building production-ready data pipelines using Spark. Maintain a solid understanding of distributed systems, query execution principles, and data storage formats tailored for analytical workloads.

Essential Skills

IcebergDorisTrinoSparkData EngineeringAnalytics EngineeringDistributed SystemsQuery PerformanceData ArchitectureData ModelingOLAPLakehouseSchema EvolutionData Lifecycle ManagementCompactionSnapshotsRetentionTrino Connector ConfigurationTrino Query TuningTrino Resource GovernanceTrino Workload ManagementDoris Data ModelingDoris Query Performance OptimizationBatch ProcessingStream ProcessingScalabilityCost EfficiencyPerformance BenchmarkingSLA MonitoringTroubleshootingData QualityData ReliabilityOperational ExcellenceAutomationDocumentationRunbooksBig Data AnalyticsAzure Data LakeJava

Good to Have

Apache Spark OptimizationShuffle TuningJoin StrategiesAdaptive ExecutionPipeline FrameworksMonitoringAlertingIncident ManagementCapacity PlanningCross-team Technical InitiativesPlatform AdoptionEnablement

Highlights

  • Actively hiring

More Details

RoleIceberg, Doris, Trino
DepartmentData & Analytics
Employment TypeFull Time, Work from office

About the Company

Infosys logo

Infosys

IT Consulting

Iceberg, Doris, Trino at Infosys | SkillMX | SkillMX