Evaluating Three Predefined Scheduling and Batching Configurations in Recommendation Systems: A Simulation-Based Study

Abstract

This study evaluates the impact of three predefined scheduling and batching configurations on recommendation-system serving performance using a custom Python-based simulation framework. The framework compares FIFO Scheduling with Fixed Batching, Priority Scheduling with Fixed Batching, and Priority Scheduling with Elastic Batching. The evaluation is limited to these three configurations rather than a complete two-by-two factorial design. Experiments use request features derived from the MovieLens-1M dataset and synthetic Poisson arrivals under low-, medium-, and high-load conditions. Performance is evaluated using throughput, P95 latency, P99 latency, latency-distribution plots, timeout rate, and Welch’s t-tests on request-level latency observations. The results show that the evaluated strategies behave similarly under low load. Under medium load, Priority Scheduling with Fixed Batching remains close to the FIFO baseline, while Priority Scheduling with Elastic Batching produces substantially higher P95 and P99 latency. Under high load, Priority Scheduling with Elastic Batching achieves the highest throughput but still has much worse P95 and P99 latency than the FIFO baseline. The findings show mixed latency outcomes for elastic batching. The simulation results indicate a trade-off in which throughput-oriented batching can increase latency variability under heavier workloads. The study does not include deployed-system testing, hardware-utilization measurement, or recommendation-accuracy evaluation.