Inspired by this forum discussion.

As pointed out, the existing "RL" load-balancing algorithm is not really using a reinforcement learning technique. It merely adjusts its actions/rewards based on what it remembers of the system's performance. In particular:

• it does not modelize the possible states of the system
• it does not modelize the probability of transitioning from one state to another when an action is applied

Furthermore, I have noticed, while testing the "rl" algorithm, that it has a tendency to converge towards a very low bundle size (generally just 1). Consequently, even though it produces a reasonably balanced workload, the resulting overall throughput is significantly below optimal.

I am proposing to deprecate "rl" and replace it with a new "rl2" algorithm, closer to the spirit of reinforcement learning, which:

• builds a (partial) map of the states of the system by randomly walking the state space
• choses actions with an increasing probability over time from the known states
• resets itself and re-learns from scratch when significant disturbances are detected, for instance when a new job with a different performance profile is submitted to the grid