JPPF Issue Tracker
Please log in to bookmark issues
CLOSED  Feature request JPPF-466  -  New Load-balancing algorithm: RL2
Posted Jul 25, 2016 - updated Jul 30, 2016
icon_info.png This issue has been closed with status "Closed" and resolution "RESOLVED".
Issue details
  • Type of issue
    Feature request
  • Status
  • Assigned to
  • Type of bug
    Not triaged
  • Likelihood
    Not triaged
  • Effect
    Not triaged
  • Posted by
  • Owned by
    Not owned by anyone
  • Category
  • Resolution
  • Priority
  • Targetted for
    icon_milestones.png JPPF 5.2
Issue description
Inspired by this forum discussion.

As pointed out, the existing "RL" load-balancing algorithm is not really using a reinforcement learning technique. It merely adjusts its actions/rewards based on what it remembers of the system's performance. In particular:
  • it does not modelize the possible states of the system
  • it does not modelize the probability of transitioning from one state to another when an action is applied
Furthermore, I have noticed, while testing the "rl" algorithm, that it has a tendency to converge towards a very low bundle size (generally just 1). Consequently, even though it produces a reasonably balanced workload, the resulting overall throughput is significantly below optimal.

I am proposing to deprecate "rl" and replace it with a new "rl2" algorithm, closer to the spirit of reinforcement learning, which:
  • builds a (partial) map of the states of the system by randomly walking the state space
  • choses actions with an increasing probability over time from the known states
  • resets itself and re-learns from scratch when significant disturbances are detected, for instance when a new job with a different performance profile is submitted to the grid

Comment posted by
Jul 30, 20:35
Implemented in trunk revision 4177