JPPF, java, parallel computing, distributed computing, grid computing, parallel, distributed, cluster, grid, cloud, open source, android, .net
JPPF

The open source
grid computing
solution

 Home   About   Features   Download   Documentation   On Github   Forums 
March 28, 2023, 10:33:17 PM *
Welcome,
Please login or register.

Login with username, password and session length
Advanced search  
News: New users, please read this message. Thank you!
  Home Help Search Login Register  
Pages: [1]   Go Down

Author Topic: Selecting prefered node - LoadBalancer/Execution policy  (Read 1800 times)

flo

  • Guest
Selecting prefered node - LoadBalancer/Execution policy
« on: August 11, 2016, 08:28:38 AM »

Hello to everyone, hello Laurant,

I thought about asking you for this issue before starting to edit the code of JPPF.
My idea is to have a network with maybe 6 nodes and one driver. Three of the nodes are running with hardware configuration A and the other one with B.
The Jobs can be splittet in two types with different requirements r1 and r2. Jobtype r1 can only be performed on node A and r2 can run with both hardware configuration.
To gain the efficience of the grid I want that the first Element in the Queue, if it is a r2 job, check first for idle B-nodes and prefere that one. In case that all of them are running, the job should be dispatched to a idle A-Node.

So my questions:
- Is there a conveniance way to setup a configuration like that or is it necessary to work with the source code?
- Would you realize that as a combination of load balancing profile and execution policy or prefere one of them?
Which one would you choose in the second case?

I hope to get some of your ideas. Thank you very much in advance.

Kind regards,
Flo
Logged

lolo

  • Administrator
  • JPPF Council Member
  • *****
  • Posts: 2272
    • JPPF Web site
Re: Selecting prefered node - LoadBalancer/Execution policy
« Reply #1 on: August 12, 2016, 06:20:49 AM »

Hello Flo,

I had implemented a "Preference" execution policy that was supposed to achieve this, but I did it so naĆÆvely that it turned out to be equivalent to an "Or" policy.
I don't think using a custom load-balancer will help either, because it won't prevent a job of type r1 from being sent to node B: when the load-balancer is invoked, the node has already been selected for the job. The load-balancer would help specifying that more tasks of the job r1 should be sent to node A, and less tasks to node B, but that's all.

The closest you can get to your requirements would be by using a combination of an execution policy and job priority: if you give job r1 a higher prority than job r2, along with an execution policy that only selects nodes of type A, then you will ensure the following:
- job r1 will only execute on node type A
- if jobs of type r1 and type r2 are competing for nodes, then jobs of type r1 will always get nodes of type A first
- nodes of type B will only be used by job type r2
- job type r2 may execute on node A only if no job r1 is competing for it

Would this be sufficient for your requirements?

Thanks,
-Laurent
« Last Edit: August 12, 2016, 07:16:47 AM by lolo »
Logged

flo

  • Guest
Re: Selecting prefered node - LoadBalancer/Execution policy
« Reply #2 on: August 17, 2016, 09:44:14 AM »

Hello Laurant,

sorry for the delay of my reply.
So far I have already tried out a bit with the execution policy and thought that it is not necessary to edit the load-balancer. Thank you that you have confirmed that!
Regarding to you answer I can tell you, that I am pleased not to edit the job priority. That is the reason why I had to think about an other solution.
I found a way how it worked and the requierements are satisfied completely:

(1) First you have to add a attribute to the node configurations which contains information about the type of hardware (A or B).
To achieve this, I have added this to the jppf-node.properties.

- (2) You need a list or a file to know which job is a job from type r1 or r2 - this is realized in the jppf-driver.properties.
Definitely it is just possible to do it in this way if you know all your jobs. An alternative to that could be a new attribute in the job information, but I have not tried that so far.

- The execution policy is running through some comparing operations to check if the node is accepted or not.
First of all you have to check if the job is a type r1-job and the node is a A-node. If both are true the statement true is returned from the ExecutionPolicy.accepts to the function findIdleChannelIndex in server.nio.nodeserver.TaskQueueChecker.java.

If the job is a r2-job and the node is a B-node you can also return the true-statement immediatelly.
In case you have a r2-job and not a B-node you have to check other connected nodes to the driver for the systemattributes.
stefaneidelloth posted a way I used to get the attributes of each node:
http://www.jppf.org/forums/index.php/topic,7659.msg11360.html#msg11360

So you loop through the systemInformation of each node and check if it is a A- or a B-node. If you find a idle B-node you can return false.

I hope you understand what I tried to explain.


Kind regards
Flo

Logged
Pages: [1]   Go Up
 
JPPF Powered by SMF 2.0 RC5 | SMF © 2006–2011, Simple Machines LLC Get JPPF at SourceForge.net. Fast, secure and Free Open Source software downloads