JPPF, java, parallel computing, distributed computing, grid computing, parallel, distributed, cluster, grid, cloud, open source, android, .net
JPPF

The open source
grid computing
solution

 Home   About   Features   Download   Documentation   On Github   Forums 
December 13, 2019, 03:30:47 AM *
Welcome,
Please login or register.

Login with username, password and session length
Advanced search  
News: New users, please read this message. Thank you!
Home Help Search Login Register
Pages: [1]   Go Down

Author Topic: [JPPF v3] Driver and Node configuration  (Read 2335 times)

Iced

  • JPPF Padawan
  • *
  • Posts: 8
[JPPF v3] Driver and Node configuration
« on: February 07, 2012, 01:13:41 AM »

Hello, I've been experimenting with JPPF 3.0 and have been trying to figure out the configuration to no avail.

Currently, in my test environment the driver and single node reside on the same server with 16 cores available

To test, the server is configured for
Code: [Select]
jppf.load.balancing.algorithm = autotuned
jppf.load.balancing.strategy = autotuned

# "autotuned" profile
strategy.autotuned.size = 1
strategy.autotuned.minSamplesToAnalyse = 100
strategy.autotuned.minSamplesToCheckConvergence = 50
strategy.autotuned.maxDeviation = 0.2
strategy.autotuned.maxGuessToStable = 50
strategy.autotuned.sizeRatioDeviation = 1.5
strategy.autotuned.decreaseRatio = 0.2

The node is configured with the following:
Code: [Select]
processing.threads = 15
task.bundle.strategy = autotuned
#jppf.load.balancing.strategy = autotuned
#jppf.load.balancing.size = 15

Based on what I could read of the documentation (and I realize some hasn't been updated yet), I would expect the above node configuration to override the server configuration.  I intend to install nodes on various workstations with processing.threads = 1 (to borrow a typically unused core) and servers with processing.threads = 3 that also have a typically unused core(s).  I want to explicitly configure the nodes to have control over what resources are used vs. letting the driver "autotune" it.  The driver is set to autotuned since the documentation states that it has to either be manual or autotuned for the node to be able to override the configuration.

I have a client sending a job with about 32 tasks to the driver for dispatching, however the driver continues to only send 1 task at a time to the node and I'm at a loss.  Any guidance would be very appreciated.
« Last Edit: February 16, 2012, 10:36:39 PM by Iced »
Logged

lolo

  • Administrator
  • JPPF Council Member
  • *****
  • Posts: 2258
    • JPPF Web site
Re: JPPF 3.0 Driver and Node configuration
« Reply #1 on: February 07, 2012, 07:37:31 AM »

Hello,

Currently, there is no way to have a node override the load balanncing parameters in the driver. I remember that we had this feature once, but it has been removed a long time ago (way before even v2.0).
Did you find a place in our v3.0 documentation that still mentions it? We would need to be fix the doc if that's the case.

If you need explicit control over load-balancing, then you should avoid the adaptive alogirthms: "autotuned", "proportional" and "rl".
If think that, given the type of configuration you will have for your nodes, you could use the "nodethreads" algorithm, which computes the number of tasks to send to each each node depending on the number of processing threads in that node, with a possibility to apply a multiplicator to that number of tasks.

So with this algorithm, the number of tasks sent to each node will be: multiplicator x node.processingThreads
Concretely, you could have the following driver config:
Code: [Select]
jppf.load.balancing.algorithm = autotuned
jppf.load.balancing.strategy = test
strategy.test.multiplicator = 2

Node1  woud have the following:
Code: [Select]
processing.threads = 1
Node2  woud have the following:
Code: [Select]
processing.threads = 3
In this configuration, the driver will always send at most 2 x 1 = 2 tasks to Node1, and 2 x 3 = 6 tasks to Node2.

I hope thisd helps.

Sincerely,
-Laurent
Logged

Iced

  • JPPF Padawan
  • *
  • Posts: 8
Re: JPPF 3.0 Driver and Node configuration
« Reply #2 on: February 07, 2012, 05:50:41 PM »

Hi Laurent,

Thanks for your help.  I found documentation (v3)  in the Node Configuration section of the JPPF Performance page

It seemed to suggest that I could override the server config:
Quote
So basically the types of overrides you can specify are as follows:
    override the server's "autouning" settings with a fixed bundle size for a specific node
    override the "autotuning" parameters with a set of node-specific parameters

The configuration you mentioned works  :)  I now have the following settings

server/driver:
Code: [Select]
jppf.load.balancing.algorithm = nodethreads
jppf.load.balancing.strategy = nodethreads
strategy.nodethreads.multiplicator = 1

node1:
Code: [Select]
processing.threads = 15
node2:
Code: [Select]
processing.threads = 3
node3:
Code: [Select]
processing.threads = 1
Logged

lolo

  • Administrator
  • JPPF Council Member
  • *****
  • Posts: 2258
    • JPPF Web site
Re: JPPF 3.0 Driver and Node configuration
« Reply #3 on: February 07, 2012, 06:31:39 PM »

Hello Iced,

Thanks for pointing to the documentation page where you saw this. This section of the documentation is definitely obsolete and we should have removed it.
Sorry for the inconvenience.

Sincerely,
-Laurent
Logged
Pages: [1]   Go Up
JPPF Powered by SMF 2.0 RC5 | SMF © 2006–2011, Simple Machines LLC Get JPPF at SourceForge.net. Fast, secure and Free Open Source software downloads