JPPF, java, parallel computing, distributed computing, grid computing, parallel, distributed, cluster, grid, cloud, open source, android, .net
JPPF

The open source
grid computing
solution

 Home   About   Features   Download   Documentation   Forums 
October 18, 2019, 03:30:04 PM *
Welcome,
Please login or register.

Login with username, password and session length
Advanced search  
News: New users, please read this message. Thank you!
  Home Help Search Login Register  
Pages: [1]   Go Down

Author Topic: how to make sure jppf utilizes all available processors  (Read 948 times)

mmingfeilam

  • Guest
how to make sure jppf utilizes all available processors
« on: May 16, 2015, 01:27:42 AM »

i am running jppf on a 48 core machine, what is the best way to ensure that it's utilizing fully all the cores?  i also started multiple nodes by running the startNode.bat file multiple times, but i noticed that at most only 2 nodes are doing work.  thanks.
« Last Edit: May 17, 2015, 01:04:36 PM by mmingfeilam »
Logged

lolo

  • Administrator
  • JPPF Council Member
  • *****
  • Posts: 2256
    • JPPF Web site
Re: how to make sure jppf utilizes all available processors
« Reply #1 on: May 19, 2015, 09:05:07 AM »

Hello,

There isn't a single answer to this question, since it depends on many factors which I'll try to list here:
  • What's in the jobs/tasks
    For a single cpu core to be utilized fully, it has to be doing computations most of the time. This means that your tasks must be mostly CPU-bound rather than I/O-bound and they must last long enough, or be numerous enough, to have a significant impact of the CPU load. For instance, if your tasks are doing mostly I/O, then they will not max out the cpu. If a job only has a few short tasks, then the network round trip between node and server will outweigh the computational cost of the tasks. On the other hand, a single task that spawns its onw threads doing heavy computations could max the cpu out all by itself
  • Processing threads in each node
    the more processing threads, the more potential cpu utilization.
  • Number of nodes on each machine
    As you already know, you can have any number of JPPF nodes running on a single machine. You can even do it remotely and dynamically using the node provisioning facility. You can also make a node start additional slave nodes at startup, using the currently undocumented configuration property "jppf.node.provisioning.startup.slaves = number_of_slave_nodes", so you won't have to run startNode.bat multiple times.
  • Load-balancing
    The load-balancing in the server determines how many tasks will be sent to a node at a given time. The more tasks you send to a node, the more chances it has to max the cpu out. If you send very few tasks to each node, then you have the network round-trip overhead versus parallel network I/O trade-off. To find an (close to) optimal load-balancing configuration, you will probably have to try different configurations. I'd suggest to use the administration console for this, as it allows you to change the load-balancing configuration easily and dynamically (you can also do it programmatically)
    Some of the built-in load-balancing algorithms ("proportional", "autotuned" and "rl") are adaptive, and will automatically adjust the number of tasks they send to each node, using an objective function that attempts to minimize the mean round-trip time. The other algorithms ("manual" and "nodethreads") have fixed settings and would be more adapted in scenarios where the workload is well-defined and doesn't vary much
    Finally, if the built-in algorithms do no satisfy your needs, you can also implement your own, as in this demo
I hope this helps,
-Laurent
Logged
Pages: [1]   Go Up
 
JPPF Powered by SMF 2.0 RC5 | SMF © 2006–2011, Simple Machines LLC Get JPPF at SourceForge.net. Fast, secure and Free Open Source software downloads