JPPF, java, parallel computing, distributed computing, grid computing, parallel, distributed, cluster, grid, cloud, open source, android, .net
JPPF

The open source
grid computing
solution

 Home   About   Features   Download   Documentation   Forums 
October 22, 2019, 07:51:57 AM *
Welcome,
Please login or register.

Login with username, password and session length
Advanced search  
News: New users, please read this message. Thank you!
  Home Help Search Login Register  
Pages: [1]   Go Down

Author Topic: JPPF Pool Size  (Read 924 times)

mmingfeilam

  • Guest
JPPF Pool Size
« on: May 30, 2015, 12:28:10 AM »

i know that you can set pool size both in the properties file or programmatically, what i want to know is what is the best way to determine the size.  if i want to run 200 tasks, do i need pool size to be 200, or would a smaller number suffice?  thanks.
Logged

lolo

  • Administrator
  • JPPF Council Member
  • *****
  • Posts: 2256
    • JPPF Web site
Re: JPPF Pool Size
« Reply #1 on: May 30, 2015, 07:57:16 AM »

Hello,

Having multiple connections in a pool serves two basic purposes:

1) it provides the ability to submit multiple JPPF jobs concurrently from the same JPPFClient. In this scenario, the number of connections represents the maximum number of jobs that can be sent concurrently to the server. See demos here and here.

2) it also enables sending a single job over multiple connections in parallel, the goal being to provide parallel I/O speedup for large jobs. To accomplish this, you need to set up proper load-balancing parameters in the client configuration (different from the default which are desgined to send all tasks of a job through the same connection) and specify the number of connections in the job SLA.
Example load-balancing setup:
Code: [Select]
# default is "manual" with size = 1,000,000
jppf.load.balancing.algorithm = proportional
jppf.load.balancing.profile = proportional
# "proportional" profile
jppf.load.balancing.profile.proportional.performanceCacheSize = 1000
jppf.load.balancing.profile.proportional.proportionalityFactor = 1
jppf.load.balancing.profile.proportional.initialSize = 10
jppf.load.balancing.profile.proportional.initialMeanTime = 3e10
The code to submit a job over 2 connections in parallel would be like this:
Code: [Select]
JPPFClient client = ...;
JPPFJob job = ...;
// use up to 2 connections to submit the job
job.getClientSLA().setMaxChannels(2);
client.submitJob(job);

Note that 1) and 2) can be used together in any combination you choose.

Knowing this, I would say that most of the time a single connection will be enough. Using multiple connections will be useful in scenarios where you have multiple jobs to execute concurrently from the same client, or if you have jobs with very large memory footprints, which would benefit from parallel I/O.
Logged

mmingfeilam

  • Guest
Re: JPPF Pool Size
« Reply #2 on: June 01, 2015, 08:11:29 PM »

which property sets the connection size?  jppf.load.balancing.profile.proportional.initialSize = 10?  i thought that's the number of bundles each task can handle.  i am using one client to run 1 job, but the job has multiple tasks.  so really in essence i am running multiple "jobs".  for now i will set pool.setMaxSize(20).  thanks. 
Logged

lolo

  • Administrator
  • JPPF Council Member
  • *****
  • Posts: 2256
    • JPPF Web site
Re: JPPF Pool Size
« Reply #3 on: June 02, 2015, 06:06:28 AM »

There are two properties you can use to set the size of a connection pool:
- when using auto-dicovery, then the pool size of all discovered drivers is set with "jppf.pool.size = n". This is documented here
- when using manual connection parameters, the pool size is configured with "driver_name.jppf.pool.size = n", where driver_name is the name of one of the drivers defined in "jppf.drivers = driver1 ... driverN". This is documented here.

-Laurent
Logged
Pages: [1]   Go Up
 
JPPF Powered by SMF 2.0 RC5 | SMF © 2006–2011, Simple Machines LLC Get JPPF at SourceForge.net. Fast, secure and Free Open Source software downloads