JPPF, java, parallel computing, distributed computing, grid computing, parallel, distributed, cluster, grid, cloud, open source, android, .net
JPPF

The open source
grid computing
solution

 Home   About   Features   Download   Documentation   On Github   Forums 
February 26, 2021, 04:24:55 PM *
Welcome,
Please login or register.

Login with username, password and session length
Advanced search  
News: New users, please read this message. Thank you!
  Home Help Search Login Register  
Pages: [1]   Go Down

Author Topic: Forcing max task batch size  (Read 2352 times)

row_perfect

  • Guest
Forcing max task batch size
« on: November 27, 2012, 11:29:43 PM »

Hi Guys,

For most of my jobs, the proportional strategy is fine, however, for a few specific jobs, I would like to force the maximum task batch to be 1. Is there a way to do it without resorting to single-task separate jobs?

Thanks, Paul
Logged

lolo

  • Administrator
  • JPPF Council Member
  • *****
  • Posts: 2265
    • JPPF Web site
Re: Forcing max task batch size
« Reply #1 on: November 28, 2012, 05:41:46 AM »

Hi Paul,

You can achieve this with a custom load-balancer which extends the proportional algorithm implementation and which is job-aware.

This would give a load-balancer implementation like this:
Code: [Select]
public class CustomProportionalBundler extends ProportionalBundler implements JobAwareness {
  private JobMetadata metadata = null;

  public CustomProportionalBundler(final LoadBalancingProfile profile) {
    super(profile);
  }

  @Override
  public int getBundleSize() {
    if (metadata != null) {
      Integer n = (Integer) metadata.getParameter("force.bundle.size");
      if ((n != null) && (n > 0)) return n;
    }
    return super.getBundleSize();
  }

  @Override
  public JobMetadata getJobMetadata() {
    return metadata;
  }

  @Override
  public void setJobMetadata(final JobMetadata metadata) {
    this.metadata = metadata;
  }
}

Then, to plug it into the driver, implement the service provider:
Code: [Select]
public class CustomProportionalBundlerProvider implements JPPFBundlerProvider {
  @Override
  public Bundler createBundler(final LoadBalancingProfile profile) {
    return new CustomProportionalBundler(profile);
  }

  @Override
  public LoadBalancingProfile createProfile(final TypedProperties configuration) {
    return new ProportionalTuneProfile(configuration);
  }

  @Override
  public String getAlgorithmName() {
    return "custom_proportional";
  }
}

and add a file "META-INF/services/org.jppf.server.scheduler.bundle.spi.JPPFBundlerProvider" which contains "my.package_name.CustomProportionalBundlerProvider".

Once you have added these files to the driver's classpath, you can configure your new load-balancer in the config file:
Code: [Select]
jppf.load.balancing.algorithm = custom_proportional
jppf.load.balancing.strategy = my_profile

# "my_profile" profile: we can use the same parameters as for "proportional" algorithm
strategy.my_profile.performanceCacheSize = 3000
strategy.my_profile.proportionalityFactor = 1
strategy.my_profile.initialSize = 10
strategy.my_profile.initialMeanTime = 1e9

Then you're ready to use the new algorithm for your jobs:
Code: [Select]
JPPFJob job = ...;
job.getMetadata().setParameter("force.bundle.size", 1);
jppfCLient.submit(job);
...

I hope this helps,
-Laurent
Logged

row_perfect

  • Guest
Re: Forcing max task batch size
« Reply #2 on: November 28, 2012, 09:44:50 AM »

Wow, that's extremely helpful, thanks!
Logged

row_perfect

  • Guest
Re: Forcing max task batch size
« Reply #3 on: November 29, 2012, 08:32:39 AM »

I'm very happy with this, thanks.
Logged
Pages: [1]   Go Up
 
JPPF Powered by SMF 2.0 RC5 | SMF © 2006–2011, Simple Machines LLC Get JPPF at SourceForge.net. Fast, secure and Free Open Source software downloads