Print

[deekay]

Hi,

I tried submitting the heavy jobs in the infinite loop. I have started 4 nodes and provided unique name to reach job. When i start the application JPPF submits the first job to only 1 node. and rest of the 3 nodes sits idle. Can you please suggest how can i submit the jobs to rest of the idle nodes.

Please find below code -

Code: [Select]

int i=0;
      while(true){
      // Create a job
      JPPFJob job = runner.createJob(args[0], i);

      // execute a blocking job
       runner.executeBlockingJob(job);
       i++;
      }
    }

Code: [Select]

public JPPFJob createJob(String args, int i) throws Exception
  {
    // create a JPPF job
    JPPFJob job = new JPPFJob();

    // give this job a readable unique id that we can use to monitor and manage it.
    job.setName("Template Job Id :"+i);

   // for(int i=0; i<10; i++){
    // add a task to the job.
    job.addTask(new JPPFTask(args));
   // }
    // add more tasks here ...

    // there is no guarantee on the order of execution of the tasks,
    // however the results are guaranteed to be returned in the same order as the tasks.
    return job;
  }


Code: [Select]

public void executeBlockingJob(final JPPFJob job) throws Exception
  {
    // set the job in blocking mode.
    job.setBlocking(true);

    // Submit the job and wait until the results are returned.
    // The results are returned as a list of JPPFTask instances,
    // in the same order as the one in which the tasks where initially added the job.
    List<JPPFTask> results = jppfClient.submit(job);

    // process the results
    for (JPPFTask task: results)
    {
      // if the task execution resulted in an exception
      if (task.getException() != null)
      {
      System.out.println(task.getException().getMessage());
      }
      else
      {
     
       System.out.println(task.getResult());
       
      }
    }
  }
Your help is highly appreciated 

Regards,
Deekay

[lolo]

Hello Deekay,

If you want to submit multiple jobs concurrently, you will need 2 things that I'm not seeing in your code:

1) Use non-blocking jobs. If you don't do that, then call to runner.executeBlockingJob() will be blocking, and your jobs will only be submitted one at a time.
So you may want to modify the code of your loop code as follows:

Code: [Select]

int i=0;
while(true) {
  // Create a job
  JPPFJob job = runner.createJob(args[0], i);
  // configure the job as non-blocking
  job.setBlocking(false);
  JPPFResultCollector collector = new JPPFResultCollector(job);
  job.setResultListener(collector);
  // submit the job. For a non-blocking job, this method returns null immediatly
  // the results can be obtained later using the JPPFResultCollector
  jppfClient.submit(job);
  i++;
}
Here there are 2 points to account for:
- this code does not process the results of the jobs execution
- you will probably end up with an OutOfMemoryError, since the infinite loop will keep submitting jobs faster than they are executed on the grid. Submitted jobs are added to an internal queue in the client.

2) In addition to asynchronous job submission, you will also need to setup a pool of connections to the driver, which is done in the client's configuration file. There are 2 ways to do this, depending on whether you use automatic server discovery or manual network configuration (properties "jppf.pool.size" and "driver_name.jppf.pool.size" respectively). This will ensure that multiple jobs can be sent to the driver concurrently, as each connection can only handle one job at a time.

I hope this helps.

Sincerely,
-Laurent

[deekay]

Hi Laurent,

Thanks for your reply. I tried your below suggestion to run the jobs in Asynchronous mode and set the pool size to 4. I started 4 different nodes and started the application which submits 100 jobs. These 100 jobs are submitted to only 1 node and other 3 nodes are sitting idle.




Regards,
Deekay

[lolo]

Hi Deekay,

Could you please post your client configuration and a sample task code that reproduces this behavior?

Thanks,
-Laurent

[deekay]

Please find attached Client configuration file and code file

Code: [Select]

public void run()
  {
    // write your task code here.
    System.out.println("Hello, this is the node executing a template JPPF task");
    try{
Thread.sleep(6000);
    }catch(Exception ex){
    }
    // ...

    // eventually set the execution results
    setResult("the execution was performed successfully");
  }

 main method -
 

Code: [Select]

public static void main(final String...args)
  {
    try
    {
      // create the JPPFClient. This constructor call causes JPPF to read the configuration file
      // and connect with one or multiple JPPF drivers.
      jppfClient = new JPPFClient();

      // create a runner instance.
      TemplateApplicationRunner runner = new TemplateApplicationRunner();
      long i =0;
      while(i<100){
      // Create a job
      JPPFJob job = runner.createJob(i);

      // execute a blocking job
     // runner.executeBlockingJob(job);
      // execute a non-blocking job
      runner.executeNonBlockingJob(job);
i++;
      }
    }
    catch(Exception e)
    {
      e.printStackTrace();
    }
    finally
    {
      if (jppfClient != null) jppfClient.close();
    }
  }


Regards,
Deekay

[lolo]

Hi Deekay,

The problem is that you're calling executeNonBlockingJob() for each. While in this method the call to jppfClient.submit(job) is non-blocking, the statement "List<JPPFTask> results = collector.waitForResults();" is actually a blocking one. So basically you're still in a scenario where jobs are executing on job at a time.
So, yes this method doesn't fully demonstrate asynchronous job submission, and I will change it for the next relelease.

I have modified the application runner code so that you have truly asynchronous job execution, and I'm attaching it to this post. You should see a very different behavior.

Sincerely,
-Laurent

[deekay]

Hi Laurent,

 The modified code which you provided is working as expected. Thanks for your help.


Regards
Deekay