adequate
adequate
adequate
adequate
 

JPPF, java, parallel computing, distributed computing, grid computing, parallel, distributed, cluster, grid, cloud, open source, android, .net
JPPF

The open source
grid computing
solution

 Home   About   Features   Download   Documentation   Forums 
June 24, 2018, 11:59:51 PM *
Welcome,
Please login or register.

Login with username, password and session length
Advanced search  
News: New users, please read this message. Thank you!
  Home Help Search Login Register  
Pages: [1]   Go Down

Author Topic: (Solved) Getting return value of individual task in JPPF job  (Read 139 times)

aishwarya93

  • JPPF Padawan
  • *
  • Posts: 7
(Solved) Getting return value of individual task in JPPF job
« on: April 20, 2017, 03:59:05 PM »

Hi,

I have a single server single node JPPF cluster.

JPPF Version

Code: [Select]
2017-04-20 09:08:12,898 [INFO ][org.jppf.utils.FileUtils.initJPPFTempDir(547)]: JPPF temp folder C:\Users\AIVAIK~1\AppData\Local\Temp\.jppf
2017-04-20 09:08:13,008 [INFO ][org.jppf.utils.VersionUtils.logVersionInformation(80)]: --------------------------------------------------------------------------------
2017-04-20 09:08:13,008 [INFO ][org.jppf.utils.VersionUtils.logVersionInformation(81)]: JPPF Version: 5.2.6, Build number: 1901, Build date: 2017-04-02 09:50 CEST
2017-04-20 09:08:13,008 [INFO ][org.jppf.utils.VersionUtils.logVersionInformation(82)]: starting node with PID=4780, UUID=3371A890-5D67-9A84-E410-E248A7959978
2017-04-20 09:08:13,008 [INFO ][org.jppf.utils.VersionUtils.logVersionInformation(83)]: --------------------------------------------------------------------------------
2017-04-20 09:08:19,164 [INFO ][org.jppf.classloader.ClassLoaderRequestHandler.run(156)]: maxBatchSize = 1
2017-04-20 09:08:19,227 [INFO ][org.jppf.execute.AbstractExecutionManager.<init>(111)]: running 4 processing threads
2017-04-20 09:08:19,227 [INFO ][org.jppf.execute.AbstractExecutionManager.createThreadManager(137)]: Using default thread manager

Environment

Windows 10 Enterprise

Code: [Select]
java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)

Part of my TemplateRunner code looks like this -

Code: [Select]
  public void executeBlockingJob(final JPPFClient jppfClient) throws Exception {
   

//File IO - set filepath to appropriate claims file
        String filePath = "C:\\Users\\aivaikuntam\\Desktop\\10";

        // open read & write
        try (
                BufferedReader br = new BufferedReader(new FileReader(filePath));
                Writer writer = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(filePath + "_output")))) {

            String currentLine;

            // create new thread/task per claim
            while ((currentLine = br.readLine()) != null) try {
// Create a job
JPPFJob job = createJob("Claims Edit Loop");

GenericTask t = new GenericTask(currentLine);
job.add(t);

// set the job in blocking mode.
job.setBlocking(true);

// Submit the job and wait until the results are returned.
// The results are returned as a list of Task<?> instances,
// in the same order as the one in which the tasks where initially added to the job.
List<Task<?>> results = jppfClient.submitJob(job);

System.out.println("After " + t.getOutput());

            } catch (IOException e) {
                e.printStackTrace();
            }
        } catch (FileNotFoundException e) {
            e.printStackTrace();
        } catch (IOException e) {
            e.printStackTrace();
        }

   
 }

My code for GenericTask which is a callable looks like this -

Code: [Select]
public class GenericTask implements Callable<String>, Serializable {

    String currentLine;
    Claim claim;
String output = "Test" ;
StringBuilder sb;

    public GenericTask(String currentLine) {
        this.currentLine = currentLine;
        this.claim = new Claim();
    }

@Override
    public String call() {

                // Does a lot of processing to come up with the sb that I need
       
output = sb.toString();
System.out.println(output); //This works perfectly and has what I need
return output;
    }

public String getOutput() {
//System.out.println(" " + sb.toString()); // This throws a null pointer exception
System.out.println("Output is " + output); // This just says Output is test
        return output;
    }
}

My problem is that on the Node console, I can see the right value of the variable output but when I call t.getOutput() from the Runner, it returns the value "test" and not the actual value I want.

Does submitting the JPPF job for execution modify the value of output when the control comes back to the Runner? How do I get the value of output (which is sb in this case) back to my Runner so I can write it to a file ?

Thanks!
« Last Edit: May 09, 2017, 09:24:03 PM by aishwarya93 »
Logged

lolo

  • Administrator
  • JPPF Council Member
  • *****
  • Posts: 2223
    • JPPF Web site
Re: Getting return value of individual task in JPPF job
« Reply #1 on: April 20, 2017, 10:12:54 PM »

Hello,

When you add a Callable task to a job, it is in fact wrapped as an object whose class implements Task. You can check it like this:

Code: [Select]
JPPFJob job = new JPPFJob();
GenericTask t = new GenericTask(currentLine);
Task<?> wrapperTask = job.add(t);
GenericTask myTask = (GenericTask) wrapperTask.getTaskObject();
System.out.println("t == myTask : " + (t == myTask)); // check both are the same object

Your original task can always be obtained by calling getTaskObject() on the wrapping Task object. This is the same for the result tasks:

Code: [Select]
List<Task<?>> results = jppfClient.submitJob(job);
for (Task<?> jppfTask: results) {
  GenericTask myTask = (GenericTask) jppfTask.getTaskObject();
  // ... process the result ...
}

With regards to this, the following line in your code example, the following line will never work, because it displays the task that is submitted, not the one that is returned in the results
Code: [Select]
System.out.println("After " + t.getOutput());
Instead, you should write something like this:
Code: [Select]
GenericTask myResult =  (GenericTask)  results.get(0).getTaskObject);
System.out.println("After " + myResult.getOutput());
System.out.println("t == myResult : " + (t == myResult)); // should display false

I also noticed that you only have one task per node. While this will work, it may not be the most efficient way to distribute your tasks on the grid. I would recommend that you consider having multiple tasks per job, as JPPF is particularly efficient at distributing many small tasks.

I hope this clarifies,
-Laurent
Logged

aishwarya93

  • JPPF Padawan
  • *
  • Posts: 7
Re: Getting return value of individual task in JPPF job
« Reply #2 on: April 24, 2017, 05:49:25 PM »

Hi Laurent,

Thank you so much - that worked. I will definitely rethink about having a job per task.

Thanks,
Logged

aishwarya93

  • JPPF Padawan
  • *
  • Posts: 7
Re: (Reopened) Getting return value of individual task in JPPF job
« Reply #3 on: May 02, 2017, 12:07:01 AM »

Hi,

I have a modified problem of similar nature.

I have three global variables -

Code: [Select]
public List<Claim> output;
    public ManageCache cache;
    public List<Claim> input;



My TemplateRunner code has this -

Code: [Select]
// Create a job
            JPPFJob job = createJob("Claims Edit Loop");
           
            for (int i = 0; i < 10; i++) {
                GenericTask t = new GenericTask(input.get(i), output.get(i));
                 job.add(t);
            }

            List<Task<?>> results = jppfClient.submitJob(job);
           
        for (int i = 0; i <10; i++ ){
            System.out.println(output.get(i).get("FirstNameEdit"));
        }
         

My GenericTask has the following code -

Code: [Select]
public class GenericTask implements Callable<String>, Serializable {

    Claim input;
    Claim output;

    public GenericTask(Claim input, Claim output) {
        this.input = input;
        this.output = output;
    }

    @Override
    public String call() {
        for (Object key : input.keySet()) {
            // output booleans & error messages for this claim and its value
            // does a lot of processing to get the results which are as below

                    output.put(key.toString() + "Edit", ((Boolean) m1.invoke(edit, input) ? true : m2.invoke(edit)).toString());
                    System.out.println(output.get(key) + " " + output.get(key + "Edit"));
             }
        }
        return null;
    }
}


Problem - On the node terminal, the values of output.get(key) and output.get(key + "Edit") are what I want.
On the TemplateRunner side I'm getting null for output.get(key + "Edit"). I'm guessing this is a pass by reference vs pass by value problem but I'm not sure how to solve it.

Thanks,
Logged

lolo

  • Administrator
  • JPPF Council Member
  • *****
  • Posts: 2223
    • JPPF Web site
Re: (Reopened) Getting return value of individual task in JPPF job
« Reply #4 on: May 02, 2017, 08:28:05 AM »

Hello,

This behavior is normal and is due to the serialization and deserialization that occurs between the client application and the nodes. Keep in mind that the "output" that is processed in the node is a copy-by-serialization of the output created in the client. Similarly, the output that is returned as the result of jppfClient.submitJob(job) is a copy-by-serialization of the one processed by the node. This is important, because the state of these 3 instances are different. What you want to do, is to display the output that is returned, rather than the one initially created in the client as you are doing now.

Thus, I would suggest the following:
- add a getOutput() method to GenericTask, so the ouput with a processed state is accessible
- rewrite the loop that displays the output like this:
Code: [Select]
for (int i = 0; i <10; i++ ){
  GenericTask task = (GenericTask) results.get(i).getTaskObject();
  Claim processedOutput = task.getOutput();
  System.out.println(processedOutput.get("FirstNameEdit"));
}

Sincerely,
-Laurent
Logged
Pages: [1]   Go Up
 
JPPF Powered by SMF 2.0 RC5 | SMF © 2006–2011, Simple Machines LLC Get JPPF at SourceForge.net. Fast, secure and Free Open Source software downloads