JPPF, java, parallel computing, distributed computing, grid computing, parallel, distributed, cluster, grid, cloud, open source, android, .net
JPPF

The open source
grid computing
solution

 Home   About   Features   Download   Documentation   Forums 

Job Dependencies demo

What does the sample do?

This demo illustrates a solution to submit jobs that have a dependency on one or more other jobs. It supports any non-cyclic dependency graph and will work even when the jobs in the dependency graph are submitted from multiple JPPF clients running on multiple machines.

How do I run it?

1. Grid setup

  • you will first need to build the server extension which manages the job dependencies: start a command prompt in JPPF-x.y.z-samples-pack/JobDependencies then type "ant". This will create a file named JobDependencies.jar. Copy this file to your server's /lib folder
  • then start a JPPF server and at least one node. For information on how to set up a node and server, please refer to the JPPF documentation.
  • during the server startup, you should see the following two lines in the console output:
    [2016-05-09 21:36:13.585] processor: In DependencyProcessor()
    [2016-05-09 21:36:13.585] processor: Initializing DependencyProcessor

2. Running the demo

  • open a command prompt in the JPPF-x.y.z-samples-pack/JobDependencies folder
  • start the demo by entering "run.bat" on Windows, "./run.sh" on Linux/Unix or by launching the Ant script "ant run"
  • During the demo, the console output will show the jobs that complete, along with their execution results

3. Defining your own dependency graph

  • The jobs graph is defined in the file JobDependencies/dependency_graph.txt
  • for example, the dependency graph defined in the vanilla version of this sample is defined as:

    Configuration

    Corresponding graph


    Job F ==> Job A, Job B, Job E | remove Job E ==> Job C, Job D Job D ==> Job B, Job C Job C ==> Job A Job B ==> Job A Job A ==>

4. Detection of cycles in the dependency graph

When a cycle is detected, the server will print a message to the console, similar to this:

[2016-05-09 21:48:00.621] cycle detected while adding dependency 'Job C' to 'Job A' :
Job A ==> Job B ==> Job C ==> Job A
Furthermore, no dependency will be assigned to the job, so that it can be executed and avoid being stuck in the server queue.

5. Distributed scenario: submitting jobs in the dependency graph from separate clients

As mentioned above, this demo will work even when the jobs are submitted from multiple client applications. To illustrate this, we will proceed as follows:

  • Make a copy of the JPPF-x.y.z-samples-pack/JobDependencies folder, let's call it JobDependencies2 for this exercise
  • in JobDependencies/dependency_graph.txt, remove or comment out the last 3 declared dependencies, so the dependencies should look like this:
    Job F ==> Job A, Job B, Job E | remove
    Job E ==> Job C, Job D
    Job D ==> Job B, Job C
  • in JobDependencies2/dependency_graph.txt, remove or comment out the first 3 declared dependencies, the dependencies should look like this:
    Job C ==> Job A
    Job B ==> Job A
    Job A ==>
  • if not already done, start a server and at least one node
  • start the demo in JobDependencies with "run.bat" or "./run.sh". You will observe that it prints the following message:
    [2016-07-22 08:51:33.667] runner: ***** awaiting results for 'Job F' *****
  • start the demo in JobDependencies2 with "run.bat" or "./run.sh". It will print completion messages for the jobs A, B and C
  • now if you get back to the console for JobDependencies, you will see that jobs D, E, F were completed as well, following the completion of their dependencies submitted by the other instance of the demo in JobDependencies2

Source files

package org.jppf.example.job.dependencies:

package org.jppf.example.job.dependencies.management:

I have additional questions and comments, where can I go?

If you need more insight into the code of this demo, you can consult the Java source files located in the CustomLoadBalancer/src folder.

In addition, There are 2 privileged places you can go to:


JPPF Copyright © 2005-2017 JPPF.org Get JPPF at SourceForge.net. Fast, secure and Free Open Source software downloads