Print

[Najla]

Hi,

I'm a PhD student in CS and having a research on parallel programming frameworks.

I'm a new JPPF user, I have a question regarding to parallelism in JPPF:

JPPF is intended to parallelize computation intensive application; the JPPF client submits the application as set of tasks to the JPPF driver. My Question is on task decomposition, is client equipped by an algorithm that divides the application into parallel tasks; uniform and as small as possible.

There is no gain if the application decomposing has not been performed well

Do you know anything about this, is there any segmentation mechanism anywhere in JPPF? or tasks are just executed as they sumitted by client?

[Najla]

please reply 

[lolo]

Hello Najla,

Sorry for this late response.

The decomposition of the application into atomic tasks is the responsibility of the JPPF user. JPPF provides an API that attempts to make it as easy as possible to execute these tasks on a grid, however it does not provide that decomposition.

The part that is automated is the distribution of tasks among the nodes. It is performed by the JPPF driver (or server, which is an equivalent term), using a configurable load-balancing algorithm which determines how many tasks in a set of tasks will be sent to each node for execution.
The load-balancing algorithm is based on our observations that the overall performance of the execution is a function of the number of atomic tasks that are sent to each node at a time.
To give a very simple example: imagine we have decomposed our application into 10 atomic tasks. The JPPF client submits all these tasks in a single set. For clarity purposes, let's say we have a single node.
If the driver dispatches the tasks one at a time to the node, then the overall execution will take longer than if they were all dispatched at once to the node. How much longer depends on the computational weight of each task, the network context (e.g. latency and bandwidth), the hardware on which the node runs, and the number of threads a node can use.

We have 2 sorts of load-balancing algorithms:
- a "manual" one, where you configure a fixed value for the number of tasks dispatched to each node
- automatic adaptive load-balancing algorithms, where the number of tasks sent to each node is computed from the measured past performance of the node. These algorithms are auto-adaptive, meaning that they will update their parameters whenever the performance profile of the node changes. This could happen for instance if you submit a very different sort of tasks to the grid.

We are currently working on providing pluggability for these algorithms, so that JPPF users can write their own algorithm if they wish to, instead of using the ones we provide by default. This should be available in the next major version (v2.0).

I hope this answers your questions,
-Laurent

[Najla]

I have some inquires regarding to your answer:

1- Do you mean that application should be implemented and built by the user as a set of atomic JPPF tasks? I thought we can run any Java application on JPPF without the need to re-implement it as a set of tasks?

2- how to measure the computational weight of each task and where it is evaluated? at client or at driver?


3- I tried to run multiple servers on my laptops: first time I had got a complete initialization report including number of ports etc.
Where in the second driver I only receive the message "Build is successful". Why? is this because they initiated on the same machine?
Do you have any document/paper on multiple server grid system. I need such a document?


4- finally, regarding the selection of which server in grid system the nodes will be associated, how it is done randomly or first available port?

Thanks again for your reply,

All the best,
Najla

[Najla]

My email nalnabhan@gmail.com

[Najla]

Please reply

I need an answer before Tuseday

[lolo]

Hello Najla,

1- Do you mean that application should be implemented and built by the user as a set of atomic JPPF tasks? I thought we can run any Java application on JPPF without the need to re-implement it as a set of tasks?

There's 2 typical situations that we can consider:

• the application is already parallelized; in this case JPPF makes it easy to reuse the existing code and to automatically wrap your existing tasks into JPPF tasks for parallel execution
• the application was not written with parallellism in mind; here you will have to manually perform the decomposition of the application into atomic execution units (or tasks) that can be executed in parallel

2- how to measure the computational weight of each task and where it is evaluated? at client or at driver?

The computational weight of a task is basically the time it takes to execute it on a given machine. Hence, it would generally be evaluated on a node. Please note that JPPF does not explicitly measure this. What we measure is the round trip of a set of tasks from the server to a node and back. The performance feedback that the server gets is composed of the time it took for the round trip and the number of taks in the set of tasks. The time measurement implicitly includes the execution time, the network overhead, and so on, without providing any other information.

3- I tried to run multiple servers on my laptops: first time I had got a complete initialization report including number of ports etc.
Where in the second driver I only receive the message "Build is successful". Why? is this because they initiated on the same machine?
Do you have any document/paper on multiple server grid system. I need such a document?

If you want to run multiple servers on the same machine, you need to ensure that they use different ports. If not, the second driver you start will terminate and log a java.net.BindException in the server's log file.
We have documentation on how to configure the drivers: http://www.jppf.org/wiki/index.php?title=Configuring#Configuring_the_driver

4- finally, regarding the selection of which server in grid system the nodes will be associated, how it is done randomly or first available port?

There are 2 possibilities:

• if you are using automatic discovery (the default), each server will broadcast its TCP IP address and the ports it listens to. The node will then connect to the first server it finds
• if you are not using automatic discovery, the node will use the server host and ports specified in its configuration file. Please note that, in the case where automatic discovery is used, and when the node cannot find a server, it will automatically fall back to this mode.

I need an answer before Tuesday

I understand the urgency of your problem. However, please understand that this forum is animated by people who volunteer their time and, while we do our utmost to provide answers and assistance in a timely fashion, we cannot guarantee a response time. We can only do our best, and I apologize if this causes you any inconvenience.

Thank you very much for your time,
-Laurent

[Najla]

Thank you so much