Print

[steve4j]

Hallo Laurent,

thanks for your help in the issue tracker so far http://www.jppf.org/tracker/tbg/jppf/issues/JPPF-500.
Now since i got the DBCP & Hibernate workin my project "gridcomputing for interfaces" i came along this Thread http://www.jppf.org/forums/index.php/topic,1961.0.html
Where you said (4y ago) that one node can only handle one job at a time.

Since my project starts like min. 15 to max. 80 interfaces per min. i have to ask if i realy need like at least 100 nodes with each a single JVM.
This would produce so much overhead from each jvm that i could not use JPPF as ligthweight parallel processing framework.

Is there a way to start lets say like 20 nodes with 5 open "Jobpool" each so i need only 20 JVM not 100?
The nodes are needed for fail-security, but i cannot start that much jvms.

Can there be a feature request for a new Release or something that could help to fix this?

Thank you for your efforts

[lolo]

Hello,

You do not have to have as many nodes as there are jobs at any given time. When all nodes are busy, the server's queuing mechanism ensures that unexecuted jobs are preserved until a node becomes available. Similar queuing occurs in the client as well, since the maximum number of jobs that can be submitted to a server is determined by the number of connections in a connection pool.

The one-job-per-node model (or one job per connection in the client) is a design choice which significantly simplifies the communication model, makes the grid more robust and less prone to crashes, allows better control of the resources usage , enables load-balancing which adapts to the workload, etc. The benefits are too important to abandon them now.

I'll be happy to provide suggestions on how the performance or topology can be optimized, however I'd need little more information. In particular, it is not clear to me what you mean by "interface". What does it represent in terms of JPPF jobs and tasks? How many tasks do your jobs have? Remember that the granularity of jobs and tasks is a critical factor for performance, and it should be given some consideration at design time when building a distributed application.

Thanks for your time,
-Laurent

[steve4j]

Hallo Laurent,

thanks for your super-fast response.
Interfaces in my project are "connectors" between two systems.
Lets say for a example there is a system wich provides businessdata in CSV formated file on a ftp server. This file needs to be picked up every 15min  and be iterated, mapped and written to a database.
So my project has 75 different interfaces with a total of 150 versions.
My historical data tells me that 15-80 interfaces are started at the same time due the day.
A interface version can only run at the same time.

The current model of this project is a stand alone rich client (swing).
But since we dont want to be bound to hardware / OS / other problems we want to commit the jobs into a cluster and use a scheduler www.quartz-scheduler.org.

Thanks for your help,
Steve