Print

[ales]

We are currently working on options to configure a JPPF grid to connect multiple computers that are in three different subnets.
There are 4 classrooms, each with 20-30 computers. All computers in classroms are either in subnet A or B. The third subnet is for staff only. All subnets are /24.

The main idea is to deploy tasks on the grid from the staff subnet (and probably also add a few computers to the grid).

I have tried with pure a P2P grid (drivers with local node). But as expected, only the drivers within the same subnet find each other. To solve this, I manually specified 2 drivers from the staff network, so all the drivers are connected to them (and also to every other driver in the same subnet). So far so good. But when I start the admin GUI, it connects to the 2 drivers on the staff network and lists all the computers in classrooms that are connected to them. But the problem is that the status and statistics are visible only for the two drivers in the staff subnet that the admin GUI is immediately connected to. If I run a job with a lot of tasks, it DOES run on ALL available drivers, but you cannot see that from the admin GUI, therefore you cannot be sure if it really works everytime.

I also considered using a driver->n*nodes architecture, but the main problem is to specify multiple drivers that a node can connect to (I am aware that a node can only connect to a single driver).
Server discovery doesn't work across different subnets and manually hardcoding a single driver isn't feasible as there would be multiple drivers on the staff network and there will be no guarantee that all of them (if any) will be online.

Is there an option to configure a node to try to connect to the first available driver on a list?
The other option is to deploy drivers with local nodes on the classroom computers, disable discovery an manually specify a list of drivers on the staff subnet. But the problem described in the 3rd paragraph will still persist - how will we be able to know if all the drives are crunching jobs if we cannot see the statistics on the drivers in classroom computers.

[lolo]

Hello,

The problem with the administration and monitoring console, and the underlying JMX-based APIs, is a long-standing limitation in JPPF.
This will be addressed in the next version of JPPF, via the implementation of 2 feature requests in the roadmap for v3.3:

• JPPF-26 Enable node management via the driver connection
• JPPF-112 Refactor the admin console for management and monitoring via the driver

Unfortunately, there is currently no other practical alternative for this.

Regarding your question:

Is there an option to configure a node to try to connect to the first available driver on a list?

Yes this is possible, we actually have a sample which illustrates how this can be done: Initialization Hook sample.
You might also want to combine this with an alternate configuration source for your nodes, so they can read their configuration from a common location: database, shared file system, remote server (e.g. http, ftp), cloud storage, etc. This will greatly facilitate maintenance tasks, should the list of available drivers change over time.

Sincerely,
-Laurent

[ales]

Thanks for the quick reply! The "alternate configuration source" can be used for drivers and nodes, or just for nodes?

Which of the two option is better (less overhead, no duplicate execution, evenly distributed tasks etc.)?


Option A (left): Drivers in classrooms, that connect to all the drivers in the staff network (but not with each other)
Option B (right): Nodes in classrooms, that connect to one of the available drivers (doesn't mather which one, equal distribution is not necessary)

And may I also ask when will the v3.3 be released?

[lolo]

Hello,

The "alternate configuration source" can be used for drivers and nodes, or just for nodes?

It can be used for any JPPF component: drivers, nodes and clients.

Which of the two option is better

Here I automatically apply the rule "simpler is better", thus I would prefer the solution with as few drivers as possible. Option B provides simplicity, and also a failover mechanism: should one of the drivers fail, all nodes will connect with the other driver (using a mechanism such as shown in the previously mentioned sample), and the client will automatically adapt as well.

Regarding JPPF 3.3 we are planning on releasing it by March 31st as you can see in the roadmap.
However, we will publish several "partial" releases in the meantime, as unstable releases. What I can say is that I'm currently working on the two features I mentioned earlier, and they will be part of the next partial release. We do our best to update the status of new features according to the work done, so you might want to check in the issue tracker as well. For now, I am in testing phase for JPPF-26, then I'll need to update the documentation accordingly, so it shouldn't be too long.

I hope this helps.

-Laurent

[lolo]

Hello,

This is to let you know that the 2 features mentioned previously have been implemented, and we delivered a v3.3_alpha build, which you can get get here: http://sourceforge.net/projects/jppf-project/files/jppf-project/latest-unstable

This is really cool stuff, to finally have the ability to manage nodes that are in a different network
Please feel free to give it a try, we will be grateful for any feedback you have on this.

Sincerely,
-Laurent