JPPF Forums

JPPF Help => Developers help => Topic started by: dowobeha on December 06, 2012, 05:06:18 PM

Title: Patch to allow dynamic port allocation (to enable OGS/SGE integration)
Post by: dowobeha on December 06, 2012, 05:06:18 PM
Hi,

I'm new here, so if I'm posting in the wrong place, or not following protocol, please feel free to point me in the right direction. :)

I have been working on a tight integration of JPPF as a parallel environment within the Open Grid Scheduler (formerly SGE, Sun Grid Engine). The idea is that there are users (like me) who have a computing environment that is set up under OGS or SGE who would like to be able to launch JPPF jobs that are scheduled and managed by OGS / SGE. The idea is that when a user launches a new job that needs JPPF, OGS will allocate a number of compute nodes for the job. OGS then starts up a JPPF driver and JPPF nodes on these particular compute nodes, and then runs the job itself. When the job is complete, OGS shuts down the JPPF driver and nodes.

I have recently succeeded in getting this set up.

One requirement that surfaced in performing this integration is the need for dynamic port allocation within JPPF.

In an OGS environment, a single multi-core computer may be configured to present itself as multiple nodes to the OGS scheduler. Consequently, it is entirely possible that when multiple JPPF jobs are launched via this OGS integration, the result could be that a single computer may end up running more than one JPPF driver. In this scenario, it is necessary to configure the JPPF driver such that it dynamically sets the value of jppf.server.port to a vacant port number. I have made changes to org.jppf.server.JPPFDriver and org.jppf.server.nio.NioServer to enable this behavior in JPPF.

The attached patches are based off of v_3_1_4. After applying the attached patches, if the user sets jppf.server.port=0, the JPPF driver will dynamically allocate a free port number and print out the newly bound port. This port will be broadcast to available clients.

At your convenience, it would be great if you could review the attached patches. I hope you will be able to include them in the repository. The changes are about 21 lines of code, and about a third of that is comments.

Cheers,
Lane Schwartz
Title: Re: Patch to allow dynamic port allocation (to enable OGS/SGE integration)
Post by: lolo on December 06, 2012, 05:14:47 PM
Hi Lane,

Thank you very much for this!
This is definitely a useful feature and I'll be happy to include it.
Shall we add a Javadoc @author tag with your name as a measure of recognition for your contribution?

Sincerely,
-Laurent
Title: Re: Patch to allow dynamic port allocation (to enable OGS/SGE integration)
Post by: dowobeha on December 06, 2012, 05:27:17 PM
Hi Lane,

Thank you very much for this!
This is definitely a useful feature and I'll be happy to include it.
Shall we add a Javadoc @author tag with your name as a measure of recognition for your contribution?

Sincerely,
-Laurent

Laurent,

Sure. It's a minor patch, but any attribution would be welcome. :)

Cheers,
Lane
Title: Re: Patch to allow dynamic port allocation (to enable OGS/SGE integration)
Post by: dowobeha on December 06, 2012, 05:31:26 PM
Laurent,

If you wouldn't mind, please let me know whenever the first version comes out that includes this new feature. There's no rush, but I plan to push my OGS-JPPF integration feature upstream to the OGS folks for inclusion in the Open Grid Scheduler. I will wait to do so until the dynamic port allocation is distributed with JPPF.

Cheers,
Lane
Title: Re: Patch to allow dynamic port allocation (to enable OGS/SGE integration)
Post by: lolo on December 07, 2012, 07:07:00 AM
Hi Lane,

No problem, I will update you in this thread when it is done.
I have scheduled this feature for the upcoming release v3.2, and registered a feature request for it: JPPF-101 Ability to provide dynamic binding for the driver (http://www.jppf.org/tracker/tbg/jppf/issues/JPPF-101)

-Laurent
Title: Re: Patch to allow dynamic port allocation (to enable OGS/SGE integration)
Post by: lolo on December 18, 2012, 01:03:09 PM
Hi Lane,

This is to let you know that dynamic port allocation is now available as part of the JPPF 3.2 release.

Sincerely,
-Laurent