JPPF, java, parallel computing, distributed computing, grid computing, parallel, distributed, cluster, grid, cloud, open source, android, .net
JPPF

The open source
grid computing
solution

 Home   About   Features   Download   Documentation   Forums 
July 20, 2019, 07:52:25 PM *
Welcome,
Please login or register.

Login with username, password and session length
Advanced search  
News: New users, please read this message. Thank you!
  Home Help Search Login Register  

Recent Posts

Pages: [1] 2 3 ... 10
1
Forums / Hello!
« Last post by cynthia51 on June 12, 2019, 08:35:12 AM »
Hi everyone,

Finally! I am in this forum, glad to meet and mingle with each and everyone of you. See you around guys!
2
Forums / Re: Please post a message when you register
« Last post by cynthia51 on June 12, 2019, 08:33:41 AM »
Hi everyone,

Finally! I am in this forum, glad to meet and mingle with each and everyone of you. See you around guys!
3
Installation and Configuration / Re: Running JPPF in Kubernetes
« Last post by boris.klug on May 21, 2019, 08:48:35 AM »
Thats great news!

If there is a any (e.g. alpha) version to test, I will try it out!
4
Installation and Configuration / Re: Running JPPF in Kubernetes
« Last post by lolo on May 20, 2019, 09:22:56 AM »
Hello,

We are currently working on providing Docker images for JPPF components, along with tools and configuration files to run JPPF grids on Docker swarm and Kubernetes infrastructures. Please see JPPF-589 Docker images for JPPF components which we intend to deliver in the first allpha release of the upcoming JPPF 6.2.

We are definitely interested by any input or requirements that you think should be part of it.

Sincerely,
-Laurent
5
Installation and Configuration / Running JPPF in Kubernetes
« Last post by boris.klug on May 20, 2019, 08:03:41 AM »
Hi!

I am thinking about running JPPF in a Kubernetes cluster. I know, Kubernetes is more for running small, lightweight apps/containers. As JPPF (or the jobs/tasks that run in JPPF) are more CPU heavy load apps, you have to force Kubernetes to run one JPPF node on one Kubernetes node and not to change this.

The advantage would be that you can get up a Kubernetes cluster running easily at a local installation but also at almost every cloud provider our of the box.

Has anybody tried this yet?
6
Developers help / Re: Load Balancer: Is there a way to react to node events ?
« Last post by dinguot on May 06, 2019, 11:52:27 AM »
Hello Laurent,

I've tried the two solutions. Both works perfectly well  :)
I think i'll go for the programmatic one.

I also set the profile to a fix one sized to one:
Code: [Select]
# it can also be the name of a user-defined algorithm. Default value is "manual"
jppf.load.balancing.algorithm = manual

# name of the set of parameter values (aka profile) to use for the algorithm
jppf.load.balancing.profile = manual_profile

# "manual" profile
jppf.load.balancing.profile.manual_profile.size = 1

I did this to ensure tasks will be computed in the same order I submitted them. Indeed I submitted the longest tasks first.
For information I did several test on JNI on both client and node side. On node side I had to limit node to one thread (as JNI is not thread safe), and set the library path:
Code: [Select]
jppf.processing.threads = 1

jppf.jvm.options = -Xmx128m -Djava.util.logging.config.file=config/logging-node.properties -Djava.library.path=./shared

  • on node, JNI is used to perform the computation (in my POC it's only waiting few seconds)
  • on client, JNI (reverse) is used to gather data and call back the jvm to create and submit the report. Basically it would allow a C/C++ client as long as it is built as a shared library

so far all is working fine !

Dinguot
7
Developers help / Re: Load Balancer: Is there a way to react to node events ?
« Last post by lolo on May 05, 2019, 10:56:39 AM »
Hello Dinguot,

Thank you for the information.
I could indeed reproduce the behavior you are observing. The problem is that, when the 2nd node is started, al the tasks have already been dispatched to the first node, one by one.

Explanation: the nodes are able to process multipe jobs concurrently. To a node, multiple dispatches of the same job are equivalent to dispatches of multiple jobs. Here, it takes very little time to send each dispatch of 1 task (as specified with 'size = 1' in the load-balancer settings), thus all dispatches have already been sent to the node when the 2nd node starts.

To resolve this this, you have several options:

- you can limit the maximum number of jobs/dispatches a node can handle concurrently, with the configuration property "jppf.node.mas.jobs = <max_jobs>". You can set it in the driver configuration, where it applies to all nodes, or set it in any or all nodes configuration, in which case it overrides the driver setting.

- you can prevent the driver from sending multiple dispatches of the same job to any given node, by setting the allowMultipleDispatchesToSameChannel attribute of the job's SLA to false, for instance:
Code: [Select]
JPPFJob job = new JPPFJob();
job.getSLA().setAllowMultipleDispatchesToSameChannel(false);

I hope this helps,
-Laurent
8
Developers help / Re: Load Balancer: Is there a way to react to node events ?
« Last post by dinguot on May 03, 2019, 02:35:50 PM »
Hello Lolo :),

thanks for your quick answer.
for now task

I already investigated manual balancer:
Code: [Select]
# it can also be the name of a user-defined algorithm. Default value is "manual"
jppf.load.balancing.algorithm = manual

# name of the set of parameter values (aka profile) to use for the algorithm
jppf.load.balancing.profile = manual_profile

# "manual" profile
jppf.load.balancing.profile.manual_profile.size = 1

Please find steps I tried:
  • Start a driver (startDriver.bat)
  • Start a node (startNode.bat)
  • Start the client
  • Wait a few to ensure computations started
  • Start a second node (startNode.bat)

It looks to me that tasks are not re-balanced when the second node starts. The first node executes all of them.

for information for investigation purpose I limilted each node to one thread:
Code: [Select]
jppf.processing.threads = 1
By the way, all this ran on 6.1 !

Dinguot
9
Developers help / Re: Load Balancer: Is there a way to react to node events ?
« Last post by lolo on May 03, 2019, 10:37:52 AM »
Hello,

Before considering a custom load-balancer, you might want to tune the one you have already configured, or use one of the available built-in algorithms.

To be able to assist further, we will need additional information regarding your usage scenario:
- could you provide you JPPF driver load-balancing settings?
- when exactly does the new node join the grid? Is it while the tasks are still executing? If so, then it is likely that all the tasks were already dispatched to the first node, and none remain to be sent to the new node. You could confirm this by testing a load-balancer setting that sends no more than (for instance) 10 tasks at a time to each node. The setting would be like this:
Code: [Select]
# name of the load-balancing algorithm
jppf.load.balancing.algorithm = manual
# name of the set of parameter s(profile) for the algorithm
jppf.load.balancing.profile = fixed_size
# "fixed_size" profile
jppf.load.balancing.profile.fixed_size.size = 10

Thank you for your time,
-Laurent
10
Forums / Re: Please post a message when you register
« Last post by dinguot on May 03, 2019, 10:23:54 AM »
hello
Pages: [1] 2 3 ... 10
JPPF Powered by SMF 2.0 RC5 | SMF © 2006–2011, Simple Machines LLC Get JPPF at SourceForge.net. Fast, secure and Free Open Source software downloads