JPPF, java, parallel computing, distributed computing, grid computing, parallel, distributed, cluster, grid, cloud, open source, android, .net
JPPF

The open source
grid computing
solution

 Home   About   Features   Download   Documentation   Forums 
March 25, 2019, 02:29:15 AM *
Welcome,
Please login or register.

Login with username, password and session length
Advanced search  
News: New users, please read this message. Thank you!
  Home Help Search Login Register  
Pages: [1]   Go Down

Author Topic: JPPF node 5.2.1 keeps restarting  (Read 902 times)

webcentric

  • Guest
JPPF node 5.2.1 keeps restarting
« on: August 29, 2016, 03:40:51 PM »

Hello.

We have an odd issue with latest 5.2.1 version. When we install both driver and node, node crashes after some time. If we configure option "jppf.recovery.enabled = true" we don't have master crash, instead we get constant reconnects of the node. We tried with basic(default) configuration, without running any job/tasks. Do you have any idea what is going on? Is there any major change in 5.2 that could cause this behavior?

This is not happening with any of 5.1.x version.

Thanks,
Milos
Logged

lolo

  • Administrator
  • JPPF Council Member
  • *****
  • Posts: 2247
    • JPPF Web site
Re: JPPF node 5.2.1 keeps restarting
« Reply #1 on: August 30, 2016, 08:27:19 AM »

Hello Milos,

That seems odd indeed. Testing with freshly downloaded driver and node distributions, I do not see any such issue.
Is there anything you can see in the driver and node log files? Would you mind posting these logs?

Thanks,
-Laurent
Logged

webcentric

  • Guest
Re: JPPF node 5.2.1 keeps restarting
« Reply #2 on: August 30, 2016, 04:58:14 PM »

Hi Laurent,

Unfortunately, we didn't manage to catch log output since nodes were started with ./startNode.sh &. But, the curious thing was that, when we turned off existing nodes (connected to different driver), they were working just fine.
Do you have any idea if nodes attached to concurrent drivers might have some kind of problem running on same host?
We will try tomorrow to run again 5.2.1 driver/nodes and if anything happens we will post logs.


Regards,
Milan
Logged

lolo

  • Administrator
  • JPPF Council Member
  • *****
  • Posts: 2247
    • JPPF Web site
Re: JPPF node 5.2.1 keeps restarting
« Reply #3 on: August 31, 2016, 06:18:09 AM »

Hi Milan,

Thanks for investigating this. Running a multi-server topology on the same host is a very stable feature. We actually have automated tests that do just that as part of our continuous integration builds and they never fail.

While waiting for the logs, the only possibility I see is that issues will arise if you have a mix of 5.2.x node and a driver from a prior version or vice-versa. Is there any possibility that this could be the case in your environment?

Thanks,
-Laurent
Logged

webcentric

  • Guest
Re: JPPF node 5.2.1 keeps restarting
« Reply #4 on: August 31, 2016, 09:42:06 AM »

Hi Laurent,

Both drivers are on the same host, with different ports, if that was what you meant (of course, nodes were on remote hosts). Nodes configuration was with discovery option false (jppf.discovery.enabled = false) and with specific ports options for each driver/nodes group.
I also found some logs that might give you a hint (configuration was with specific algorithm/profile settings, different jppf.recovery.enabled and nodes were started with JPPFNode startup script from some of 5.1.x version since I was unable to find it in 5.2.1):

- jppf.recovery.enabled = true

STATUS | wrapper  | 2016/08/26 13:33:23 | --> Wrapper Started as Daemon
STATUS | wrapper  | 2016/08/26 13:33:23 | Java Service Wrapper Community Edition 64-bit 3.5.30
STATUS | wrapper  | 2016/08/26 13:33:23 |   Copyright (C) 1999-2016 Tanuki Software, Ltd. All Rights Reserved.
STATUS | wrapper  | 2016/08/26 13:33:23 |     http://wrapper.tanukisoftware.com
STATUS | wrapper  | 2016/08/26 13:33:23 |
STATUS | wrapper  | 2016/08/26 13:33:23 | Launching a JVM...
INFO   | jvm 1    | 2016/08/26 13:33:23 | WrapperManager: Initializing...
INFO   | jvm 1    | 2016/08/26 13:33:24 | node process id: 32556, uuid: A2F707FC-D603-8D20-020F-6B643B69ABF1
INFO   | jvm 1    | 2016/08/26 13:33:24 | Attempting connection to the class server at driver521.com:31111
INFO   | jvm 1    | 2016/08/26 13:33:24 | RemoteClassLoaderConnection: Reconnected to the class server
INFO   | jvm 1    | 2016/08/26 13:33:24 | JPPF Node management initialized on port 32006
INFO   | jvm 1    | 2016/08/26 13:33:24 | Attempting connection to the node server at driver521.com:31111
INFO   | jvm 1    | 2016/08/26 13:33:25 | Reconnected to the node server
INFO   | jvm 1    | 2016/08/26 13:33:25 | Node successfully initialized
INFO   | jvm 1    | 2016/08/26 13:33:59 | resetting with stopJmx=true
INFO   | jvm 1    | 2016/08/26 13:33:59 | resetting with stopJmx=true
INFO   | jvm 1    | 2016/08/26 13:33:59 | node process id: 32556, uuid: A2F707FC-D603-8D20-020F-6B643B69ABF1
INFO   | jvm 1    | 2016/08/26 13:33:59 | Attempting connection to the class server at driver521.com:31111
INFO   | jvm 1    | 2016/08/26 13:34:00 | RemoteClassLoaderConnection: Reconnected to the class server
INFO   | jvm 1    | 2016/08/26 13:34:00 | JPPF Node management initialized on port 32012
INFO   | jvm 1    | 2016/08/26 13:34:00 | Attempting connection to the node server at driver521.com:31111
INFO   | jvm 1    | 2016/08/26 13:34:00 | Reconnected to the node server
INFO   | jvm 1    | 2016/08/26 13:34:00 | Node successfully initialized
INFO   | jvm 1    | 2016/08/26 13:34:18 | resetting with stopJmx=true
INFO   | jvm 1    | 2016/08/26 13:34:18 | resetting with stopJmx=true
INFO   | jvm 1    | 2016/08/26 13:34:18 | node process id: 32556, uuid: A2F707FC-D603-8D20-020F-6B643B69ABF1
INFO   | jvm 1    | 2016/08/26 13:34:18 | Attempting connection to the class server at driver521.com:31111
INFO   | jvm 1    | 2016/08/26 13:34:18 | RemoteClassLoaderConnection: Reconnected to the class server
.
.
.


- jppf.recovery.enabled = false

STATUS | wrapper  | 2016/08/29 16:36:20 | TERM trapped.  Shutting down.
STATUS | wrapper  | 2016/08/29 16:36:21 | TERM trapped.  Forcing immediate shutdown.
STATUS | wrapper  | 2016/08/29 16:36:21 | TERM trapped.  Forcing immediate shutdown.
STATUS | wrapper  | 2016/08/29 16:36:21 | TERM trapped.  Forcing immediate shutdown.
ERROR  | wrapper  | 2016/08/29 16:36:21 | JVM did not exit on request, termination requested.
STATUS | wrapper  | 2016/08/29 16:36:21 | JVM received a signal SIGKILL (9).
STATUS | wrapper  | 2016/08/29 16:36:21 | JVM process is gone.
STATUS | wrapper  | 2016/08/29 16:36:21 | JVM exited after being requested to terminate.
STATUS | wrapper  | 2016/08/29 16:36:21 | <-- Wrapper Stopped


Btw. Nodes 5.2.1 started yesterday on empty host (without any node connected to concurrent driver) are still up and running. I activated the same kind of nodes on the host WITH concurrent nodes and they crashed after half an hour.


Regards,

Milan
Logged

lolo

  • Administrator
  • JPPF Council Member
  • *****
  • Posts: 2247
    • JPPF Web site
Re: JPPF node 5.2.1 keeps restarting
« Reply #5 on: September 01, 2016, 08:08:51 AM »

Hi Milan,

After seeing that you are running the nodes as services / daemons with the JSW, I checked in the wrapper-driver.conf and wrapper-node.conf conifguration files, and I saw that there is an incorrect version for jppf-jmxremote_optional.x.y.z.jar, and this could cause the nodes and drivers to crash:

In wrapper-driver.conf:
Code: [Select]
wrapper.java.classpath.8=lib/jppf-jmxremote_optional-1.1.1.jarshould be replaced with:
Code: [Select]
wrapper.java.classpath.8=lib/jppf-jmxremote_optional-1.2.jar
In wrapper-node.conf:
Code: [Select]
wrapper.java.classpath.7=lib/jppf-jmxremote_optional-1.1.1.jarshould be replaced with
Code: [Select]
wrapper.java.classpath.7=lib/jppf-jmxremote_optional-1.2.jar
As you can see, the fix is pretty easy. Could you please give it a try and let us know if it resolves the problem?

For reference, I registered this as bug JPPF-473 Incorrect classpath configured in wrapper-driver.conf and wrapper-node.conf

Sincerely,
-Laurent
Logged

webcentric

  • Guest
Re: JPPF node 5.2.1 keeps restarting
« Reply #6 on: September 01, 2016, 11:51:10 AM »

Hi Laurent,

Thanx for this tip, but I have already configured that at the very beginning. We started 5.2.1 yesterday on hosts without concurrent nodes and so far it is working. There was definitely problem with new nodes running on same host along with old ones (3.1 ver, I think).

Btw. When talking about wrapper config file, you should also check these lines:
wrapper.java.classpath.2=lib/jppf-common.jar
wrapper.java.classpath.3=lib/jppf-server.jar
wrapper.java.classpath.4=lib/jppf-node.jar

Regards,
Milan
Logged

marrylama

  • JPPF Padawan
  • *
  • Posts: 1
    • Full Version Software free Download
Re: JPPF node 5.2.1 keeps restarting
« Reply #7 on: October 30, 2016, 10:08:43 AM »

thnx.
Logged
<a href="http://www.softwaremess.com">Top full games</a>

anji.ram1991

  • JPPF Padawan
  • *
  • Posts: 1
Re: JPPF node 5.2.1 keeps restarting
« Reply #8 on: December 21, 2018, 11:38:41 AM »

Hai Laurent,

Am facing same problem, when am running the driver in 4.2 version and try to connect from 5.2 Node it is not connecting the same.

For this any solutions.

Thanks & Regards
Ramanjaneyulu.


Hi Milan,

Thanks for investigating this. Running a multi-server topology on the same host is a very stable feature. We actually have automated tests that do just that as part of our continuous integration builds and they never fail.

While waiting for the logs, the only possibility I see is that issues will arise if you have a mix of 5.2.x node and a driver from a prior version or vice-versa. Is there any possibility that this could be the case in your environment?

Thanks,
-Laurent
Logged

lolo

  • Administrator
  • JPPF Council Member
  • *****
  • Posts: 2247
    • JPPF Web site
Re: JPPF node 5.2.1 keeps restarting
« Reply #9 on: December 22, 2018, 03:47:29 AM »

Hello Ramanjaneyulu,

Unfortunately, there is no compatibility between a 4.2 node and a 5.2 driver. As a matter of principle, runtime compatibility of binaries between clients, drivers and nodes is guaranteed only when they all match the same major + minor version. In your situation, the best solution is to upgrade all your JPPF components to 5.2 (5.2.10 is the latest release) or even to 6.0.1 if the cost is not too high for you.

Sincerely,
-Laurent
Logged
Pages: [1]   Go Up
 
JPPF Powered by SMF 2.0 RC5 | SMF © 2006–2011, Simple Machines LLC Get JPPF at SourceForge.net. Fast, secure and Free Open Source software downloads