adequate
adequate
adequate
adequate
 

JPPF, java, parallel computing, distributed computing, grid computing, parallel, distributed, cluster, grid, cloud, open source, android, .net
JPPF

The open source
grid computing
solution

 Home   About   Features   Download   Documentation   Forums 
May 23, 2018, 05:25:44 AM *
Welcome,
Please login or register.

Login with username, password and session length
Advanced search  
News: New users, please read this message. Thank you!
  Home Help Search Login Register  
Pages: [1]   Go Down

Author Topic: Detect node Done  (Read 99 times)

shiva.verma

  • JPPF Master
  • ***
  • Posts: 27
Detect node Done
« on: February 07, 2018, 08:37:39 AM »

Hi,

Any suggestion to detect if the node is done.

Basically I am looking for (on the node machine):
1. Detect if node is not doing anything.
2. Detect if server donot have any job remaining.

I am willing to run a shell-command or a java program to detect when a node is actually done.

Thanks in advance
-Shiva
Logged

lolo

  • Administrator
  • JPPF Council Member
  • *****
  • Posts: 2221
    • JPPF Web site
Re: Detect node Done
« Reply #1 on: February 08, 2018, 10:22:27 AM »

Hello Shiva,

There are multiple ways you can detect that a node is idle. Which one to use depends on what you intend to do with the information.

1) You can use a node life cycle listener from within the node. In particular, you would be interested in the beforeNextJob() notification, which is emitted just after the node has become idle

2) You can also poll the node state remotely via the JMX-based management APIs.

a) either by connecting directly to the node (works only if you are in the same subnet as the node):

Code: [Select]
try (JMXNodeConnectionWrapper nodeJmx = new JMXNodeConnectionWrapper(host, port, isSecure)) {
  nodeJmx.connectAndWait(5000L); // wait up to 5s for the connection ot be established.
  if (nodeJmx.isConnected()) {
    JPPFNodeState.ExecutionState nodeState = nodeJmx.state();
    switch(nodeSTate) {
      case IDLE:
        // process idle node
        break;
      ...
    }
  }
}

b) by forwarding the JMX requests via the JPPF driver (will work even if the node is on a different subnet, this is what the admin console does):

Code: [Select]
String nodeIP = "192.168.1.12";
try (JMXDriverConnectionDriver driverJmx = JMXDriverConnectionDriver(host, port, isSecure)) {
  driverJmx.connectAndWait(5000L); // wait up to 5s for the connection ot be established.
  if (driverJmx.isConnected()) {
    JPPFNodeForwardingMBean forwarder = driverJmx.getNodeForwarder();
    // filter the request to only include our node with its specific IP adress
    ExecutionPolicy filter = new IsInIPv4Subnet(nodeIP);
    NodeSlector selector = new ExecutionPolicySelector(filter);
    // Get the states of the selected nodes (there should only be one)
    Map<String, Object> result = forwarder.state(selector);
    JPPFNodeState nodeState = (JPPFNodeState) result.values().iterator().next();
    JPPFNodeState.ExecutionState executionState = nodeState.getExecutionStatus();
    switch(nodeSTate) {
      case IDLE:
        // process idle node
        break;

      ...
    }
  }
}

c) Since the driver keeps the information on whether the nodes are idle, you can query the driver, by asking how many nodes are idle and by filtering the request to only include a specific node:

Code: [Select]
String nodeIP = "192.168.1.12";
try (JMXDriverConnectionDriver driverJmx = JMXDriverConnectionDriver(host, port, isSecure)) {
  driverJmx.connectAndWait(5000L); // wait up to 5s for the connection ot be established.
  if (driverJmx.isConnected()) {
    // filter the request to only include our node with its specific IP adress
    ExecutionPolicy filter = new IsInIPv4Subnet(nodeIP);
    NodeSlector selector = new ExecutionPolicySelector(filter);
    int n = driverJmx.nbIdleNodes(selector);
    if (n > 0) { // the node is idle
        // process idle node
    }
  }
}

I hope thuis answers your questions.

Sincerely,
-Laurent
Logged

shiva.verma

  • JPPF Master
  • ***
  • Posts: 27
Re: Detect node Done
« Reply #2 on: February 13, 2018, 09:51:33 PM »

Thanks Laurent for the suggestion. I will give it a try in a while and update if I encounter any issue.

I would also need suggestion to see if driver do not have any other jobs left to be shared. I want to double sure that I am not getting the node as idle in-between two jobs. I would like to kill the node (and process the post-node work activities) once I am sure there is nothing left for this node to be done.

Thanks again
Logged

lolo

  • Administrator
  • JPPF Council Member
  • *****
  • Posts: 2221
    • JPPF Web site
Re: Detect node Done
« Reply #3 on: February 14, 2018, 08:19:26 AM »

Hi Shiva,

To determine if a driver still has unfinished jobs, you can use the driver's job management MBean, via one of these two approaches:

1) Polling the driver to introspect the jobs it still has, as in this example that works from a client application:

Code: [Select]
try (JPPFClient client = new JPPFClient()) {
  JMXDriverConnectionWrapper driverJmx = client.awaitWorkingConnectionPool().awaitWorkingJMXConnection();
  // get a proxy to the remote job management MBean
  DriverJobManagementMBean jobManager = driverJmx.getJobManager();
  // obtain information on all jobs in the driver
  JobInformation[] jobInfos = jobManager.getJobInformation(JobSelector.ALL_JOBS);
  if (jobInfos.length <= 0) {
    // process the case where the driver has no job to process
  } else {
    // process the information on the jobs
  }
} catch (Exception e) {
  e.printStackTrace();
}

2) Subscribing to job status notifications, as in this example:

First you need to implement a notification listener that will process the job notifications, for example:

Code: [Select]
public class JobNotificationListener implements NotificationListener {
  // holds the uuids of the jobs remaining in the driver
  private final Set<String> jobUuids = new HashSet<>();

  @Override
  public void handleNotification(Notification notification, Object handback) {
    JobNotification jobNotif = (JobNotification) notification;
    // get the job uuid
    String uuid = jobNotif.getJobInformation().getJobUuid();
    synchronized(jobUuids) {
      switch (jobNotif.getEventType()) {
        case JOB_QUEUED:  // the job was just added to the driver
          jobUuids.add(uuid);
          break;

        case JOB_ENDED:  // the job just completed and is no longer in the driver
          jobUuids.remove(uuid);
          break;
      }
    }
  }

  public Set<String> getJoUuids() {
    synchronized(jobUuids) {
      // return a thread-safe copy
      return new HashSet<>(jobUuids);
    }
  }
}

Then, you have to register the notification listener with the job management MBean to start receiving the notifications:

Code: [Select]
try (JPPFClient client = new JPPFClient()) {
  JMXDriverConnectionWrapper driverJmx = client.awaitWorkingConnectionPool().awaitWorkingJMXConnection();
  // get a proxy to the remote job management MBean
  DriverJobManagementMBean jobManager = driverJmx.getJobManager();

  JobNotificationListener myListener = new JobNotificationListener();
  // register the job notification listener so we can receive the notifications
  jobManager.addNotificationListener(myListener, null, null);

  ...
} catch (Exception e) {
  e.printStackTrace();
}

I hope this helps,
-Laurent
Logged
Pages: [1]   Go Up
 
JPPF Powered by SMF 2.0 RC5 | SMF © 2006–2011, Simple Machines LLC Get JPPF at SourceForge.net. Fast, secure and Free Open Source software downloads