JPPF, java, parallel computing, distributed computing, grid computing, parallel, distributed, cluster, grid, cloud, open source, android, .net
JPPF

The open source
grid computing
solution

 Home   About   Features   Download   Documentation   On Github   Forums 
July 12, 2020, 06:18:47 AM *
Welcome,
Please login or register.

Login with username, password and session length
Advanced search  
News: New users, please read this message. Thank you!
  Home Help Search Login Register  
Pages: [1]   Go Down

Author Topic: Using JPPF for a Custom Grid Solution  (Read 1294 times)

ansari.jamal

  • Guest
Using JPPF for a Custom Grid Solution
« on: July 28, 2014, 08:55:08 AM »

Hi,

I am evaluating different Open Source Grid computing model for implementing a Grid solution for following business use case:

- We want to use idle time of Desktops in our network to implement a Grid solution.
- Grid nodes will have business logic for processing in DLL running as Windows Service, with multiple thread support depending on workload (and maybe time of day) but only execute on a single unit of work at a time.
- Grid Controller running on any J2EE container like WAS or Weblogic or anyother and should be able to communicate with Grid Nodes hosting business logic in DLL
- Grid Controller should distribute the job to Grid nodes based on their current load.
- WebService based interface to start a job on Grid.
- We have around 200 million records and need to scan them with user provided input in Web Service call.
- Console base or UI based monitoring application for Grid

Need your help and expert advice on this, to find out if we can use JPPF for implementing this business use case.

Thanks,
Ansari Jamal
Logged

lolo

  • Administrator
  • JPPF Council Member
  • *****
  • Posts: 2262
    • JPPF Web site
Re: Using JPPF for a Custom Grid Solution
« Reply #1 on: August 15, 2014, 09:18:38 PM »

Hello,

My apologies for this late answer, I completely missed your post. I'll do my best to answer your request, in the hope that it's not too late and that it will be useful to you.
First, if you need background to understand my comments, I highly recommend to you read the JPPF overview.

Quote
We want to use idle time of Desktops in our network to implement a Grid solution
This is indeed one of the capabilities of the JPPF nodes. This was in fact the main focus of the 4.0 release. This can be achieved by using the node idle host mode, offline nodes, and you can even plug a screensaver, whether a personalized built-in one or a fully custom one. We also provide a demo which illustrates how this can all be put together and used.

Quote
Grid nodes will have business logic for processing in DLL running as Windows Service, with multiple thread support depending on workload (and maybe time of day) but only execute on a single unit of work at a time.
Even though the JPPF nodes are standalone Java applications, they can be wrapped as Windows services (or Linux daemons) using the Tanuki Java Service Wrapper. This is explained in our documentation here. Additionally, each node manages a configurable number of processing threads, which can be adjusted dynamically to adapt to the workload at any time.

Quote
Grid Controller running on any J2EE container like WAS or Weblogic or anyother and should be able to communicate with Grid Nodes hosting business logic in DLL
I am assuming here that Grid Controller is what we call the JPPF server or driver. The server is designed solely as a standalone Java application. Would this be a blocker for you? Would you mind explaining which specific features of a J2EE app server you are looking for? It is very possible that these features are already built-in in the JPPF architecture.

Quote
Grid Controller should distribute the job to Grid nodes based on their current load
The JPPF server provides, out of the box, multiple load balancing strategies, including algorithms which automatically adapt to the Grid workload. You may also provide your own custom load-balancing strategies. Additionally, you can define sophisticated execution strategies based on a fine-grained job SLA.

Quote
WebService based interface to start a job on Grid
Unfortunately, we do not have a web service-based interface for starting jobs. This is a feature which we hope to deliver for the 5.0 release, planned for early 2015.
What we have, however, is a J2EE connector: this is a JPPF Grid client implemented as a JCA connector and deployed within a J2EE application server. The supported application servers are Websphere Application Server, Weblogic, JBoss, Glassfish and Geronimo. With this architecture, you can easily implement and deploy a web service which uses the J2EE connector to start and monitor jobs.

Quote
We have around 200 million records and need to scan them with user provided input in Web Service call
Not a problem, this is the type of workload that JPPF is designed to handle. JPPF is especially good at distributing and executing large number of small tasks. When needed, a JPPF grid can scale horizontally to thousands of nodes and more.

Quote
Console base or UI based monitoring application for Grid
JPPF provides 2 interfaces: a Java API for management and monitoring, and a graphical administration and monitoring tool. I invite you to look at our screenshots section to get a feel of what it can do. The management and monitoring features include, but are not limited to:
- monitoring of the Grid topology, using a tree view or graph view
- monitoring and control of the nodes and servers state, with the ability to remotely restart or shutdown the grid components
- dynamic management of the nodes' number of threads and threads priority
- monitoring of the health and diagnostics help for the remote JVMs
- monitoring and control of the jobs: suspend, resume, cancel or re-prioritize dynamically
- on the fly provisioning of nodes on each physical or virtual host in the Grid which allows dynamically growing or shrinking the Grid on demand
- global Grid statistics on jobs and tasks execution, number of nodes and clients, network traffic, distributed class loading, etc.
- dynamic load-balancing reconfiguration
- user-defined charts

Sincerely,
-Laurent
« Last Edit: August 16, 2014, 07:46:08 AM by lolo »
Logged
Pages: [1]   Go Up
 
JPPF Powered by SMF 2.0 RC5 | SMF © 2006–2011, Simple Machines LLC Get JPPF at SourceForge.net. Fast, secure and Free Open Source software downloads