JPPF, java, parallel computing, distributed computing, grid computing, parallel, distributed, cluster, grid, cloud, open source, android, .net
JPPF

The open source
grid computing
solution

 Home   About   Features   Download   Documentation   On Github   Forums 
November 18, 2019, 02:07:25 AM *
Welcome,
Please login or register.

Login with username, password and session length
Advanced search  
News: New users, please read this message. Thank you!
  Home Help Search Login Register  
Pages: [1]   Go Down

Author Topic: JPPFExecutorService & DataProvider  (Read 2387 times)

row_perfect

  • Guest
JPPFExecutorService & DataProvider
« on: August 24, 2012, 01:30:14 PM »

Hi Guys,

It looks to me like these two APIs are mutually exclusive.

Would it be possible to plug in a DataProvider in the same way that metadata can be set?

Thanks, Paul
Logged

lolo

  • Administrator
  • JPPF Council Member
  • *****
  • Posts: 2256
    • JPPF Web site
Re: JPPFExecutorService & DataProvider
« Reply #1 on: August 25, 2012, 09:14:13 AM »

Hi Paul,

Indeed, this is something that we missed. I registered an enhancement request for it: JPPF-50 Add data provider to the JPPFExecutor service job configuration.

Sincerely,
-Laurent
Logged

row_perfect

  • Guest
Re: JPPFExecutorService & DataProvider
« Reply #2 on: August 25, 2012, 11:04:12 PM »

Thanks Laurent for the quick and helpful response.

I've noticed something odd about the JPPFExecutorService - probably misunderstanding on my part...

I have a client that generates a large number of tasks (~100k) in a streaming manner and needs to get them running on the grid as quickly as possible - so the executor service seems ideal way of submitting them. However the behaviour I observe is that the client seems to submit only a single job (despite thousands of calls to submit()) at a time at that contains a very limited number of tasks (10).

I've tried with batching on and off (both timed and by count).


Edit: I've just turned up logging and realised it is submitting jobs, it's just that according to the admin tool, only one executes at a time.

Any ideas?


I just worked out it's to do with client connection pooling. I should RTM.

Something else that I tried, but breaks due to serialization, is ExecutorCompletionService to collect tasks as they finish - not critical, but it's a very nice pattern.
« Last Edit: August 26, 2012, 12:33:17 AM by row_perfect »
Logged

lolo

  • Administrator
  • JPPF Council Member
  • *****
  • Posts: 2256
    • JPPF Web site
Re: JPPFExecutorService & DataProvider
« Reply #3 on: August 26, 2012, 12:46:49 PM »

Hi Paul,

Thanks for the update. I registered the feature request JPPF-53 Add completion service for JPPFExecutorService

-Laurent
Logged

clgv

  • JPPF Master
  • ***
  • Posts: 25
Re: JPPFExecutorService & DataProvider
« Reply #4 on: August 27, 2012, 10:51:51 AM »

When you are at improving the JPPFExecutorService, there is something else missing.
It would be great to be able to register a TaskResultListener to the ExecutorService.
Logged
Pages: [1]   Go Up
 
JPPF Powered by SMF 2.0 RC5 | SMF © 2006–2011, Simple Machines LLC Get JPPF at SourceForge.net. Fast, secure and Free Open Source software downloads