JPPF, java, parallel computing, distributed computing, grid computing, parallel, distributed, cluster, grid, cloud, open source, android, .net
JPPF

The open source
grid computing
solution

 Home   About   Features   Download   Documentation   On Github   Forums 
January 20, 2021, 11:46:02 AM *
Welcome,
Please login or register.

Login with username, password and session length
Advanced search  
News: New users, please read this message. Thank you!
  Home Help Search Login Register  
Pages: [1]   Go Down

Author Topic: Memory Leak suspect in org.jppf.client.balancer.queue.JPPFPriorityQueue  (Read 1136 times)

Konstantin Eroshenko

  • Guest

Hi

Looks like there is a memory leak in org.jppf.client.balancer.queue.JPPFPriorityQueue class from jppf-client-4.2.8.jar.

JPPFPriorityQueue class has a field inherited from its parent AbstractJPPFQueue

Code: [Select]
protected final SetSortedMap<Integer, T> sizeMap = new SetSortedMap<>();
There are a few places in the class's code where values are put into this map. But I found only one place where values are removed, namely in JPPFPriorityQueue method

Code: [Select]
public ClientTaskBundle nextBundle(final ClientJob bundleWrapper, final int nbTasks)
So, as far as I understand (and as tests which ended up with OOM showed), sizeMap is not properly cleaned up when

Code: [Select]
public boolean cancelJob(final String jobId)
method is called.

Please confirm/disprove my suspection.

Thank you in advance.
Logged

lolo

  • Administrator
  • JPPF Council Member
  • *****
  • Posts: 2262
    • JPPF Web site
Re: Memory Leak suspect in org.jppf.client.balancer.queue.JPPFPriorityQueue
« Reply #1 on: October 16, 2015, 01:11:40 PM »

Hello,

Thank you very much for this report and your analysis. I confirm there is a memory leak due to improper cleanup of the sizeMap.
In my testing I also realized that it only happens when the jobs are not sent all at once to the server (i.e. subsets of the tasks, until the whole set of tasks is executed) , most likely due to the load balancer settings. Thus, maybe that gives you a temporary workaround (changing the load balancer settings in the client)?

Anyway, I registered a bug for this: JPPF-418 Memory leak in client queue
I'm working on a fix and will deliver a patch ASAP. I'll update you on my progress in this thread.

Sincerely,
-Laurent
Logged

lolo

  • Administrator
  • JPPF Council Member
  • *****
  • Posts: 2262
    • JPPF Web site
Re: Memory Leak suspect in org.jppf.client.balancer.queue.JPPFPriorityQueue
« Reply #2 on: October 18, 2015, 09:32:34 AM »

Hello again,

I wanted to let you know that I implemented a fix for JPPF-418.
This fix is now available in the patch 01 for JPPF 4.2.8

Sincerely,
-Laurent
Logged

Konstantin Eroshenko

  • Guest
Re: Memory Leak suspect in org.jppf.client.balancer.queue.JPPFPriorityQueue
« Reply #3 on: October 19, 2015, 09:15:36 AM »

Hi, Laurent

You've made the patch rather fast, thanks a lot once again.
I'll write you in addition when I test it.
Pretty sure, it's all right, but nevertheless.
Logged

Konstantin Eroshenko

  • Guest
Re: Memory Leak suspect in org.jppf.client.balancer.queue.JPPFPriorityQueue
« Reply #4 on: October 20, 2015, 07:49:02 AM »

Hello, Laurent

The memory leak definitely has gone.
Could you please build corresponded artifact and publish to maven repository?
I have difficulties with artifact substitution in our repository, due to bureaucratic machinery.

Thanks

Logged

lolo

  • Administrator
  • JPPF Council Member
  • *****
  • Posts: 2262
    • JPPF Web site
Re: Memory Leak suspect in org.jppf.client.balancer.queue.JPPFPriorityQueue
« Reply #5 on: October 20, 2015, 09:11:37 AM »

Hi,

Quote
Could you please build corresponded artifact and publish to maven repository?
For this I will need to deliver a new (and last I hope) maintenance release v4.2.9. This will take a couple of days since I don't have too much bandwith right now. This should be done by ths end of this week, probably during the week-end.

-Laurent
Logged

lolo

  • Administrator
  • JPPF Council Member
  • *****
  • Posts: 2262
    • JPPF Web site
Re: Memory Leak suspect in org.jppf.client.balancer.queue.JPPFPriorityQueue
« Reply #6 on: October 25, 2015, 07:47:11 AM »

This is now done. The new files should be in Maven Central in a couple of hours from the time I'm writing this.

-Laurent
Logged
Pages: [1]   Go Up
 
JPPF Powered by SMF 2.0 RC5 | SMF © 2006–2011, Simple Machines LLC Get JPPF at SourceForge.net. Fast, secure and Free Open Source software downloads