JPPF Issue Tracker
star_faded.png
Please log in to bookmark issues
bug_report_small.png
CLOSED  Bug report JPPF-157  -  Improper handling of serialized tasks with a size > 2GB
Posted Jun 19, 2013 - updated Dec 27, 2014
icon_info.png This issue has been closed with status "Closed" and resolution "RESOLVED".
Issue details
  • Type of issue
    Bug report
  • Status
     
    		 Closed
  • Assigned to
    lolo4j
  • Progress
       
  • Type of bug
    Not triaged
  • Likelihood
    Not triaged
  • Effect
    Not triaged
  • Posted by
    lolo4j
  • Owned by
    lolo4j
  • Category
    Core
  • Resolution
    RESOLVED
  • Priority
    Normal
  • Reproducability
    Always
  • Severity
    Normal
  • Targetted for
    icon_milestones.png JPPF 3.3.x
Issue description
When a node is returning, after execution, a task whose size is larger than 2GB, the following exception is raised in the driver upon receiving this task: 
java.lang.NegativeArraySizeException
  at org.jppf.io.MultipleBuffersLocation.<init>(MultipleBuffersLocation.java:70)
  at org.jppf.io.IOHelper.createDataLocationMemorySensitive(IOHelper.java:74)
  at org.jppf.server.nio.AbstractNioMessage.readNextObject(AbstractNioMessage.java:155)
  at org.jppf.server.nio.AbstractNioMessage.read(AbstractNioMessage.java:106)
  at org.jppf.server.nio.nodeserver.AbstractNodeContext.readMessage(AbstractNodeContext.java:232)
  at org.jppf.server.nio.nodeserver.WaitingResultsState.performTransition(WaitingResultsState.java:69)
  at org.jppf.server.nio.nodeserver.WaitingResultsState.performTransition(WaitingResultsState.java:1)
  at org.jppf.server.nio.StateTransitionTask.run(StateTransitionTask.java:82)
  at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
  at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
  at java.util.concurrent.FutureTask.run(FutureTask.java:166)
  at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
  at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
  at java.lang.Thread.run(Thread.java:722)

This is due to a limitation in the communication protocol between node and server. Each message between node and server is made of serialized objects, where each serialized object is preceded by an integer that determines its size. When the size is > 2GB, there is an integer overflow which can cause the node to write, and the server to raed, a negative size.

I also see that this causes the server to disconnect form the node, and resubmiit the task to the same or another node, thus the job never terminates.

The same kind of issue would occur between client and server and needs to be addressed as well.
Steps to reproduce this issue
- configure a node with 4 GB of heap (-Xmx4g)

- submit a job with the following task: 
public class SerializationOverflowTask extends JPPFTask {
  private List<byte[]> data = null;
 
  @Override
  public void run() {
    int size = 256 * 1024 * 1024; // 256 MB
    data = new ArrayList<byte[]>();
    for (int i=0; i<10; i++) data.add(new byte[size]);
  }
}

==> you get the NegativeArraySizeException in the driver's log

#5
Comment posted by
lolo4j
Jun 23, 09:57
Fixed. Changes committed to SVN:

The issue was updated with the following change(s):
  • This issue has been closed
  • The status has been updated, from New to Closed.
  • This issue's progression has been updated to 100 percent completed.
  • The resolution has been updated, from Not determined to RESOLVED.
  • Information about the user working on this issue has been changed, from lolo4j to Not being worked on.
  • Time spent on this issue, from No time spent to 3 days, 14 hours.