Hello,
Do the memory requirements of the application affect what should be chosen for these heap size values?
Yes, if you consider the tasks as part of the application. As a general rule, the tasks are the greatest memory consumers in a JPPF grid. The tasks being executed in nodes, this is where they have the most impact on heap sizing. Some elements to consider for sizing:
- a task will have a different footprint before, during and after its execution. It is best to consider their peak footprint to avoid out of memory conditions
- multiple tasks can be sent to each node at once, therefore their footrprints will add up. In this regard, load-balancing can be used as an effectiive way to limit the number of tasks sent to the nodes, and consequently to exert some measure of control over the peak memory consumption
- when receiving tasks and sending back results, a node respectively deserializes then serializes them, which creates a significant, albeit temporary, additional pressure on memory usage. A very quick approximation is that while a task and its serialized form coexist in the JVM heap, the task's footprint is doubled
The driver has different constraints. In particular, it doesn't need to deserialize the tasks, it only needs the jobs SLA, metadata and header information. The driver leverages this with a built-in mechanism that enables it to offload the tasks to disk when their memory footprint grows over a certain threshold. Of course, the drawback is that offloading tasks comes with a considerable slowdown of their processing.
Additionally, the driver is designed to work with many clients and many nodes, so it is always recommended to size it with as much memory as possible.
I hope this answersq your question.
Sincerely,
-Laurent
