The goal is to provide persistence of the jobs in the driver queue, to allow recovery of the jobs whenever the driver is stopped or killed, without needing the client to resubmit.

Some preliminary considerations:

• granularity of persistence indicator: should it be at job level or driver level? (I'd rather set it per job)
• how should the jobs be persisted, i.e. write-behind vs write-through
• persistence should be pluggable, meaning we should provide an interface and at least one default implementation (e.g. file-based like we do for memory-based offload) and allow users to plug their own custom implementations
• make sure the limitations due to class loading are well understood and handled properly. In particular, we may need to revisit the job classpath management in the SLA to make it more usable and automated (i.e. ability to force the node to use the job classpath)
• what to do when the client reconnects to the driver? Should the resubmit behavior be deactivated?
• once tasks are executed, their results shall be persisted as well
• clients should have the ability to query the driver for pending/completed jobs, allowing them to retrieve the results