JPPF, java, parallel computing, distributed computing, grid computing, parallel, distributed, cluster, grid, cloud, open source, android, .net
JPPF

The open source
grid computing
solution

 Home   About   Features   Download   Documentation   On Github   Forums 
November 13, 2019, 01:16:49 PM *
Welcome,
Please login or register.

Login with username, password and session length
Advanced search  
News: New users, please read this message. Thank you!
  Home Help Search Login Register  
Pages: [1]   Go Down

Author Topic: Using a custom classloader  (Read 4006 times)

clgv

  • JPPF Master
  • ***
  • Posts: 25
Using a custom classloader
« on: March 27, 2012, 12:30:56 PM »

How can I use a different classloader with JPPF on the client side?
I want to use JPPF with Clojure (clojure.org). Therefore I need to insert the Clojure classloader into the classloading organization on the client side.
Logged

lolo

  • Administrator
  • JPPF Council Member
  • *****
  • Posts: 2256
    • JPPF Web site
Re: Using a custom classloader
« Reply #1 on: March 28, 2012, 08:33:24 AM »

Hello,

The JPPF client uses the class loader which loads the class of your JPPF tasks.
So, my understanding is that all you should have to do is to create instances of JPPFTask as decribed in the Closure documentation for the 'proxy' macro, add them to a JPPFJob, and submit the job.
I am working on a sample code that does that, but it's going to take some time, as I'm not familiar with Clojure.
Is this making sense?

Sincerely,
-Laurent
Logged

clgv

  • JPPF Master
  • ***
  • Posts: 25
Re: Using a custom classloader
« Reply #2 on: March 28, 2012, 09:10:45 AM »

That is exactly what I did. I have a working JPPFTask; when submitting it to the JPPFClient, I get a ClassNotFoundException from AbstractJPPFClassLoader.findClass (Line 140) for my derived JPPFTask class.

One thing I noticed is that the derived JPPFTask class has Clojure's DynamicClassLoader as ClassLoader while the imported JPPFClient class has Java's AppClassLoader as ClassLoader.

I append the example source file.


UPDATE: Please ignore the (:use debug.inspect) and the inspect call - these are debug tools of mine.
« Last Edit: March 28, 2012, 09:37:27 AM by clgv »
Logged

lolo

  • Administrator
  • JPPF Council Member
  • *****
  • Posts: 2256
    • JPPF Web site
Re: Using a custom classloader
« Reply #3 on: March 29, 2012, 07:45:11 AM »

Hi,

Thanks for providing a sample Clojure script.
I was able to reproduce the same class loading issue you are having.
This is due to the fact that, when a node need to load a class from a remote client, the client will look it up in the class loader's "classpath" via getResourceAsStream(). This means that the bytecode for the dynamic class must be accessible somewhere, for example as a file or as a byte[] kept in memory. Unfortunately, the Clojure class loader (clojure.lang.DynamicClassLoader) does not cache the bytecode, so the dynamically created classes will never be found.

So, the only way I found to make it work is to modify the code of clojure.lang.DynamicClassLoader to have it cache the bytecode of dynamically created classes, and overriding getResourceAsStream() so that it looks up into this cache if the resource is not found otherwise. I made a quick and dirty implementation which caches the byte code in memory, and I'm attaching it to this post.
To make it work, you will need to compile this version of the class loader, and make sure it is specified in the classpath (on the client side)  before the clojure-1.3.0.jar file.

By doing that I was able to run the sample and got the output "Execution result: JPPF likes Clojure.", which is a true statement  :)

Some improvements that could be made, would be to store the bytecode in (temporary) files instead of memory, and only store the file paths in memory. You might want to talk to the Clojure developers about this enhancement, I'm sure they will come up with a more elegant solution than mine.

I hope this helps.

Sincerely,
-Laurent
Logged

clgv

  • JPPF Master
  • ***
  • Posts: 25
Re: Using a custom classloader
« Reply #4 on: March 29, 2012, 10:37:26 AM »

Thanks for the fast answer.
So the problem is that the DynamicClassLoader only keeps the Class object around and not its byte[] representation?
I will discuss the problem along with your solution on the Clojure mailing list.
Thanks again.
Logged

lolo

  • Administrator
  • JPPF Council Member
  • *****
  • Posts: 2256
    • JPPF Web site
Re: Using a custom classloader
« Reply #5 on: March 29, 2012, 11:49:20 AM »

Quote
So the problem is that the DynamicClassLoader only keeps the Class object around and not its byte[] representation?
Yes, this is a good summary of the issue.

Sincerely,
-Laurent
Logged

clgv

  • JPPF Master
  • ***
  • Posts: 25
Re: Using a custom classloader
« Reply #6 on: March 29, 2012, 01:17:55 PM »

Ok, I set up a github repository with a clojure project to demonstrate the issue and to speed up the mailing list discussion (hopefully): https://github.com/guv/clj-jppf-example

Since I have not looked it up yet: Do Apache License and Eclipse Public License work well together? If not, what are the issues?
« Last Edit: March 29, 2012, 05:02:38 PM by clgv »
Logged

lolo

  • Administrator
  • JPPF Council Member
  • *****
  • Posts: 2256
    • JPPF Web site
Re: Using a custom classloader
« Reply #7 on: April 01, 2012, 08:32:09 PM »

Hi,

Quote
Do Apache License and Eclipse Public License work well together?

I believe the EPL has a provision which requires that any modification to the licensed work should be contributed back to the original work. It seems to me that's what you're trying to do here, is that correct?
If that's the case, then I don't see any issue. However I recommend you do a bit of reading on the topic, the following links may help you get started:
http://mmilinkov.wordpress.com/2009/04/30/epl-asl/
http://www.apache.org/legal/3party.html

Sincerely,
-Laurent
Logged
Pages: [1]   Go Up
 
JPPF Powered by SMF 2.0 RC5 | SMF © 2006–2011, Simple Machines LLC Get JPPF at SourceForge.net. Fast, secure and Free Open Source software downloads