OSGi classloader curiosities - plugins-version-2 - java.lang.NoSuchMethodException

[Brendan Patterson]

I've done quite a deep dive in trying to figure out what seems to be a class loader issue.

I'm writing a plugin that can connect to an imap server. When I was using plugins-version-1 everything related to connecting always worked.

This now also works perfectly....sometimes in some ways, but not others. And I'm pretty sure it comes down to the OSGi container.

I actually have 4 ways to establish this connection to the email server:

1) stand alone unit test from stand alone jvm - always works 100%

2) Confluence -> servlet module - seems to always work

3) Confluence -> xwork module - not working, getting a: java.lang.NoSuchMethodException

4) Confluence -> job module - also doesn't work

So depending on which module my plugin uses it seems to have access to different objects?

I've enabled javax.mail debugging and here is the underlying issue (at least as far down as I know how to dig):

java.lang.NoSuchMethodException:

com.sun.mail.imap.IMAPSSLStore.<init>(javax.mail.Session, javax.mail.URLName)

This exact method is in the mail-1.4.5.jar I'm trying to use. And indeed it is found perfectly using methods (1) and (2) above.

I'm attaching my pom.xml (pom.txt) file as well as I've tried to define both the dependencies properly and the DynamicImport-Package section.

So two question:

* Any ideas why this class isn't being found?

* Why does the Confluence servlet module and xwork module have access to different classes? Shouldn't they be using the same APIs, same class loader?

[BenW · Accepted solution]

Short answer: You have multiple copies of Javax Mail on your classpath. Either remove your "java.mail........" dynamic import statements, or make your "javax.mail:mail" artifact provided-scoped.

Long answer:

I see that you are compile-scoping your "javax.mail:mail" artifact. Anything that is compile scoped does *not* (and should not) be specified as an OSGi bundle instruction (Import Package or Dynamic Import Package) because it will already be found on the classpath of your plugin's bundle. Anything that gets compile-scoped is essentially at the same level as the plugin's own code.

By specifying these dynamic import packages, your plugin is finding conflicting instances of this class: sometimes it uses the one bundled into the plugin, other times it uses the one which is provided from another bundle. And one of these versions does not have that method. You should check and see (via UPM's OSGi tab) in your plugin's supported product versions which package versions, if any, of this bundle are being exported. Best practice for plugins 2 plugins is to use exported packages like this when they exist. But if it either doesn't exist or is too old of a version, then remove the dynamic import packages and keep the compile-scoped artifact.

Additionally, you should only use DynamicImport-Package (as opposed to Import-Package) when you really need to, e.g. when a bundle may be installed at some point during runtime and your plugin needs to dynamically bind to it. This is the case with the Licensing API but should not be the case with any of your other specified packages. Switch those over to a <Import-Package> tag for additional plugin stability.

[David at David Simpson Apps]

@Ben, are you looking to double your karma? :-)

[BenW]

Just trying to help out the ecosystem ;)

[Brendan Patterson]

w00t! Great info! I will try toggling those.

I am pretty sure I have tried every iteration of scope and with + withtout DynamicImport-Package. However it's very good to know about the interplay here between those 'directives'.

I have cloned and debugged the JavaMailAPI source as well. I think this might actually have to do with the fact that the JavaMail API is trying to instantiate that class using reflection. Here is the offending code:

Class[] c = {javax.mail.Session.class, javax.mail.URLName.class};
	    Constructor cons = serviceClass.getConstructor(c);

I think OSGi + reflection = BAD is that right? I think that's what Don Brown told me at AtlasCamp 2011. What is the mitigation strategy for 3rd party libs that use reflection? I'm researching, but I think that's the problem point.

[Brendan Patterson]

OK thanks to Ben's comments I tried some more things and seem to be back to a partially working state. Here is my new (pom-v2.txt) Now methods 1, 2, 3 work. 4 does not - that's the scheduled job-module type. Same error. It must be loading classes differently. But I'm going try to hack around it.

I have looked through the OSGi tab from the beginning which is interesting and ususally listed what I thought it would.

I've learned a lot clearly but here are some extra learnings not documented anywhere else:

• when mucking around with OSGi stuff you can't just recompile, repackage and deploy, you really need to clean up the plugin code...thankfully not really reinstall the whole server or even restart it.
• if you're in the trenches and get a concoction working commit it to Git immediately before it evaporates.
• though I don't know how to confirm this it is possible that different plugin-module types load classes differently, I don't know why but I can sure reproduce it.
• if you've got a 3rd party API (like Java Mail) that is causing grief it might use reflection to instantiate things. that can take some extra work to sort out

(I'll write more if I figure things out - Atlassians feel free to correct my so called 'learnings' :)

[BenW]

Check out the Job Module docs. Note the part where it says:

In Confluence 4.0.1 and later, Jobs are autowired by Spring (see: CONF-20162 for workarounds in earlier versions).

By the way, you still have two compile-scoped dependencies (JUnit and Google Collections).

[Brendan Patterson]

OK I think I got everything working by routing invocations through the ServletModule.

The classloaders in one way or another seem to choke on the internal 'introspection' stuff that the Java API uses. In some cases a class cannot be found when the JavaMailAPI attempts a Class.forName("xyz").newInstance. In other cases I guess it can't read in an internal config file...this is the link to this issue (I'm amazed it's even documented)

http://www.oracle.com/technetwork/java/faq-135477.html#castmultipart

I read along the way that Confluence DOES wire different modules differently via Spring, but I don't know if that directly affects OSGi also. More reading and rereading is immenent :)

Thanks Ben and David for all your help. You guys are total Pro Circuit!

[Brendan Patterson]

Hi David,

I'm 100% interested in that issue. Thanks so much for passing it along and all your other help too!!

Brendan

[David at David Simpson Apps]

@Brendan: As an aside, when you get over this hurdle, you may also be interested in UPM-1954.

[David at David Simpson Apps]

Do you need to update your DynamicImport-Package to be a bit more vague with the package, but more exact on the version, e.g.

com.sun.mail.imap*;version="1.4.5"

instead of

com.sun.mail.imap;version="0.0"

Version 1.4.5 of everything under com.sun.mail.imap

Perhaps it's grabbing a different version of the class which is missing the method? This may be what's resulting in a NoSuchMethodException

[David at David Simpson Apps]

No half measures -- that's very bountiful.

[Brendan Patterson]

I added a bounty for this question. Claim it! :)