Find Questions…

Close ×
First time here? Check out the FAQ!

UWC can't upload pages to Confluence. Fails on internal null pointer exception

Doug Schmidt asked this question · 46 karma ·

I'm having a horid experience using the UWC to attempt to import our company's FogBugz wiki into Confluence. I'd really appreciate some pointers / tips on how to import pages that are already in Confluence markup format.

<rant>

Every cryptic failure I've hit makes the UWC look more and more like an incomplete, unprofessional, unsupported product. It feels like Atlassian couldn't care less about winning new business and solving actual customer problems.

There really needs to be basic quick start guide with examples to get newbies going. The current documentation of the UWC is a mess. It is half-baked. It hints that it will describe things in detail, but the links to the hints you need go nowhere, or are not written from a new users point of view (which, BTW, are the *only* type of users for UWC: ie. People who just want to use the tool once to get their data into Confluence! Grrr!)

</rant>

Since FogBugz is not a supported wiki, I wrote a Ruby script to create the "exported" wiki according to this UWC page

My exported wiki has about 400 pages (stored as UTF-8 text files in Confluence Wiki Markup), and some pages have attachments. We don't care about importing the history of each page. Just the current status of each page is fine by us.

somePath/mywiki/page1.txt
somePath/mywiki/page1/attachment1.ext
somePath/mywiki/page1/attachment2.ext
somePath/mywiki/page2.txt
somePath/mywiki/page3.txt
somePath/mywiki/page3/attachment1.ext
somePath/mywiki/page4.txt
...

I've configured my confluenceSettings.properties file to point to my empty Confluence space, and have tested the connection successfully, from both the UWC GUI and from the command line.

Properties of interest from my confluenceSettings.properties file:

wikitype=NoSyntaxConversions # Because *.txt are already in Confluence markup format
pages=somePath/mywiki
attachments= # Because UWC assumes attachments are stored in a page-relative folder
sendToConfluence=true
uploadOrphanAttachments=false

When I attempt a UWC conversion operation, UWC appears to process each of the 400 .txt files in the pages folder as its own page, since uwc.log shows a "Converting page file: foo.txt" message for each file. So far so good.

But then UWC attempts to start uploading the converted pages to my Confluence space, and UWC fails quickly, after about 4-6 successful page uploads. Here is the relevant content from the final 5 lines of my uwc.log:

2013-04-02 14:04:00,760 INFO [Thread-5] - ::: total time to convert files: 8 seconds.
2013-04-02 14:04:00,762 INFO [Thread-5] - Uploading Pages to Confluence...
2013-04-02 14:04:08,735 ERROR [Thread-5] - Unexpected problem while converting: null
2013-04-02 14:04:08,752 ERROR [Thread-5] -
Conversion Status... NONE

That's it, just a cryptic null exception message. So helpful! But on the console that launched UWC, I see the exception stack trace. Why isn't the exception stack trace part of the uwc.log? Cuz that's what the developer will want to know if/when my questions gets investigated. Here is that stack trace, BTW:
2013-04-02 14:04:00,760 INFO [Thread-5] - ::: total time to convert files: 8 seconds.
2013-04-02 14:04:00,762 INFO [Thread-5] - Uploading Pages to Confluence...
2013-04-02 14:04:08,735 ERROR [Thread-5] - Unexpected problem while converting: null
java.lang.NullPointerException
at java.util.Hashtable.put(Unknown Source)
at com.atlassian.uwc.ui.ConverterEngine.createPageTable(ConverterEngine.java:2112)
at com.atlassian.uwc.ui.ConverterEngine.sendPage(ConverterEngine.java:2014)
at com.atlassian.uwc.ui.ConverterEngine.sendPage(ConverterEngine.java:1719)
at com.atlassian.uwc.ui.ConverterEngine.writePages(ConverterEngine.java:1356)
at com.atlassian.uwc.ui.ConverterEngine.convert(ConverterEngine.java:421)
at com.atlassian.uwc.ui.ConverterEngine.convert(ConverterEngine.java:215)
at com.atlassian.uwc.ui.UWCGuiModel.convert(UWCGuiModel.java:180)
at com.atlassian.uwc.ui.listeners.ConvertListener$Worker.construct(ConvertListener.java:277)
at com.atlassian.uwc.ui.SwingWorker$2.run(SwingWorker.java:110)
at java.lang.Thread.run(Unknown Source)
2013-04-02 14:04:08,752 ERROR [Thread-5] -
Conversion Status... NONE
But there is so much more missing from the UWC log:
  • Should log the stack trace for any exceptions that occur
  • Should log the page name for each upload attempt (just like it already does for each page during the converter page), so we can figure out which pages work and which one it is crashing on.
  • Should log the name of the top-level wiki type, and probably the path of that wiki's .properties file, just to confirm what type of conversions were going to be performed.
So I'm totally in the dark about why the UWC is failing. I'm guessing I'm doing something wrong, but I have no way of diagnosing what that is. Is my filesystem layout wrong? Is my configuration wrong? Is the specific data in one of the *.txt files wrong?
Any guidance would be appreciated.
1444 views

Doug Schmidt · 46 karma ·

OK, making some progress here now.

I enabled extra debug logging in the log4j configuration by uncommenting the line:

log4j.logger.com.atlassian.uwc=DEBUG

Now I can see each attempt to upload the page.

It also appears that the suggested file system layout proposed in the UWC wiki is not correct.

I moved all the attachments to a separate folder, and dropped the ".txt" file extension from each page file.

The new file system structure I am using is:

somePath/mywiki/pages/page1
somePath/mywiki/pages/page2
somePath/mywiki/pages/page3
somePath/mywiki/pages/page4
somePath/mywiki/attachments/page1/attachment1.ext
somePath/mywiki/attachments/page1/attachment2.ext
somePath/mywiki/attachments/page3/attachment1.ext

Now all the 280 pages are able to be uploaded (10 markup errors detected. I can patch those).

None of the attachments are being uploaded though, so I'll have to figure that out still.

But at least there is some progress.

5 Answers:

Doug Schmidt · 46 karma ·

Compiled using JDK 1.6, got the minimal pages imported, and then abandoned the conversion process. 300 pages (some of which were stale and didn't need conversion) was just not a big enough peice of content to justify outsourcing the migration. We're doing it manually.

Overall, very disappointed with the realities of using the UWC. It's a very difficult task, and it requires (very unexpectedly) an engagement with a 3rd party company (the authors of UWC) to successfully convert from other formats. We're a small company with a limited budget, and simply could not afford to go further down the rabbit hole for the small amount of critical content.

Ellen Feaheny [AppFusions] · 5,839 karma ·

We are working on a Confluence Migration Toolkit -

Simplified from the current UWC.

We'll see if this helps the situation going forward. It is tricky problem.

Contact info@appfusions.com for more info.

Best,

Ellen

Ellen Feaheny [AppFusions] · 5,839 karma ·

Hi Doug

If you would like some support on the UWC, please contact us at info@appfusions.com - and we can certainly get you there.

There's ALOT to migrations, since every flavor of migration to Confluence has different tricks - and the UWC is a tool, not the solution. It helps - but not always perfect depending on the content, formats, and flavors of content being moved.

It's just not as simple as it may look - or maybe actually that is what you are figuring out?

We can likely save you alot of time and headaches with some direct support.

Ellen

Doug Schmidt · 46 karma ·

Yep, I understand that migrations are complex, and I also understand that UWC is only one tool in the complete migration solution. I get that. I've written other tools in our solution (like the Ruby script that takes our FogBugz wiki data and exports the pages in Confluence markup, and exports all the attachments to the filesystem).

So I think I've reduced our particular migration project into a very limited and managable subset. We don't care to migrate the page history, or the user audit trail of who created which page and when. We just want the current FogBugz wiki content in Confluence as close as practical.

All the FogBugz markup (html) has been transformed into Confluence markup, including:

  • Attachment links and inline images.
  • Links to FogBugz case numbers have been transformed into JIRA issue macros (our bug DB has already been succefully imported into a JIRA project)
  • Page hierarchy: Links to other FogBugz wiki pages have been transformed into Confluence [pageTitle] links

Our entire exported wiki is on the small side:

  • 280 pages, stored as UTF-8 text in Confluence markup. 940 KB total.
  • 310 attachments (82 MB in total, most are small PDFs or PNGs)

It seems like this type of constrained problem should be something that the UWC should handle quite well.

There is an existing converter (NoSyntaxConversions) which seems to imply it will do what we need (ie. the page is already in Confluence format and needs no syntax conversions).

I'm not 100% sure that my page-specific attachments are stored in the location UWC expects it (uploads are failing after the first 6 pages, none of which had any attachments), so maybe I need to store attachments elsewhere for UWC, or configure my master settings file a bit differently, but I'm not seeing any error messages or other guidance about that.

So in conclusion, I think my constrained project is something that should be supportable through the forums, and not require a costly "engagement" with a consultancy.

Ellen Feaheny [AppFusions] · 5,839 karma ·

Doug -

Pleasure to meet you.

Hmm - interesting perspective.

Here's a few points from our engineering and why it's not a simple answer - and never is with migrations (and who knows what else is under the covers..) -

1) FogBugz is not in the list of supported wikis. Could it evolve to work with it, likely - but not a guarantee out of the box. So not a good assumption.

2) On NPEs, well - we try to put in some helpful things for a dev - sorry that it is different than what you expected. Hard one.

3) One thing - you are not using the latest code - which is here:

https://bitbucket.org/appfusions/universal-wiki-converter/commits/272b4014fd9171da2df1a9ef7e9c53cef7c5d96f

How do we know - b/c line 2112 of the ConverterEngine isn't in the createPageTable method at this time.

4) NoSyntaxConversions is from pre-4.0, so it assumes wiki markup, not confluence xhtml. And it certainly doesn't support attachments. Attachments are handled by a designated AttachmentConverter

We're guessing you imported you content to something that you believe is Confluence xHTML?

There's a property: someprefix.0001.engine-markuptoxhtml.property=false that might help you.

5) Using run_cmdlin.sh - perhaps set which converter properties the UWC is going to be used. For example:

$ run_cmdline.sh confluenceSettings.properties converter.NoSyntaxConversions.properties

https://migrations.atlassian.net/wiki/display/UWC/UWC+Command+Line+Interface for details

Oh, and the link your pointed to is an example recommendation for development architecture. It's not user documentation, and is not intended to be used with a particular converter properties file. It's equivalent to us saying: hey, when you write your exporter, have a sensible file structure so its easy for you to write something that will parse that.

As for other things - likely a bunch, yet consulting a migration and the many trial runs, script tweaking without eyes to the real environment is a lose-lose proposition, we have learned.

info@appfusions.com if you would like to be constructive and get expertise and a fast successful migration.

Otherwise, at only 280 pages, small enough to do manually?

Good luck!


Doug Schmidt · 46 karma ·

Thanks Ellen, those are very helpful tips.

We are definitely right on the edge of doing stuff manually. Unfortunately, Confluence 4.3.x does not appear to allow you to manually paste their older wiki markup into their online editor. If you typed the wiki markup, char by char, instead of pasting it, then AutoComplete kicks in and everything works. But simple pasting doesn't.

As for the imported content, it is in the form of Confluence Wiki Markup, not Confluence xHTML (which I think is also called Confluence Storage Format?).

---

To be super clear, I think AppFusions is doing fine by providing UWC tool (with source, yay!) and building a business model around migrations.

What I object to is Atlassian's delegation of this (admittedly hard) problem to a 3rd party and then implying to potential customers (during their sales cycle), that migration is covered because UWC exists.

As an evaluating user, when you read all the Atlassian-written copy about UWC (and their site has tons of references to UWC), everything is written as if the migration will be straightfoward, when in fact the only truthful statement about migrating stuff into Confluence should really be "Prepare to spend some money outsourcing your migration to AppFusions. You might get lucky and be able to use a canned migration, but probably not."

But to all the non-developers at our company who looked at UWC during the decision process to move to the Atlassian platform, they all thought migration would be easy enough. It would have been nice to plan for this migration in a more realistic manner.

·3 users liked this

Doug Schmidt · 46 karma ·

Latest UWC from BitBucket doesn't compile from source.

Doug.Schmidt@DOUG-VM2008 /c/Sites/universal-wiki-converter (master)
$ ant
Buildfile: c:\Sites\universal-wiki-converter\build.xml

init:

compile.module.uwc.production:
[javac2] c:\Sites\universal-wiki-converter\build.xml:156: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
[javac2] Compiling 626 source files to c:\Sites\universal-wiki-converter\target\uwc\classes
[javac2] c:\Sites\universal-wiki-converter\src\com\atlassian\uwc\converters\IllegalPageNameConverterTest.java:480: error: unmappable character for encoding Cp1252
[javac2] input = "Detta èr en sida med îèÜ och ?Çà";
[javac2] ^
[javac2] 1 error

BUILD FAILED

c:\Sites\universal-wiki-converter\build.xml:156: Compile failed; see the compiler error output for details.

Total time: 1 second

I don't particularly care if that unit test passes, so I just changed that input string to "foo".

Doug Schmidt · 46 karma ·

Now that unit test compiles, but I now get different errors:

$ ant
Buildfile: c:\Sites\universal-wiki-converter\build.xml

init:

compile.module.uwc.production:
[javac2] c:\Sites\universal-wiki-converter\build.xml:156: warning: 'includeantruntime' was not set, defaulting to build.sysclasspath=last; set to false for repeatable builds
[javac2] Compiling 626 source files to c:\Sites\universal-wiki-converter\target\uwc\classes
[javac2] c:\Sites\universal-wiki-converter\src\com\atlassian\uwc\ui\FeedbackWindow.java:296: error: incompatible types
[javac2] method = (State.Type) methodObj;
[javac2] ^
[javac2] required: java.awt.Window.Type
[javac2] found: com.atlassian.uwc.ui.State.Type
[javac2] c:\Sites\universal-wiki-converter\src\com\atlassian\uwc\ui\FeedbackWindow.java:321: error: incomparable types: java.awt.Window.Type and com.atlassian.uwc.ui.State.Type
[javac2] if (method == State.Type.NOTE) {
[javac2] ^
[javac2] c:\Sites\universal-wiki-converter\src\com\atlassian\uwc\ui\FeedbackWindow.java:337: error: incomparable types: java.awt.Window.Type and com.atlassian.uwc.ui.State.Type
[javac2] if (method == State.Type.STEP) {
[javac2] ^
[javac2] c:\Sites\universal-wiki-converter\src\com\atlassian\uwc\ui\FeedbackWindow.java:341: error: incomparable types: java.awt.Window.Type and com.atlassian.uwc.ui.State.Type
[javac2] else if (method == State.Type.MAX) {
[javac2] ^
[javac2] Note: Some input files use or override a deprecated API.
[javac2] Note: Recompile with -Xlint:deprecation for details.
[javac2] Note: Some input files use unchecked or unsafe operations.
[javac2] Note: Recompile with -Xlint:unchecked for details.
[javac2] 4 errors

BUILD FAILED

c:\Sites\universal-wiki-converter\build.xml:156: Compile failed; see the compiler error output for details.

Total time: 4 seconds

Doug Schmidt · 46 karma ·

Here's my config.

$ java -version
java version "1.7.0_17"
Java(TM) SE Runtime Environment (build 1.7.0_17-b02)
Java HotSpot(TM) Client VM (build 23.7-b01, mixed mode, sharing)

$ ant -version
Apache Ant(TM) version 1.9.0 compiled on March 5 2013

Ellen Feaheny [AppFusions] · 5,839 karma ·

Doug -

I am sorry - we cannot continue to provide free support in this area. (ALL) Migrations are a process. They are involved, and varied (since unique to all customers' environments and content).

It is not a quick 10 minute off-the-cuff effort either - in fact, for us to throw darts to your problems here (which are many) in response w/o full picture would be IT irresponsible of us (to assume that risk on your from-wiki content).

Our team is quite busy, and we need to allocate the proper time/dedication to support you properly. I understand your frustration - and if you would like me to help you in communcating your support needs and what is involved to your management, I can help with that (I do it for many customers, which helps). Just email me.

Our support is not just a sell up - its a reality. Engineering support is not always free - nor should it be. Hard to live on free, for any of us..

I hope you can understand.

We offer short blocks of focused time as well.

Cheers,

Ellen

info@appfusions.com

Scott Harman · 310 karma ·

Hi Doug,

Build it as JDK 1.6 and that should resolve your compile error

·1 user liked this

gerhei · 1 karma ·

Hi Doug, little late but did you find any solution? Same issue here but even building as JDK 1.6 has no effect.

Looking for something else?

Find Questions…

or Browse other questions tagged:

or Ask a Question