JIRA won't reindex after upgrade from 6.2 to 6.3 (Script Runner)

[Angus Law]

Hi,

After upgrading from JIRA 6.2.4 to 6.3.15, and upgrading Script Runner from 2.1.17 to 3.0.10, JIRA fails to reindex (doesn't get past 0%)

It will work if Script Runner is disabled, but this isn't really an option 

Happy to post the script itself if required for analysis. Just to confirm - it is working on the previous versions!

Thanks,
Angus 

Log error message:

2015-02-24 14:33:53,453 JiraTaskExectionThread-1 INFO Angus.Law@thomsons.com 873x831x1 rl3xsi 192.168.3.75 /secure/admin/IndexReIndex.jspa [action.admin.index.IndexAdminImpl] Re-indexing is 0% complete. Current index: Issue
2015-02-24 14:33:53,453 JiraTaskExectionThread-1 INFO Angus.Law@thomsons.com 873x831x1 rl3xsi 192.168.3.75 /secure/admin/IndexReIndex.jspa [jira.issue.index.DefaultIndexManager] Reindexing all issues
2015-02-24 14:34:25,495 IssueIndexer:thread-9 ERROR Angus.Law@thomsons.com 873x831x1 rl3xsi 192.168.3.75 /secure/admin/IndexReIndex.jspa [jira.issue.index.DefaultIndexManager] Wait attempt timed out - waited 30000 milliseconds
com.atlassian.jira.issue.index.IndexException: Wait attempt timed out - waited 30000 milliseconds
at com.atlassian.jira.issue.index.DefaultIndexManager.obtain(DefaultIndexManager.java:784)
at com.atlassian.jira.issue.index.DefaultIndexManager.access$1100(DefaultIndexManager.java:92)
at com.atlassian.jira.issue.index.DefaultIndexManager$IndexLock.tryLock(DefaultIndexManager.java:1142)
at com.atlassian.jira.issue.index.DefaultIndexManager.getIndexLock(DefaultIndexManager.java:765)
at com.atlassian.jira.issue.index.DefaultIndexManager.getIssueSearcher(DefaultIndexManager.java:830) <+3>
at java.lang.reflect.Method.invoke(Unknown Source)
at com.atlassian.util.profiling.object.ObjectProfiler.profiledInvoke(ObjectProfiler.java:83)
at com.atlassian.jira.config.component.SwitchingInvocationHandler.invoke(SwitchingInvocationHandler.java:28)
at com.sun.proxy.$Proxy14.getIssueSearcher(Unknown Source)
at com.atlassian.jira.issue.search.SearchProviderFactoryImpl.getSearcher(SearchProviderFactoryImpl.java:27)
at com.atlassian.jira.issue.search.providers.LuceneSearchProvider.search(LuceneSearchProvider.java:420)
at com.atlassian.jira.issue.search.providers.LuceneSearchProvider.search(LuceneSearchProvider.java:97)
at com.atlassian.jira.issue.search.providers.LuceneSearchProvider.search(LuceneSearchProvider.java:86)
at com.atlassian.jira.bc.issue.search.DefaultSearchService.search(DefaultSearchService.java:116)
at com.atlassian.jira.bc.issue.search.SearchService$search.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:124)
at Script1.run(Script1.groovy:66)
at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:345)
at org.codehaus.groovy.jsr223.GroovyScriptEngineImpl.eval(GroovyScriptEngineImpl.java:145)
at javax.script.AbstractScriptEngine.eval(Unknown Source)
at javax.script.ScriptEngine$eval$1.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120)
at com.onresolve.scriptrunner.runner.ScriptRunnerImpl.runStringAsScript(ScriptRunnerImpl.groovy:150)
at com.onresolve.scriptrunner.runner.ScriptRunner$runStringAsScript$4.call(Unknown Source)
at org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:45)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:108)
at org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:120)
at com.onresolve.scriptrunner.customfield.GroovyCustomField.getValueFromIssue(GroovyCustomField.groovy:147)
at com.atlassian.jira.issue.fields.CustomFieldImpl.getValue(CustomFieldImpl.java:437)
at com.atlassian.jira.issue.index.indexers.impl.NumberCustomFieldIndexer.addDocumentFields(NumberCustomFieldIndexer.java:41)
at com.atlassian.jira.issue.index.indexers.impl.NumberCustomFieldIndexer.addDocumentFieldsSearchable(NumberCustomFieldIndexer.java:31)
at com.atlassian.jira.issue.index.indexers.impl.AbstractCustomFieldIndexer.addIndex(AbstractCustomFieldIndexer.java:46)
at com.atlassian.jira.issue.index.DefaultIssueDocumentFactory$Builder.add(DefaultIssueDocumentFactory.java:102)
at com.atlassian.jira.issue.index.DefaultIssueDocumentFactory$Builder.addAll(DefaultIssueDocumentFactory.java:91)
at com.atlassian.jira.issue.index.DefaultIssueDocumentFactory.apply(DefaultIssueDocumentFactory.java:60)
at com.atlassian.jira.issue.index.DefaultIssueDocumentFactory.apply(DefaultIssueDocumentFactory.java:37)
at com.atlassian.jira.issue.index.DefaultIssueIndexer$DefaultDocumentCreationStrategy.get(DefaultIssueIndexer.java:664)
at com.atlassian.jira.issue.index.DefaultIssueIndexer$IndexIssuesOperation.perform(DefaultIssueIndexer.java:502)
at com.atlassian.jira.issue.index.DefaultIssueIndexer$4$1.get(DefaultIssueIndexer.java:337)
at com.atlassian.jira.issue.index.DefaultIssueIndexer$4$1.get(DefaultIssueIndexer.java:333)
at com.atlassian.jira.index.SimpleIndexingStrategy.get(SimpleIndexingStrategy.java:9)
at com.atlassian.jira.index.SimpleIndexingStrategy.get(SimpleIndexingStrategy.java:5)
at com.atlassian.jira.index.MultiThreadedIndexingStrategy$1.call(MultiThreadedIndexingStrategy.java:38)
at com.atlassian.jira.index.MultiThreadedIndexingStrategy$1.call(MultiThreadedIndexingStrategy.java:35)
at com.atlassian.jira.util.concurrent.BoundedExecutor$2.call(BoundedExecutor.java:85)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.lang.Thread.run(Unknown Source)
2015-02-24 14:34:25,570 IssueIndexer:thread-9 ERROR Angus.Law@thomsons.com 873x831x1 rl3xsi 192.168.3.75 /secure/admin/IndexReIndex.jspa [onresolve.scriptrunner.customfield.GroovyCustomField] *************************************************************************************
2015-02-24 14:34:25,571 IssueIndexer:thread-9 ERROR Angus.Law@thomsons.com 873x831x1 rl3xsi 192.168.3.75 /secure/admin/IndexReIndex.jspa [onresolve.scriptrunner.customfield.GroovyCustomField] Script field failed on issue: PS-8823, field: Tot.Paid CR Days
com.atlassian.jira.issue.index.SearchUnavailableException
at com.atlassian.jira.issue.index.DefaultIndexManager.getIssueSearcher(DefaultIndexManager.java:832)
at com.atlassian.util.profiling.object.ObjectProfiler.profiledInvoke(ObjectProfiler.java:83)
at com.atlassian.jira.config.component.SwitchingInvocationHandler.invoke(SwitchingInvocationHandler.java:28)
at com.sun.proxy.$Proxy14.getIssueSearcher(Unknown Source)
at com.atlassian.jira.issue.search.SearchProviderFactoryImpl.getSearcher(SearchProviderFactoryImpl.java:27)
at com.atlassian.jira.issue.search.providers.LuceneSearchProvider.search(LuceneSearchProvider.java:420)
at com.atlassian.jira.issue.search.providers.LuceneSearchProvider.search(LuceneSearchProvider.java:97)
at com.atlassian.jira.issue.search.providers.LuceneSearchProvider.search(LuceneSearchProvider.java:86)
at com.atlassian.jira.bc.issue.search.DefaultSearchService.search(DefaultSearchService.java:116)
at com.atlassian.jira.bc.issue.search.SearchService$search.call(Unknown Source)
at Script1.run(Script1.groovy:66)

[JamieA · Accepted solution]

I think you are probably running a JQL query in your script field. Somewhere between 6.2 and 6.3.x they changed it so that if you try to do that during a stop-the-world reindex it blows up.

I don;t have a good workaround for that at the moment... you will need to disable that particular field(s) before reindexing. 

[Angus Law]

Thanks - sounds like fun! I'll take a good look at the script - it's pretty crucial so we may need to hold off until we find a workaround/fix

[JamieA]

the workaround will be surrounding it with a try/catch, then after doing a full reindex do a background reindex, ideally only of those issues which have this field in their context.

[JamieA]

if you just want to get up and running, modify the script so that it exits at the top with a "return", then do a full reindex. Then remove the "return", and do a background reindex.

[Angus Law]

Thanks for your help Jamie! I'll let you know how I get on with these.

[Angus Law]

If I'm not mistaken, this is resolved with 3.0.14? :)

[JamieA]

Not to my knowledge.... it's a change in JIRA, rather than the plugin.

[Angus Law]

Ah sorry, I must have misinterpreted your release notes!