Cannot rebuild indexes, Hibernate error

joshua kugler August 27, 2012

We're in the process of upgrading from Confluence 3.3 to Confluence 4.2.13. We first did the upgrade to 3.5, and that went well. The upgrade to 4.2.13 is working, but the indexes are not rebuilding. The error int he log is below.

Java 1.6.0_34

MySQL 5.5.25

Java Mysql Connector 5.1.21

This is the search index. The "Did you mean" index rebuilds with no issue. Everything else with the site seems to be working. Any ideas?

2012-08-27 17:48:52,540 WARN [confluence-interactive-reindexing-thread] [v2.c3p0.impl.DefaultConnectionTester] statusOnException SQL State '08007' of Exception tested by statusOnException() implies that the database is invalid, and the pool should refill itself with fresh Connections.
 -- referer: http://wiki-up.lab.whamcloud.com/admin/search-indexes.action | url: /admin/reindex.action | userName: joshua | action: reindex
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Communications link failure during commit(). Transaction resolution unknown.
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
	at java.lang.reflect.Constructor.newInstance(Unknown Source)
	at com.mysql.jdbc.Util.handleNewInstance(Util.java:406)
	at com.mysql.jdbc.Util.getInstance(Util.java:381)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1012)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:986)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:981)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:926)
	at com.mysql.jdbc.ConnectionImpl.commit(ConnectionImpl.java:1660)
	at com.mchange.v2.c3p0.impl.NewProxyConnection.commit(NewProxyConnection.java:803)
	at net.sf.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:63)
	at org.springframework.orm.hibernate.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:510)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:732)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:701)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:321)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:116)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
	at $Proxy87.indexEntities(Unknown Source)
	at com.atlassian.confluence.search.IndexingTask.runInternal(IndexingTask.java:35)
	at com.atlassian.confluence.util.longrunning.ConfluenceAbstractLongRunningTask.run(ConfluenceAbstractLongRunningTask.java:26)
	at java.lang.Thread.run(Unknown Source)
2012-08-27 17:48:52,544 WARN [confluence-interactive-reindexing-thread] [v2.c3p0.impl.C3P0PooledConnectionPool] doMarkPoolStatus A ConnectionTest has failed, reporting that all previously acquired Connections are likely invalid. The pool will be reset.
 -- referer: http://wiki-up.lab.whamcloud.com/admin/search-indexes.action | url: /admin/reindex.action | userName: joshua | action: reindex
2012-08-27 17:48:52,545 ERROR [confluence-interactive-reindexing-thread] [sf.hibernate.transaction.JDBCTransaction] commit Commit failed
 -- referer: http://wiki-up.lab.whamcloud.com/admin/search-indexes.action | url: /admin/reindex.action | userName: joshua | action: reindex
com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Communications link failure during commit(). Transaction resolution unknown.
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
	at java.lang.reflect.Constructor.newInstance(Unknown Source)
	at com.mysql.jdbc.Util.handleNewInstance(Util.java:406)
	at com.mysql.jdbc.Util.getInstance(Util.java:381)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1012)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:986)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:981)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:926)
	at com.mysql.jdbc.ConnectionImpl.commit(ConnectionImpl.java:1660)
	at com.mchange.v2.c3p0.impl.NewProxyConnection.commit(NewProxyConnection.java:803)
	at net.sf.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:63)
	at org.springframework.orm.hibernate.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:510)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:732)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:701)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:321)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:116)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
	at $Proxy87.indexEntities(Unknown Source)
	at com.atlassian.confluence.search.IndexingTask.runInternal(IndexingTask.java:35)
	at com.atlassian.confluence.util.longrunning.ConfluenceAbstractLongRunningTask.run(ConfluenceAbstractLongRunningTask.java:26)
	at java.lang.Thread.run(Unknown Source)
2012-08-27 17:48:52,549 ERROR [confluence-interactive-reindexing-thread] [confluence.util.longrunning.ConfluenceAbstractLongRunningTask] run Long running task "Rebuilding Search Index" failed to run.
 -- referer: http://wiki-up.lab.whamcloud.com/admin/search-indexes.action | url: /admin/reindex.action | userName: joshua | action: reindex
org.springframework.transaction.TransactionSystemException: Could not commit Hibernate transaction; nested exception is net.sf.hibernate.TransactionException: Commit failed with SQL exception: 
	at org.springframework.orm.hibernate.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:514)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.processCommit(AbstractPlatformTransactionManager.java:732)
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.commit(AbstractPlatformTransactionManager.java:701)
	at org.springframework.transaction.interceptor.TransactionAspectSupport.commitTransactionAfterReturning(TransactionAspectSupport.java:321)
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:116)
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
	at $Proxy87.indexEntities(Unknown Source)
	at com.atlassian.confluence.search.IndexingTask.runInternal(IndexingTask.java:35)
	at com.atlassian.confluence.util.longrunning.ConfluenceAbstractLongRunningTask.run(ConfluenceAbstractLongRunningTask.java:26)
	at java.lang.Thread.run(Unknown Source)
Caused by: net.sf.hibernate.TransactionException: Commit failed with SQL exception: 
	at net.sf.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:71)
	at org.springframework.orm.hibernate.HibernateTransactionManager.doCommit(HibernateTransactionManager.java:510)
	... 10 more
Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Communications link failure during commit(). Transaction resolution unknown.
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
	at java.lang.reflect.Constructor.newInstance(Unknown Source)
	at com.mysql.jdbc.Util.handleNewInstance(Util.java:406)
	at com.mysql.jdbc.Util.getInstance(Util.java:381)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1012)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:986)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:981)
	at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:926)
	at com.mysql.jdbc.ConnectionImpl.commit(ConnectionImpl.java:1660)
	at com.mchange.v2.c3p0.impl.NewProxyConnection.commit(NewProxyConnection.java:803)
	at net.sf.hibernate.transaction.JDBCTransaction.commit(JDBCTransaction.java:63)
	... 11 more

1 answer

1 accepted

5 votes
Answer accepted
Harry Chan
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
August 27, 2012

Hi, please try to add ?autoReconnect=true to the end of your database's JDBC URL and add the validationQuery parameter.

Follow this guide here https://confluence.atlassian.com/pages/viewpage.action?pageId=104300792 for more information.

joshua kugler August 27, 2012

I don't seem to have a config file as referenced by the above page. I have <HOMEDIR>/conf/confluence.cfg.xml. I'll paste it below. From the URL you pasted, I'm not 100% sure how to add the validations query. And I already do have the autoreconnect=true in the URL. Snipping the irrelevant parts.

&lt;?xml version="1.0" encoding="UTF-8"?&gt;

&lt;confluence-configuration&gt;
    &lt;property name="hibernate.c3p0.acquire_increment"&gt;1&lt;/property&gt;
    &lt;property name="hibernate.c3p0.idle_test_period"&gt;100&lt;/property&gt;
    &lt;property name="hibernate.c3p0.max_size"&gt;30&lt;/property&gt;
    &lt;property name="hibernate.c3p0.max_statements"&gt;0&lt;/property&gt;
    &lt;property name="hibernate.c3p0.min_size"&gt;0&lt;/property&gt;
    &lt;property name="hibernate.c3p0.timeout"&gt;30&lt;/property&gt;
    &lt;property name="hibernate.connection.driver_class"&gt;com.mysql.jdbc.Driver&lt;/property&gt;
    &lt;property name="hibernate.connection.password"&gt;...&lt;/property&gt;
    &lt;property name="hibernate.connection.url"&gt;jdbc:mysql://localhost/confluence?autoReconnect=true&amp;amp;sessionVariables=storage_engine%3DInnoDB&lt;/property&gt;
    &lt;property name="hibernate.connection.username"&gt;confluence&lt;/property&gt;
    &lt;property name="hibernate.database.lower_non_ascii_supported"&gt;true&lt;/property&gt;
    &lt;property name="hibernate.dialect"&gt;com.atlassian.hibernate.dialect.MySQLDialect&lt;/property&gt;
    &lt;property name="hibernate.setup"&gt;true&lt;/property&gt;
  &lt;/properties&gt;
&lt;/confluence-configuration&gt;

joshua kugler August 27, 2012
joshua kugler August 27, 2012

Tried rebuilding the index again, and had the same issue. This was after adding the validation query, and restarting confluence.

Harry Chan
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
August 27, 2012

Can you try to increase the lock wait_timeout on MySQL server? Instructions on how to do it can be found here https://confluence.atlassian.com/display/JIRAKB/Reindexing+JIRA+Results+in+Communications+Link+Failure+During+Commit

If that doesn't fix it, try to increase the global interactive timeout as documented here https://confluence.atlassian.com/display/CONFKB/Attachment+Upload+Failed+with+'Communications+link+failure+during+commit()'+Error.

It's definitely timing out before it can commit. That is the problem.

Harry Chan
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
August 27, 2012

Did you also add ?autoReconnect=true to the end of the JDBC connection URL?

joshua kugler August 27, 2012

As shown in the config file pasted above, I already had that in the URL.

joshua kugler August 27, 2012

That was it. Thanks. It was 180, and I increased it to 600.

Thank you!

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events