unable to create page after upgrade to 6.0.1 with apache 2.2 as proxy

Andreas Nitsche November 5, 2016

Hi,

after I upgraded to latest 6.0.1 I'm not able to create pages anymore. The loading symbol does not disapear and after some time I see the message:

Loading the editor's taking longer than usual. Give it a few moments, then refresh your page if it still doesn't load. Speak to your Confluence admin if that doesn't fix it.

Refreshing the page was not helpful and I had a look at the catalina log and the atlassian-confluence.log, both do not show an error while loading a "add page".

Do you have any ideas what to try to solve this problem?

Cheers
Andi

8 answers

3 votes
Roman Samorodov
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
November 22, 2016

Hi I just changed one system property and it worked for me.

In file <Confluence home>\confluence.cfg.xml I changed synchrony.proxy.enabled property to false.

Documentation

3 votes
Rachel Robins
Atlassian Team
Atlassian Team members are employees working across the company in a wide variety of roles.
November 15, 2016

Hi everyone, 

Sorry to hear you've had problems getting collaborative editing up and running with your reverse proxy.  It's important to know your reverse proxy must support websockets. This means Apache 2.2 won't work, you need Apache 2.4.    

The following docs have some example configs that you can use as a basis for your own config.  Note that the order of directives and blocks is important in the examples. 

 

Vojtech Galda November 15, 2016

I don't agree with you. Is possible use proxy_wstunnel_module module with 2.2. @Andreas Nitsche wrote down how to get this module, see below

Hal9000 January 27, 2017

I am running Apache 2.4, but all I needed to make it work was set synchrony.proxy.enabled property to false

小白 January 17, 2018

set synchrony.proxy.enabled property to false  work for me.

my confluence version is 6.6.1

2 votes
Panos
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
November 5, 2016

The problem comes the new collaborative script which surprise surprise is running at your port 8091 by default. So, open the port in the firewall and go to Admin -> Left panel down is the Collaborative Editing -> restart the synchrony.

Panos
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
November 5, 2016

Oh! And there is a new file called atlassian-synchrony in the log folder of your app data to help you debug.

stalamoni November 14, 2016

thanks Panos, this was the solution to my problem. but in my case i've temporary turned off collaborative editing feature and i will check firewall / router problem later wink with our network admin. 

 

Mick Carroll December 15, 2016

Thanks! Worked for me. I just turned off Collaborative Editing (we dont use it)

Adrian Zhang February 15, 2017

I am using Nginx to listen 80 and forward requests to localhost 8090,only 80/tcp port opened on firewall, and meet the problem that cannot edit any page. This solution works like charming. Thanks Panos!

0 votes
Jason Hihn April 18, 2017

I am using NGINX, and I am also interested in the setting to make this work on NGINX.

 

0 votes
Panos
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
November 30, 2016

For you, (hopefully all of you) that still have problems upgrade to 6.0.2. According to their release notes there are some bug fixes regarding synchrony.

0 votes
Andreas Müller November 14, 2016

Hi,

 

I changed from Cloud to Server 6.0.1 yesterday, imported everything from the Cloud instance, have it up and running now BUT editor doesn't work. Same message as you have. I can't create Pages and also can not edit pages. 

Tried direct connection without SSL, no go.
Running NginX with SSL as proxy now, still, no go. 

Panos
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
November 14, 2016

And thats the reason I switched back to lovely 5.10 cheeky Anyway, look the inspection console to identify the error (could be different than ours?) which will give you clues of what is going on. Further more in the application folder/logs there is a new log file, something with synchrony.log. Take a look there also

Andreas Müller November 14, 2016

As a workaround I turned off 'collaborative editing'. Let's see if future versions will get a fix for that smile

Siddheshwar mhetre
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
November 24, 2016

@Panos Karampis, how do you downgrade the Conflunece to 5.10?

Andreas Nitsche November 24, 2016

If you did not made a backup before your upgrade, don't downgrade but simply turn off the collaborative editing in gneral settings -> collaborative editing.

Siddheshwar mhetre
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
November 24, 2016

Ohh.. wow.. thanks @Andreas Nitsche, it works for me smile

FOCUS S.A. November 30, 2016

Same here, i'm not able to make the collaborative editing working with nginx. Hopefully i'm in a test env.

Andreas Nitsche November 30, 2016

Which Version of nginx is in use for your confluence instance?

Jasper Golze December 3, 2016

I had the same issue. after disabling collaborative editing now everything works again fine.

Livio Sartori December 20, 2016

Just doing a clean install of confluence 6.0.2 using nginx and I have the same problem the editor does not come back I cannot create new pages

 

Andreas Nitsche December 20, 2016

@Livio Sartori, you can simply turn off collaborative editing in general settings.

0 votes
Andreas Nitsche November 10, 2016

Hey,

I made some changes to my configuration and it would be great if somebody with an apache 2.2 can verify that.

EDIT: build part of the module was missing in the explanation

Build mod_proxy_wstunnel for your apache version

# Check apache version (on my centos 6 box it's 2.2.15)
rpm -qa | grep httpd

# I would recommend to do all the build stuff in the next steps as an 
unprivileged user to not install or break anything when doing this as 
root.
# ensure these packages are installed, otherwise commands below will fail:
# - svn
# - libtool

# Checkout apache source
svn checkout http://svn.apache.org/repos/asf/httpd/httpd/tags/2.2.15/ httpd-2.2.15

# Get patch and apply it
wget https://gist.github.com/vitkin/6661683/raw/873dd8b4de4ad1ff69757ffe48fc574374aedc57/apache-2.2-wstunnel.patch
cd httpd-2.2.15
patch -p1 -i ../apache-2.2-wstunnel.patch

# Build Apache
svn co http://svn.apache.org/repos/asf/apr/apr/branches/1.4.x srclib/apr
svn co http://svn.apache.org/repos/asf/apr/apr-util/branches/1.3.x srclib/apr-util
./buildconf
./configure --enable-proxy=shared --enable-proxy_wstunnel=shared
make

# Copy the module to apache installation
sudo cp modules/proxy/.libs/mod_proxy_wstunnel.so /etc/httpd/modules

# Create module load file
cd /etc/httpd/conf.d
echo "LoadModule proxy_wstunnel_module modules/mod_proxy_wstunnel.so" &gt; mod_proxy_wstunnel.conf

 

Default proxy for the confluence application.

&lt;VirtualHost &lt;IPv4&gt;:443 &lt;IPv6&gt;:443&gt;
	...

	ProxyRequests Off
    ProxyVia Off
    SSLProxyEngine On

    ProxyPass / http://127.0.0.1:8080/
    ProxyPassReverse / http://127.0.0.1:8080/

	...
&lt;/VirtualHost&gt;

Connector in server.xml

&lt;Connector port="8080"
        maxHttpHeaderSize="8192"
        maxThreads="200"
        minSpareThreads="10"
        maxSpareThreads="75"
        enableLookups="false"
        redirectPort="8443"
        acceptCount="100"
        connectionTimeout="20000"
        debug="0"
        URIEncoding="UTF-8"
        proxyName="confluence.example.com"
        proxyPort="443"
        secure="true"
        scheme="https" /&gt;

Ensure to change the proxyName.

Add an additional VirtualHost for the synchrony service

&lt;VirtualHost &lt;IPv4&gt;:8091 &lt;IPv6&gt;:8091&gt;
	...
	SSLEngine on
	&lt;rest of ssl configuration&gt;
	...

    ProxyRequests Off
    ProxyVia Off
    SSLProxyEngine On

    ProxyPassMatch "^/synchrony/(.*/websocket.*$)" ws://127.0.0.1:8091/synchrony/$1
    ProxyPass /synchrony http://127.0.0.1:8091/synchrony

	...

&lt;/VirtualHost&gt;

Ensure only the external IPv4 and IPv6 addresses of your server are used here because confluence itself needs to be able to open the port 8091 on 127.0.0.1. The ProxyPassMatch statement is the trick. As long as confluence only calls websockets when there's a websocket in the url this will work.

Change synchrony.bind ip for your confluence instance

JAVA_OPTS="$JAVA_OPTS -Dsynchrony.bind=127.0.0.1"

Disable synchrony.proxy

The service is enabled by default, you can find this configuration statement in <confluence-home>/confluence.cfg.xml.

&lt;property name="synchrony.proxy.enabled"&gt;false&lt;/property&gt;

 

Restart Services

Depends on how you installed confluence, hopefully you know how to stop and start the application.

For apache you should do:

apachectl -S
service httpd restart

 

Would be great if somebody could check this explanation and reply if it's working this way for your setup as well.

Cheers
Andi

0 votes
Andreas Nitsche November 5, 2016

I do have an apache in front of my confluence to do the SSL handshake for me. Looks like there is the problem because the OPTION request to port 8091 is also called via https. Is there already a solution documented?

I would start to proxy this service through apache as well. Shrink the listen for port 8091 to 127.0.0.1, setup the proxy on port 8091 for the external ip including ssl and test again. I will try that later or tomorrow.

Cheers
Andi

Panos
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
November 5, 2016

ah so i am not the only one with problem when ssl

Panos
Rising Star
Rising Star
Rising Stars are recognized for providing high-quality answers to other users. Rising Stars receive a certificate of achievement and are on the path to becoming Community Leaders.
November 5, 2016
Andreas Nitsche November 5, 2016

Just an update on how far I am at the moment.

The idea to configure synchrony to localhost was more difficult than I thought at the beginning because the documentation for synchrony.bind and synchrony.port are well hidden. I checked this site to see how I can configure synchrony for data center even if I do not have one: https://confluence.atlassian.com/doc/configuring-synchrony-for-data-center-858772125.html

Once the two options

JAVA_OPTS="$JAVA_OPTS -Dsynchrony.bind=127.0.0.1"
JAVA_OPTS="$JAVA_OPTS -Dsynchrony.port=8091"

are set, synchrony starts at localhost:8091 as expected. After that I configured apache to listen on the external ip on port 8091. Make to to not only use "Listen 8091" but use "Listen x.y.z.a:8091" and if you've enabled IPv6 according "Listen a:b:c::x:8091"

At the moment my collaboration service is NOT working yet, even if the service can be accessed through Apache with SSL. This is because a websocket is used for synchrony and my apache 2.2.15 of the CentOS 6 box I run confluence, doesn't support mod_proxy_wstunnel out of the box. This will be my next try to get this working via SSL.

Sooner or later I will setup a CentOS 7 box with apache 2.4.x which brings already the module mod_proxy_wstunnel with it.

Cheers and good night
Andi

Andreas Nitsche November 5, 2016

Ok, now it's getting a good night! This is what I did to make synchrony work through apache as ssl proxy for the websocket service.

 

# Check apache version (on my centos 6 box it's 2.2.15)
rpm -qa | grep httpd

# I would recommend to do all the build stuff in the next steps as a unprivileged user to not install or brake anything when doing this as root.
# ensure these packages are installed, otherwise commands below will fail:
# - svn
# - libtool

# Checkout apache source
svn checkout http://svn.apache.org/repos/asf/httpd/httpd/tags/2.2.15/ httpd-2.2.15

# Get patch and apply it
wget https://gist.github.com/vitkin/6661683/raw/873dd8b4de4ad1ff69757ffe48fc574374aedc57/apache-2.2-wstunnel.patch
cd httpd-2.2.15
patch -p1 -i ../apache-2.2-wstunnel.patch

# Build Apache
svn co http://svn.apache.org/repos/asf/apr/apr/branches/1.4.x srclib/apr
svn co http://svn.apache.org/repos/asf/apr/apr-util/branches/1.3.x srclib/apr-util
./buildconf
./configure --enable-proxy=shared --enable-proxy_wstunnel=shared
make

# Copy the module to apache installation
sudo cp modules/proxy/.libs/mod_proxy_wstunnel.so /etc/httpd/modules

# Create module load file
cd /etc/httpd/conf.d
echo "LoadModule proxy_wstunnel_module modules/mod_proxy_wstunnel.so" > mod_proxy_wstunnel.conf

# add proxy pass to the synchrony virtual host
<VirtualHost $external_IP4:8091 $external_IP6:8091>
    ServerName your.confluence.fqdn.org

    SSLEngine on
    SSLProtocol all -SSLv2
    SSLCipherSuite ALL:!ADH:!EXPORT:!SSLv2:RC4+RSA:+HIGH:+MEDIUM
    SSLCertificateKeyFile /path/to/key/file
    SSLCertificateFile /path/to/crt/file
    SSLCertificateChainFile /path/to/intermediate/if/needed
    SSLCACertificateFile /path/to/ca/file
    SetEnvIf User-Agent ".*MSIE.*" nokeepalive ssl-unclean-shutdown

    ProxyPass /synchrony/sockjs/v1/ ws://127.0.0.1:8091/synchrony/sockjs/v1/
    ProxyPass / http://127.0.0.1:8091/
    ProxyPassReverse / http://127.0.0.1:8091/
</VirtualHost>

# check configuration and reaload
apachectl -S
service httpd reload

 

Maybe it's useful to put this all together in a single post but that can be done after some sleep. smile

Cheers from http://www.sixhop.net/ side of life
Andi

Vojtech Galda November 9, 2016

Hello Andreas,

could you post whole apache config file for confluence virt host? We have the same problem with "wss:". Thanks

Regards,

Vojta

Andreas Nitsche November 9, 2016

Hi Vojta,

it depends on your setup how your virtual host configuration need to look like. These is the VirtualHost I use for the synchrony proxy service:

<VirtualHost <IPv4>:8091 <IPv6>:8091>
    ServerAdmin <webmaster@example.com>
    DocumentRoot /var/www/html
    ServerName <confluence.example.com>

    SSLEngine on
    SSLProtocol all -SSLv2
    SSLCipherSuite ALL:!ADH:!EXPORT:!SSLv2:RC4+RSA:+HIGH:+MEDIUM
    SSLCertificateKeyFile </etc/ssl/server.key>
    SSLCertificateFile </etc/ssl/server.crt>
    SSLCertificateChainFile </etc/ssl/intermediate_ca.pem>
    SSLCACertificateFile </etc/ssl/ca.pem.cer>
    SetEnvIf User-Agent ".*MSIE.*" nokeepalive ssl-unclean-shutdown

    ProxyPass /synchrony/sockjs/v1/ ws://127.0.0.1:8091/synchrony/sockjs/v1/
    ProxyPass / http://127.0.0.1:8091/
    ProxyPassReverse / http://127.0.0.1:8091/

    ErrorLog logs/confluence-synchrony-error_log
    CustomLog logs/confluence-synchrony-access_log
</VirtualHost>

It can be that this virtual will work for you, but nobody can give a guarantee on that. All values in <> should be checked and adjusted to your needs starting by IPv4 and ending with the SSLCACertificateFile.

Would be great to hear if that works for you.

Cheers
Andi

Vojtech Galda November 9, 2016

Hi Andi,

thank you for response. I will try it. And what about virt confluence host :443? Do you use this - 

 

# Auth changes in 2.4 - see http://httpd.apache.org/docs/2.4/upgrading.html#run-time
	ProxyPass /synchrony http://&lt;internal_domain&gt;:8091/synchrony

	&lt;Location /synchrony&gt;
		Require all granted
		RewriteEngine on
		RewriteCond %{HTTP:UPGRADE} ^WebSocket$ [NC]
		RewriteCond %{HTTP:CONNECTION} Upgrade$ [NC]
		RewriteRule .* ws://&lt;internal_domain&gt;:8091%{REQUEST_URI} [P]
	&lt;/Location&gt;

 

 

how is mentioned on https://confluence.atlassian.com/kb/proxying-atlassian-server-applications-with-apache-http-server-mod_proxy_http-806032611.html

 

Regards,

Vojta

Thomas Heisterhagen January 19, 2017

Hi - I'm running two concfluence instances with a path : my.domain.tld/confluenc1 & my.domain.tld/confluence2 . But the System is calling my.domain.tld/synchrony/... in both instances instead of my.domain.tld/confluence[1|2]/synchrony/ . Did anyone found a workaround how to configure it this way?

Andreas Nitsche January 19, 2017

Hi,

https://confluence.atlassian.com/doc/configuring-synchrony-for-data-center-858772125.html

There is a config value named synchrony.context.path, looks like this is exactly what you're searching for. Have a try.

Cheers
Andi

Suggest an answer

Log in or Sign up to answer
TAGS
AUG Leaders

Atlassian Community Events