Qt based mingw32-make build called via python os.subprocess.call() fails

[Jaco Coetzee]

Using bamboo with a command task to call a python script with calls to mingw32-make.exe fails in Bamboo with the error in the log mesage below. The python script works and no errors are returned when doing the build from a command line shell.

The errors codes seem to indicate and environment issue (http://publib.boulder.ibm.com/infocenter/rsdhelp/v7r0m0/index.jsp?topic=/org.eclipse.cdt.doc.user/concepts/cdt_c_makefile.htm) - both the error codes seem to point to a missing tool / exe - but are not specific and the system path var contains the full path to the mingw bin dirs, which should be accessable to the SYSTEM (Local System) account used by Bamboo in the Windows Services.

Python script:

import sys
from subprocess import call
from os import execvp
from buildtools import directory

def build():
    
    #debug build
    sys.stdout.write('Doing debug build \n')
    sys.stdout.flush()

    # Call QMake
    retval = call('qmake')
    
    # Call jom for debug
    if (retval == 0):
        #retval = call('jom debug')
        retval = call('mingw32-make.exe debug')
    else:
        return retval
    
    sys.stdout.write('Doing release build \n')
    sys.stdout.flush()
    
    # Call jom for release
    if (retval == 0):
        #return call('jom release')
        return call('mingw32-make.exe release')
    else:
        return retval
        

if __name__ == '__main__':
     sys.exit(build())

The error log:

14-Apr-2012 07:35:01	      Must remake target `debug/.obj/main.o'.
14-Apr-2012 07:35:01	Creating temporary batch file C:\Users\Jaco\bamboo-home\xml-data\build-dir\PLAY-COMPILING-JOB1\source\src\make920-1.bat
14-Apr-2012 07:35:01	mingw32-make.exe[1]: Entering directory `C:/Users/Jaco/bamboo-home/xml-data/build-dir/PLAY-COMPILING-JOB1/source/src'
14-Apr-2012 07:35:01	g++ -c -g -frtti -fexceptions -mthreads -Wall -DUNICODE -DQT_LARGEFILE_SUPPORT -DQT_DLL -DQT_GUI_LIB -DQT_CORE_LIB -DQT_HAVE_MMX -DQT_HAVE_3DNOW -DQT_HAVE_SSE -DQT_HAVE_MMXEXT -DQT_HAVE_SSE2 -DQT_THREAD_SUPPORT -DQT_NEEDS_QMAIN -I"c:\Users\Jaco\Applications\QtSDK\Desktop\Qt\4.8.0\mingw\include\QtCore" -I"c:\Users\Jaco\Applications\QtSDK\Desktop\Qt\4.8.0\mingw\include\QtGui" -I"c:\Users\Jaco\Applications\QtSDK\Desktop\Qt\4.8.0\mingw\include" -I"c:\Users\Jaco\Applications\QtSDK\Desktop\Qt\4.8.0\mingw\include\ActiveQt" -I"debug\.moc" -I"c:\Users\Jaco\Applications\QtSDK\Desktop\Qt\4.8.0\mingw\mkspecs\default" -o debug\.obj\main.o main.cpp
14-Apr-2012 07:35:01	CreateProcess(C:\Users\Jaco\bamboo-home\xml-data\build-dir\PLAY-COMPILING-JOB1\source\src\make920-1.bat,C:\Users\Jaco\bamboo-home\xml-data\build-dir\PLAY-COMPILING-JOB1\source\src\make920-1.bat,...)
14-Apr-2012 07:35:01	Putting child 0x009d9770 (debug/.obj/main.o) PID 10367488 on the chain.
14-Apr-2012 07:35:01	Live child 0x009d9770 (debug/.obj/main.o) PID 10367488 
14-Apr-2012 07:35:01	Main thread handle = 0x00000080
14-Apr-2012 07:35:01	
14-Apr-2012 07:35:01	This application has requested the Runtime to terminate it in an unusual way.
14-Apr-2012 07:35:01	Please contact the application's support team for more information.
14-Apr-2012 07:35:02	Reaping losing child 0x009d9770 PID 10367488 
14-Apr-2012 07:35:02	mingw32-make.exe[1]: *** [debug/.obj/main.o] Error 255
14-Apr-2012 07:35:02	Cleaning up temp batch file C:\Users\Jaco\bamboo-home\xml-data\build-dir\PLAY-COMPILING-JOB1\source\src\make920-1.bat
14-Apr-2012 07:35:02	Removing child 0x009d9770 PID 10367488 from chain.
14-Apr-2012 07:35:02	mingw32-make.exe[1]: Leaving directory `C:/Users/Jaco/bamboo-home/xml-data/build-dir/PLAY-COMPILING-JOB1/source/src'
14-Apr-2012 07:35:02	Reaping losing child 0x00a198c0 PID 10630616 
14-Apr-2012 07:35:02	Removing child 0x00a198c0 PID 10630616 from chain.
14-Apr-2012 07:35:02	mingw32-make.exe: *** [debug] Error 2
14-Apr-2012 07:35:02	Doing release build

[Jaco Coetzee · Accepted solution]

I resolved the problem by creating a Windows user account for Bamboo, giving it administrator and Login as a Service rights, and using it as the account to use for the Bamboo service to login as.

[Matthew Hubble`]

I had this problem, and resovled it by creating a "Temp" directory in the Bamboo working directory, and setting the TMP and TEMP environment variables to that, before executing any commands. My batch script starts like this:

mkdir Temp

SETLOCAL
set "TEMP=%1%\Temp"
set "TMP=%1%\Temp"

Where %1 is the first argument to the script, which bamboo passes in as the path to its working directory, which is done by putting "${bamboo.build.working.directory}" as the first text in the "Argument" of the command configuration.

[Jaco Coetzee]

Nice idea - I checked and all env vars look OK except the TMP and TEMP env variables point to a non existing directory 'TMP : C:\Windows\system32\config\systemprofile\AppData\Local\Temp' and I noticed with full debug flags that mingw32-make.exe creates .bat files in my user local temp dir while under Bamboo the .bat files are in the same directory as the source. I set the TEMP var in windows to a valid dir but it gets replaced with the path above. Further, the error message "...the aplication has requested the Runtime to..." points to an unrecoverable exception in mingw32-make.exe. Using the -i or -j 1 flags also makes no diifference - it still fails with an exception. Using python's os.subprocess.check_call() confirms its an exception in mingw32-make.exe.

[Dieter]

If permissions are ok, another reason might be different environment variables in the interactive and bamboo build environment. capture them using a SET command in your build script and compare the differences