How to write a pre-receive hook in bash or python?

[Murtaza Pitalwala]

Is there a way where you could use custom bash scripts that are currently being used with another SVN to use it with Stash as a pre-receive hook/plugin?

[cofarrell]

Hi Murtaza,

If you place your scripts under $STASH_HOME/data/repositories/$REPOID/hooks/pre-recieve.d/ as an executable file Stash/Git will invoke them as required. This will need to be done by your system administrator as Stash doesn't provide a way to manage raw scripts.

Just for the sake of completeness I should mention that we recommend the use of our repository hooks, which are Java plugins installed into Stash. They can be configured on a per-repository basis with their own custom UI. I suspect that isn't going to help much if you already have scripts, but just something to mention. Let me know if you're interested in rewriting them - we're always happy to help.

Charles

PS. Just out of curiosity, what do you scripts do? I'm wondering if there might already be something on Marketplace that does the same thing.

[Murtaza Pitalwala]

Hello Charles,

First of all, sorry for the veryyy late reply.

Second, I have completed this project where after a push is sent to Stash, it would created a Code Collaborator review with all the Error Handling and Email Alerting.

I was going to start a "New Question" based on this, but I have already asked the similar question so I thought I would just continue here.

The problem I am seeing now is as follows:

After a push has been made, it looks in the pre-receive.d hooks dir and executes any scripts that are in place here.

If I want to place a Java Code Formatting check before it would complete the push, how can I get a handle on the SHA (or transaction id as it is called in svn) and grab the necessary information such as commit title, committing user, files that have been changed etc. So that I can run the 'Java Code Formatter' and upon successful return continue with the push or reject the push.

In SVN, it sends $1 (as the repo that it is working with) and $2 (the transaction id that it is attached to at this time). In Stash, as I understand, the hooks are on per-repository basis so I dont have to worry about the repo name. But, if I can get the information as I explained above, it would be great.

Keep in mind, I am currently using Bash Scripting for this project.

Thanks,

Murtaza Pitalwala

[cofarrell]

Hi Murtaza,

No worries, it doesn't bother me. ;-)

The pre-receive hooks are just standard Git hooks, so you can refer to the normal documentation:

https://www.kernel.org/pub/software/scm/git/docs/githooks.html#pre-receive

Your script should be passed values for each ref that is being changes on stdin. From there you can run normal Git commands with those values. There should be plenty of examples to look at.

To get the list of commits you could use rev-list with formatting options:

git rev-list --format="..." $REF..$NEW_HASH

To get the files that changed:

git diff-tree --name-status -rz $REF $NEW_HASH

Just keep in mind that both pre and post receive hooks run during the push (I know that sound obvious) and the client will block until the push has finished. Basically, I would try and make your scripts run as quickly as possible, otherwise it can get quite annoying when every push is slow.

Another option, and something that we do on the Stash team, is to rely more on build failures. We have multiple checks (including bad formatting) in our build that are too slow run to run during the push but fail immediately. Because we do everything through pull requests we also have a slightly different pull request build that runs those checks (but not the slow functional tests) so we get quick feedback and (almost) never break the master build.

Finally, we also use a pre-commit hook just for checking formatting that developers can setup locally to avoid breaking any build. We check these into our main repository and people can symlink to it once and then forget about it.

Good luck.

Charles

[Murtaza Pitalwala]

Hello Charles,

Ahh I see. I thought it was something within Stash. I see your point. I will readup on the links you have suggested.

Thanks,

Murtaza Pitalwala

[Murtaza Pitalwala]

Hello Charles, I know I keep coming back to this post, and I really appreciate your feedback. I read up on git pre-commit hook and I tried to run this code

#!/bin/bash

checkingAnyArgs='/tmp/checkingAnyArgs'

while read oldrev newrev ref

do

echo "STARTING [$oldrev $newrev $ref]" >> ${checkingAnyArgs}

echo "git diff-tree --full-index -a -r \$oldrev..\$newrev -- $(git diff-tree --full-index -a -r $oldrev..$newrev) " >> ${checkingAnyArgs}

for FILE in `git diff-tree -r $oldrev..$newrev | awk '{print $6}'`

do

echo "git update-ref refs/blametree/$ref/$FILE $newrev $oldrev" >> ${checkingAnyArgs}

echo "ls -l $FILE: $(ls -l $FILE)" >> ${checkingAnyArgs}

git checkout $FILE /tmp/temp/$FILE

if [[ $(grep 'changed' $FILE) ]] ; then

echo "$FILE ' contains changed!'" >> ${checkingAnyArgs}

exit 1

fi

echo "jalopy.sh --convention URL/jalopy-1.9.4.xml --profile default -r ${FILE} -d /root/test-jalopy1" >> ${checkingAnyArgs}

/opt/jalopy/bin/jalopy.sh --convention URL/jalopy-1.9.4.xml --profile default ${FILE} -d /root/test-jalopy1 >> ${checkingAnyArgs} 2>&1

done

done

Here is the output

STARTING [2f4ccbcc830ee5c464aeb34f0468dcda4782c507 6392fa90586a0b262fb6abde643a8d955e00d32b refs/heads/master]

git diff-tree --full-index -a -r $oldrev..$newrev -- :100644 100644 ee3198f98544cda7b2075ff25f92dab63a7afc0f e5695b1a44a821041016cae7fe3063effde7b977 M Test.java

ls -l Test.java:

jalopy.sh --convention http://svn1.cvs.ula.comcast.net/svn/cvs-common/trunk/tools/jalopy/jalopy-1.9.4.xml --profile default -r Test.java -d /root/test-jalopy1

[INFO] Detecting code convention format

[INFO] Jalopy code convention detected

[INFO] Importing settings into profile "default"

[INFO] Imported 742 keys into profile "default"

No files found for path or filter expression "Test.java"

Why is it not seeing the modified file (Test.java)? What am I doing wrong here. How can I use this file?

Thanks,

Murtaza

[cofarrell]

Hi Martaza,

Assuming you're running this in Stash then it's a bare clone - the files don't actually exist on the filestem, only in Git. Which is why you see:

ls -l Test.java:

Note that it can't see the file. You can do something like:

git show $newrev:$FILE > /tmp/temp/$FILE

Or something like that. Show probably isn't the best command if you have to worry about line endings or binary files though. At that point you need to look at cat-file (and work out the blob hash).

If you get stuck I might also recommend creating a stackoverflow ticket. This is just a Git question, and I'm more than happy to help, but you'll get 1000 answers there rather than just one or two. :)

Cheers,

Charles

[Murtaza Pitalwala]

Thanks Charles. I have created a ticket at stachoverflow http://stackoverflow.com/questions/20479192/git-pre-commit-hook-run-java-formatter-before-a-push-is-committed-bash-script, and I have not gotten the answer there.

I have been seeing examples where people are doing this and its working for them, I just dont know how.

git diff --cached --name-status | while read st file; do

# skip deleted files

echo "\$st \$file: $st $file" >> ${checkingAnyArgs}

if [ "$st" == 'D' ]; then continue; fi

# do a check only on the php files

if [[ "$file" =~ ".java$" ]] && echo "php -l \"$file\"" && ! php -l "$file"; then

echo "PHP syntax check failed for file: $file" >> ${checkingAnyArgs} 2>&1

exit 1

fi

done

I tried the same code and it didnt work for me. I till read up on git cat-file.

Thanks for the help and quick response.

[cofarrell]

Hi Martaza,

What is not working about that second part? I'm assuming the php part is not running? That will have the same problem - php will need $file to exist on the filesystem, so you'll need to use cat-file or show to output the contents to another directory.

Cheers,

Charles

[Murtaza Pitalwala]

Hello Charles,

Yes, that file does not exists. But it seems like the file exists for the person who wrote the bash script that I pasted above.

I did get the cat-file to work. I will be mostly working with .java file but can cat-file option cat binary files?

[cofarrell]

Hi Martaza,

If the hook is run in a normal Git repository (ie not bare) then it probably did work. Stash uses bare repositories (to avoid having to have to waste space for all the files that we don't need anyway). It's fairly standard for Git servers to use bare repositories.

cat-file will work regardless of the file type. It outputs the exact contents from Git, unlike show. When you tested cat-file locally I'm assuming it worked for your java files?

Cheers,

Charles

[Murtaza Pitalwala]

Hello again Charles,

Would it be possible for you to shed more light at this question https://answers.atlassian.com/questions/244347?

Thanks,

Murtaza Pitalwala

[RAJESHP]

Thanks for this discussion, I have a python pre-receive force push reject hook in place per the thread and it works as expected.

Here is a related question:

the upload of the "soon to be rejected object" has already happened, so the object will be still in the repository on the server, right ? in that case what is the GC schedule for stash ?

[cofarrell]

I have answered this here:

https://answers.atlassian.com/questions/248617/pre-receive-hook-discussion-to-reject-force-push-to-git-repository

[Murtaza Pitalwala]

Hello Charles,

I have q question that I posted here: https://answers.atlassian.com/questions/249496/git-pre-pushed-object-on-remote-server.

Would you take a look at it please?

[Murtaza Pitalwala]

Yes. When I tested, it did work for the .java :) Finally.

Now, I will need to do more testing with binary files and how to work with multiple .java files attached.