Git pre-pushed object on remote server?

[Murtaza Pitalwala]

Hello,

I have a Atlassian Stash server for git.

I am looking to write a script that will run java code formatter as a pre-receive hook (before it pushes the changes to the repository).

So, what I am looking to do is **NOT** to do the work on the stash server itself rather perform the work on another server and send the status back (0 or 1) to the Stash server.

I have written the script in Python where it calls a cgi (python) script on the remote server with "ref oldrev newrev" as HTTP GET Method. Once I have the STDIN values (ref oldrev newrev) on a remote server, I created a dir, git init, git remote add origin URL, and git fetch (i even tried git pull) to get the latest contents/objects of a reporsitory in hoping to get the object that has not been pushed to the repository but its in a pre-pushed stage environment.

The hash or SHA key or "newrev" key of the object that is in the pre-pushed stage: 36ac63fe7b15049c132c310e1ee153e044b236b7

Now, when I run 'git ls-tree 36ac63fe7b15049c132c310e1ee153e044b236b7 Test.java' inside the directory I created above, it gives me error.

'fatal: not a tree object'

Now, My questions are:

How to get that object on a remote server?

What might be the git command that I run that will give me that object in that stage?

Is there any other way of doing this?

Does it make any sense of what I've asked above. Let me know if I am not clear and I will try to clear things up more.

Thanks very much in advanced for any/all the help?

[cofarrell]

Hi Murtaza,

What you're asking for is a little tricky. When you clone from a repository it is only possible to retrieve SHAs that are referenced by at least one ref. The pre-receive hooks are called before any of the pushed refs have been updated, and so they're never going to be visible externally. There is no magic trick to retrieve unreferenced SHAs that I know of.

It may be possible to create a temporary ref that points to the new SHAs. At that point you could do the fetch, validate the code on a temporary branch, and then cleanup the ref before the hook finishes. If you're going to do this I recommend not creating a branch (refs/heads/branch) but a private ref (which can be anything that's not the in 'heads' or 'tags' directory - eg refs/pre-receive-hook/random-string) , so that if someone were to fetch while the pre-receive is running they won't have remote branches being created.

Just keep in mind that this is going to slow down every push you make, which can be quite annoying for all your users.

On the Stash team we have taken a little more of an optimistic workflow. We have a pre-commit hook that does some very quick Java format validation which we check into our repository and so people can symlink into their hooks directory locally. If a developer fails to do so, or in any case, we have a validation step in our build which fails immediately if invalid code makes it through.

Does that help?

Charles

[Murtaza Pitalwala]

Hello Charles,

Thanks for the response. I like the the part where you said to create a temporary ref.

Can you point me to an example on how to create a temporary ref and point it to another directory? Or show me the commands to do this.

=======================================

I also thought of another way is to

1. Get the *.java files on the physical file system on the Stash server

2. then upload the file (with curl) to the remote server

3. Once the files are uploaded perform the work on the remote server

4. Send the response (0 or 1) back to the Stash server.

=======================================

But thats more work where if I could just create a temporary ref and point it to the new SHA in a temp dir, might be faster then my way?

Thanks for the quick response, every time :).

[cofarrell]

Hi Murtaza,

To get the physical files in Stash you're going to have to 'git cat-file' them out one at at a time because we only have a bare repository. It probably will be faster to fetch updates to your other repository.

To create (and delete) a ref you can use the update-ref command. Remember that the ref _needs_ to start with "refs/" (and ideally goes in a sub directory that isn't 'heads').

If you want to get tricky you could just get the repository directory, and write the (full) SHA to a file in "refs/" directory, and then delete it when you're done. There is a _very_ small chance the ref could be "packed" (basically moved to a flat file somewhere else) which means you're always safer using the git commands.

Again, I really would recommend relying on a build rather than a hacky, potentially slow pre-receive hook. You have been warned... ;-)

Cheers,

Charles

[Murtaza Pitalwala]

Hello Charles,

I am going to try the update-ref method and see how that goes.

But, I am not getting that, and here is what I have done.

On the Stash server, inside the /usr/local/stash/data/repo/1/objects/XX/XXXXXXX, this is where the object is.

Now, you are saying is to do the following inside a pre-receive hook:

==============================

1. create a temp directory and cd into it

2. run git init ; git remote add oring URL ; git fetch origin branch ("master" for now)

3. mkdir .git/refs/temp

4. cd .git ;

5. git update-ref refs/temp/master 'New SHA'

fatal: Cannot lock the ref '.git/refs/temp/master'.

5. I even tried: git update-ref refs/temp/master '/usr/local/stash/data/repo/1/objects/XX/XXXXXXX'

fatal: /usr/local/stash/data/repositories/1/objects/XX/XXXXXX: not a valid SHA1

==============================

Is that what you were referring to? Am I missing something?

Thanks for the warning. The simplest way of doing this is doing it on the server but my "bosses" do not want to do anything on the Stash Server itself. So, I am in charged of looking into alternatives. So far, there really arent any alternatives for Pre Receive hooks to be done on the remote server. One way of doing it is over SSH, also not an option. On the other hand, a lot more can be done with post receive hooks.

[cofarrell]

Hi Murtaza,

You can only create a ref to point to a commit, not a blob or tree. You want to create the ref to point to the new, latest commit that is being pushed. This is the getRefId() on each of the RefChange objects being passed to you in the hook.

1. In the pre-receive hook run 'git ref-update refs/temp/master $REF_ID' (using the Stash API)

2. Somewhere remote do a 'git clone' (or if you already have the repository a 'git fetch')

3. Checkout the branch and access the files, return the status code of whether it's valid or not, or whatever

4. In the pre-receive hook run 'git ref-update -d refs/tmp/master $REF_ID' (again using the Stash API). Make sure you do this in a try/finally block so you don't leave these refs dangling around if something goes wrong.

Does that make sense?

Charles

[Murtaza Pitalwala]

Charles,

As far as I know, 3 things are passed in the pre-receive hook.

1. ref = refs/heads/master

2. new sha commit id

3. old sha commit id

I am not sure what your $REF_ID refers to? What am I missing now?

[cofarrell]

Sorry. $REF_ID is the new sha commit id, so when you fetch you will guarantee to see all the new commits (on _that_ ref, you may have multiple refs in a single push).

[Murtaza Pitalwala]

Charles,

That is what I did and I posted the answer in the previous comment but it throws an error. What directory should I run that into?

5. git update-ref refs/temp/master 'New SHA' -- new commit id.

fatal: Cannot lock the ref '.git/refs/temp/master'.

[cofarrell]

Hi Murtaza,

How are you calling 'update-ref'? If you use the Java API it will just work. I know you're not running update-ref in the right directory because Stash doesn't have any '.git' directories, we have bare repositories which are the long ids of the repository. You shouldn't be running mkdir above I don't think.

You should _just_ be able to run (in the root of the Stash git repository, or preferably via the Java API which will use the correct directory):

git update-ref refs/tmp/master $REF_ID

Charles

[Murtaza Pitalwala]

Chrales,

Okay. Stash API which is update-ref?

Let me work with that and I will get back to you.