Having cleaned the repository locally, how do I get that onto the remote?

[Ewan McGregor]

For context, I work for an architectural visualisation studio. Myself and a number of 3D artists/motion graphics people and another coder work to produce videos and apps. The apps tend to involve fairly hefty 3D models/textures, and this has fairly frequently led to repositories becoming unnecessarily bloated.

Recently, we've had one go to 1.9 Gb, which is Too Close. Having done a variety of cleanup/size-reduction operations locally,

git count-objects -vH

tells me that my local repository is 1.44 Gb.

However, even when I force push to the Bitbucket remote, the settings page there tells me the Bitbucket repo is still 1.9 Gb. Am I missing something here? The local repo has the complete history in 1.44Gb, and I'd been under the impression that the --force flag for push would cause the remote's history to match the local. Although I've used git for years for personal projects, I've not frequently had cause to use remotes for those, so this is a bit beyond my experience. Is there something else I should be doing to make the remote carry out the cleanup/size-reduction measures?

(For the record: I'm aware that even 1.4 is too damn big for a sensible code repository. Being able to diff ASCII encoded FBX files is useful however, and I'm in the process of establishing a workflow to split the textures out rather than having to keep them in the repo.)

[evzijst · Accepted solution]

The force push did "orphan" those objects on our side, but as you suspected, that is not necessarily immediately updated on our side.

Calculating repo sizes can be relatively expensive and so we only do it once every so often, depending on a series of heuristics.

The other thing that is relevant here is that when rebasing/filter-branch'ing large objects out of your repo, that does not immediately delete them from disk. They are merely orphaned and become "garbage". You will still have to run git gc to reclaim disk space.

In turn, even git gc does not immediately reclaim space, as under normal circumstances garbage objects aren't deleted until they have reached a certain age (2 weeks default) and even then there are heuristics at play. You should be able to see this if you compare the output of git count-objects -vH with du -sh .git/ on the local repo where you performed the cleanup.

Your repo size on our end will eventually drop, but without manual intervention it's hard to give a reliable time for that.

We realize that this can cause problems for very large repos that are really close to the 2GB hard limit (as you are) and we are in the process of making changes to prune and reclaim disk space more aggressively for repos like your own.

[evzijst]

If you want, support can force a gc and reclaim on your repo. Just mail us the repo details at support@bitbucket.org

[Ewan McGregor]

Thanks for confirming. The cleanup steps I followed for local cleanup were "git reflog expire --expire=now --all", followed by a "git gc --aggressive --prune=now", which will have killed the garbage on this end. We've had to actually reset a few commits (and then force push them up), so I'm guessing those'll generate significant quantities of orphaned junk in the BitBucket repo, even if we've cleaned it out locally. Shouldn't repeat itself, since I've gotten the other coder up to speed on what shouldn't be going into commits and when they should be pushed up, but there was quite a lot of bulk data in there. We did actually end up having to do the "Open a new empty repo and use that" solution, since the original went over 2 Gb yesterday. The original's now deleted, but your answer's a better general solution, so will do that moving forward. Cheers.

[Tim Crall]

I'd have expected the force push to work.  But failing that could you just delete the remote repo and then create a new empty repo on the remote and push your local repo into it?

[Ewan McGregor]

My other suspicion is that the BitBucket repository settings page just doesn't show the size in real time, and only updates occasionally. It is at least still working for the moment, so we'll keep working on it for the next couple days. Since the local repos are much smaller, hopefully the BitBucket remote'll catch on that it's smaller too. If I'm wrong and it does hit the 2 Gb then we'll make up a new one.