A live mirror for project's subdirectory

[Peter “Ӹ” Rybin]

I have a huge git repository on Stash. It's so big, that it becomes painful to use, especially on a low-performance laptops: git checkout, git status are very slow, even git branch takes a while before responding. However, only a particular subdirectory is of interest for the most of laptop users.

I imagine a setting: a subdirectory gets mirrored into a separate git project in a live fashion, i.e. all commits and branches get synchronized both way. Theoretically it doesn't seem impossible to me. How possible is that in practice? Could it be a feature for Stash?

[Balázs Szakmáry · Accepted solution]

What you are proposing is more complex than it sounds and there is currently no solution for it.

The standard way of making this happen would be to separate the subdirectory into a submodule. You can use this command to create the smaller repo with only the subfolder:

git filter-branch --subdirectory-filter <Foldername> -- --all

and then delete the original subfolder in the giant repo and replace it with a submodule ref.

Pro: the users who only work on this folder will be able to clone/pull/push only the submodule repo.

Con: Somebody will need to make sure that the submodule ref. always points to the right place in the main repo. (Or that it gets periodically set to the right place.)

[Peter “Ӹ” Rybin]

Thanks for the answer. I haven't thought it out carefully. Still sounds like an interesting toy project. Submodules are fine, but we generally enjoy atomic commits throughout the project, so it's unlikely we would split the repository this way.

[Balázs Szakmáry]

The main problem is that the commits will not be identical (different hashes) in the two repos. But I guess with git fast-import/export you should be able to construct commits with the same content in the repo on the other side. If you make it a Stash plugin, publish it. :)

[Peter “Ӹ” Rybin]

Would you encourage me to try it? It probably requires to store a mapping between revisions somewhere on disk. I'm just completely new to Stash plugin premises.

[Johannes Kilian]

Another thing you might consider:

If you don't need the complete history while developing your project, you may want to use shallow clone:

git clone --depth=XXXXX <remote_repo_url>

From git clone documentaion:

--depth 
Create a shallow clone with a history truncated to the specified number of revisions.

There are several other suggestions to ease the pain of huge repositories: here's an article you might want to look at ...

[Peter “Ӹ” Rybin]

sparse checkout looks very very relevant. i should try its performance.