Help Migrating a 4.5 GB Azure Devops Git Repo to Github

[dkonasani]

Select Topic Area

Question

Body

Hi,

We are trying to migrate all our repos from Azure Devops to Github. And we are partially successful by migrating all the small repos. But we are having issue migrating a 4.8gb repo to github. The approaches we took for migration are:

1. Manually importing from Github web page using import code. And it failed with the following error:

2. By mirror cloning and pushing mirror using the following code:
   git clone --bare $cloneUrl git remote add $githubUser/$githubRepo $githubOriginUrl git push --mirror $githubPushUrl

   And the error we come across is:
   error: RPC failed; HTTP 500 curl 22 The requested URL returned error: 500
   send-pack: unexpected disconnect while reading sideband packet
   Writing objects: 100% (1301469/1301469), 4.71 GiB | 1.69 MiB/s, done.
   Total 1301469 (delta 967538), reused 1301469 (delta 967538), pack-reused 0
   fatal: the remote end hung up unexpectedly
   Everything up-to-date

3. By cloning and pushing a single branch using the below:
   git clone --branch $branch --origin origin --progress -v $cloneUrl git remote rm origin git remote add origin $githubOriginUrl git push --progress -v $githubPushUrl

   And the error we come across is:
   Compressing objects: 100% (301024/301024), done.
   error: RPC failed; HTTP 500 curl 22 The requested URL returned error: 500
   send-pack: unexpected disconnect while reading sideband packet
   Writing objects: 100% (1250103/1250103), 4.38 GiB | 3.76 MiB/s, done.
   Total 1250103 (delta 933111), reused 1249264 (delta 932508), pack-reused 0
   fatal: the remote end hung up unexpectedly
   Everything up-to-date

Any help on how to debug and fix this issue would be awesome

Thanks and Regards

[airtower-luna]

There are file size limits, if your repository is that large maybe it contains files that are too large? https://docs.github.com/en/repositories/working-with-files/managing-large-files/about-large-files-on-github

I also remember reading somewhere that the size of a push is limited, though I can't find it in the docs right now. But pushing 4 GB at once would probably hit that. 😅

[dkonasani]

Hi @airtower-luna
Thanks for looking in to this. But per my understanding Github Repos limit the total size of the repo to be less than 5GB and a single file to be less than 100MB and we are not hitting either of those.
However i am not sure on what is the limit to push at once.
Can anybody let me know how we can find that limit and if there is a way to incrementally push the commits rather than all at once

[airtower-luna]

You can try to do something like git push origin 123456:main, which would try to push commit 123456 (make sure to substitute a valid commit ID) to the remote branch main. That way you can move from older to newer commits.

Mind that making the repository smaller would still probably be a good idea, a repository that's naturally that large is pretty rare, but I don't know your specific case.

[alienarena]

I'm running into the same issue. I'm trying to migrate from svn to github.
The output below is very similar to yours:

error: RPC failed; HTTP 500 curl 22 The requested URL returned error: 500
fatal: the remote end hung up unexpectedly
Writing objects: 100% (73256/73256), 2.67 GiB | 10.53 MiB/s, done.
Total 73256 (delta 49814), reused 70421 (delta 46979)
fatal: the remote end hung up unexpectedly
Everything up-to-date

[github-actions[bot]]

🕒 Discussion Activity Reminder 🕒

This Discussion has been labeled as dormant by an automated system for having no activity in the last 60 days. Please consider one the following actions:

1️⃣ Close as Out of Date: If the topic is no longer relevant, close the Discussion as out of date at the bottom of the page.

2️⃣ Provide More Information: Share additional details or context — or let the community know if you've found a solution on your own.

3️⃣ Mark a Reply as Answer: If your question has been answered by a reply, mark the most helpful reply as the solution.

Note: This dormant notification will only apply to Discussions with the Question label. To learn more, see our recent announcement.

Thank you for helping bring this Discussion to a resolution! 💬