GitHub's Git Hosting Pain Points and a New Approach with Continuity

Cursor Origin specially designed for agentic scale. One click migrate your GitHub repos so you're not limited by more code, more PRs, more CI runs + version control challenges

editor that understands the entire codebase and cloud commands to complete tasks itself . devops : one repository. 100 environments

We're making Git hosting more reliable, performant, and scalable. This post traces 20 years of Git infrastructure and explains how that history led us to design and operate our Git storage, Origin, as if it were a database. https://lnkd.in/gUFnSYv7

A distributed version control system means that all instances of a repository are identical. There are many hard scalability and reliability challenges that make this quite difficult. Git compresses and stores code and metadata into packfiles, a simple binary serialization format that isn't ideal for managing at scale on a server. These large binary files must exist on a file system for Git to access them. https://lnkd.in/eN4jYCVd

In midst of all GitHub outages , this is the coolest paper I have read in recent times on running git in distributed environment https://lnkd.in/d5HjAsh2

A couple of months ago, I speculated the slop code generated by LLMs will start bringing the infrastructure to its knees. Well, it looks like the LLMs wasteful development practices got there first. What you are seeing happening with GitHub is only the beginning of what's to come. The slop/waste is not free and it will have to be paid in hard cash.

GitHub went from 1.4 billion commits a month in April to 2.9 billion by August. Four months. That is not just a growth story. It is an architecture story. GitHub was built around human software-development behavior: developers clone repositories, work locally, commit, push, open pull requests and run CI. Agentic development changes that workload. Agents can clone repeatedly, inspect large histories, create branches, generate commits, run tests, call APIs and discard much of that work before a human ever sees it. GitHub’s recent outages make the pressure visible. The August 17 outage lasted nearly eight hours. GitHub said a critical component in its Central US data center failed to scale during record traffic, and retries made recovery harder. This was not unexpected. GitHub began planning for 10× capacity in October 2025. By February, 10× was already not enough. The target became 30×. But GitHub is not simply adding more servers. It is also isolating services, removing shared failure points, improving failover and moving more workloads onto more elastic infrastructure. Because at some point the question stops being: How much more capacity can we add? and becomes: Does the architecture still match the workload? That is where some of the newer approaches become interesting. Entire, founded by former GitHub CEO Thomas Dohmke, is building a distributed Git network aimed at agent-driven development. GitHub can remain the source repository while agents pull from distributed mirrors instead of repeatedly hitting the same centralized infrastructure. Cursor is taking another approach with Origin: becoming a Git forge itself, with repositories, push/pull, pull requests, browsing, agents and GitHub mirroring. These are not necessarily solutions to GitHub’s specific outages. But they point toward something bigger: the market is starting to reconsider where agent workload should live. GitHub grew up serving humans. Now software-producing machines are becoming a significant source of workload on the same infrastructure — operating faster, with more concurrency and generating far more intermediate activity. So maybe the long-term answer is not simply: Make GitHub 30× bigger. Maybe it is: Stop sending unnecessary machine-generated work through the most expensive shared path. Are GitHub’s recent capacity problems temporary growing pains, or are we starting to see source-control infrastructure itself change for the agent era?

𝗚𝗶𝘁 𝗵𝗼𝘀𝘁𝗶𝗻𝗴 𝗹𝗼𝗼𝗸𝘀 𝘀𝗶𝗺𝗽𝗹𝗲 𝘂𝗻𝘁𝗶𝗹 𝗶𝘁 𝗯𝗿𝗲𝗮𝗸𝘀 𝗮𝘁 𝘀𝗰𝗮𝗹𝗲 Cursor is making Git hosting more reliable, performant, and scalable. In its latest engineering write-up, the team traces 𝟮𝟬 𝘆𝗲𝗮𝗿𝘀 𝗼𝗳 𝗚𝗶𝘁 𝗶𝗻𝗳𝗿𝗮𝘀𝘁𝗿𝘂𝗰𝘁𝘂𝗿𝗲 and explains why that history led them to design and operate Git storage, 𝗢𝗿𝗶𝗴𝗶𝗻, more like a 𝗱𝗮𝘁𝗮𝗯𝗮𝘀𝗲 than a filesystem. 𝗚𝗶𝘁 𝘄𝗮𝘀 𝗯𝘂𝗶𝗹𝘁 𝗳𝗼𝗿 𝗮 𝗱𝗶𝗳𝗳𝗲𝗿𝗲𝗻𝘁 𝘄𝗼𝗿𝗸𝗳𝗹𝗼𝘄 Git was originally designed for Linux kernel development: highly decentralized, with many maintainers working independently. That design made sense for its original use case. 𝗠𝗼𝘀𝘁 𝘁��𝗮𝗺𝘀 𝘂𝘀𝗲 𝗚𝗶𝘁 𝗶𝗻 𝗮 𝗰𝗲𝗻𝘁𝗿𝗮𝗹𝗶𝘇𝗲𝗱 𝘄𝗮𝘆 Today, most open-source teams and nearly every company rely on a 𝗰𝗲𝗻𝘁𝗿𝗮𝗹 𝗵𝗼𝘀𝘁. They still benefit from distributed workflows like offline work and delayed pushes, but the center of gravity is the server. 𝗗𝗶𝘀𝘁𝗿𝗶𝗯𝘂𝘁𝗲𝗱 𝗯𝘆 𝗱𝗲𝘀𝗶𝗴𝗻, 𝗱𝗶𝗳𝗳𝗶𝗰𝘂𝗹𝘁 𝘁𝗼 𝗵𝗼𝘀𝘁 Because every copy of a Git repo is fundamentally the same, there is nothing inherently special about the server-side copy. At first glance, that makes hosting sound easy. 𝗧𝗵𝗲 𝘀𝗶𝗺𝗽𝗹𝗲 𝗚𝗶𝘁 𝘀𝗲𝗿𝘃𝗲𝗿 𝗺𝗼𝗱𝗲𝗹 𝗵𝗶𝘁𝘀 𝗮 𝗰𝗲𝗶𝗹𝗶𝗻𝗴 Putting an HTTP server in front of a repository on disk works, but only up to a point. At scale, reliability and performance become much harder. 𝗣𝗮𝗰𝗸𝗳𝗶𝗹𝗲𝘀 𝗮𝗿𝗲 𝘁𝗵𝗲 𝗰𝗼𝗿𝗲 𝗰𝗼𝗻𝘀𝘁𝗿𝗮𝗶𝗻𝘁 Git stores code and metadata in 𝗽𝗮𝗰𝗸𝗳𝗶𝗹𝗲𝘀. They are great for local use, but much less ideal when operating Git as infrastructure for large numbers of users and repositories. 𝗚𝗶𝘁 𝗻𝗲𝘁𝘄𝗼𝗿𝗸𝗶𝗻𝗴 𝗱𝗲𝗽𝗲𝗻𝗱𝘀 𝗼𝗻 𝗽𝗮𝗰𝗸𝗳𝗶𝗹𝗲𝘀 𝘁𝗼𝗼 Pushes and fetches move data as packfiles. That means packfiles are not just a storage detail. They are deeply tied to how Git communicates over the network. 𝗦𝗲𝗿𝘃𝗲𝗿 𝗶𝗻𝘁𝗲𝗿𝗻𝗮𝗹𝘀 𝗮𝗿𝗲 𝘀𝘁𝗶𝗹𝗹 𝗼𝗽𝗲𝗻 𝘁𝗼 𝗿𝗲𝗱𝗲𝘀𝗶𝗴𝗻 The Git client and protocol create constraints, but inside the server, operators have room to rethink the architecture. As long as the server can receive and send packfiles, the backend does not have to be built around them. 𝗣𝗮𝗰𝗸𝗳𝗶𝗹𝗲-𝗳𝗶𝗿𝘀𝘁 𝗶𝗻𝗳𝗿𝗮𝘀𝘁𝗿𝘂𝗰𝘁𝘂𝗿𝗲 𝗹𝗶𝗺𝗶𝘁𝘀 𝗮𝘃𝗮𝗶𝗹𝗮𝗯𝗶𝗹𝗶𝘁𝘆 𝗮𝗻𝗱 𝘀𝗰𝗮𝗹𝗲 Over time, teams hosting Git at scale found that a packfile-centric design creates major bottlenecks. Large binary files living on filesystems are not a strong foundation for modern reliability goals. 𝗪𝗵𝘆 𝗖𝘂𝗿𝘀𝗼𝗿 𝗯𝘂𝗶𝗹𝘁 𝗢𝗿𝗶𝗴𝗶𝗻 𝗹𝗶𝗸𝗲 𝗮 𝗱𝗮𝘁𝗮𝗯𝗮𝘀𝗲 That history is what pushed Cursor to rethink Git storage. Instead of treating Git hosting like simple file serving, they are treating 𝗢𝗿𝗶𝗴𝗶𝗻 more like a 𝗱𝗮𝘁𝗮𝗯𝗮𝘀𝗲 𝘀𝘆𝘀𝘁𝗲𝗺 built for scale. #Git #DeveloperInfrastructure #Engineering #DevTools #Databases #Scalability #Reliability #SoftwareEngineering

Over the weekend I finished off deploying Uptime Kuma to my EKS cluster. Quick run through of what's in it. Uptime Kuma, custom Docker image, multi-stage build, non-root user. 𝗜𝗻𝗳𝗿𝗮: - EKS cluster, Terraform managed end to end - Everything in private subnets except the NLB, NAT Gateway for egress - 4 separate IRSA roles (LB controller, external-dns, cert-manager, GitHub Actions), nothing sharing permissions. 𝗗𝗲𝗽𝗹𝗼𝘆𝗺𝗲𝗻𝘁: - ArgoCD. One root Application watches a folder in my repo, everything else gets spun up from files in there - ArgoCD manages its own Helm release too. Terraform bootstraps it once. 𝗡𝗲𝘁𝘄𝗼𝗿𝗸𝗶𝗻𝗴: - ingress-nginx routes traffic inside the cluster - AWS Load Balancer Controller provisions the NLB, hands traffic to nginx - TLS terminates at nginx, certs auto-renew via cert-manager 𝗜𝘀𝘀𝘂𝗲𝘀 𝗜 𝗵𝗶𝘁 𝗮𝗻𝗱 𝗵𝗼𝘄 𝗜 𝗳𝗶𝘅𝗲𝗱 𝘁𝗵𝗲𝗺: - Prometheus has no persistent storage by default, restart the pod and the history's gone.  Fix: adding a PVC so metrics survive restarts of pods - Destroy pipeline was orphaning NLBs. The LB Controller creates real load balancers Terraform doesn't know about, so destroying the cluster first left them sitting there costing money.  Fix: kill the ArgoCD apps with ingresses first so the LB Controller cleans up its own NLBs, then run terraform destroy. - ArgoCD manages its own upgrade, which raises the question of what happens if the upgrade breaks ArgoCD itself.  Fix: manual break plan, helm rollback outside of Git. Repo below to check it out:

A worldwide GitHub outage raised error rates across APIs, pull requests and Copilot, interrupting development workflows. 💻 #GitHubOutage #GitHubAPI #PullRequests #GitHubCopilot https://lnkd.in/eXg4sqFP