Out of space error while pulling docker layers since yesterday

[teiram]

Why are you starting this discussion?

Question

What GitHub Actions topic or product is this about?

Misc

Discussion Details

Hi there,
since two days ago my action based on a docker image started failing consistently with the following error:

docker: failed to register layer: write /opt/intelFPGA/quartus/linux64/libsys_proj.so: no space left on device

I didn't change the docker images in use in the last days, and it was working fine everyday. It happens when my action invokes this:

docker run --rm -v /home/runner/work/calypso-binaries/calypso-binaries:/build ghcr.io/teiram/calypso-quartus-docker:0.3 /usr/local/bin/build-images

This is the whole action script in case it can help:

jobs:
  synthesis:
    permissions:
      contents:
        write
    runs-on: ubuntu-22.04
    env:
      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
    steps:
      - name: Repository checkout
        uses: actions/checkout@v4
      - name: Core generation
        run: docker run --rm -v ${{ github.workspace }}:/build ghcr.io/teiram/calypso-quartus-docker:0.3 /usr/local/bin/build-images

Any hint about what might have changed? Or how can I avoid that error?

Cheers

[Rohankaf]

hey @teiram 👋🏻
It’s caused by GitHub Actions disk space limits. Ubuntu runners only have ~14 GB of free space, and Quartus-based Docker images are huge. GitHub recently reduced available space, so your image now overflows the runner.
To fix it, try one of these:
Cleanup space before running Docker:

sudo rm -rf /usr/local/lib/android /opt/hostedtoolcache
docker system prune -af

Use ubuntu-latest (sometimes has slightly more space).
Slim down your Docker image or split the build steps.
Use a self-hosted runner if you need more disk space.

[Owensitoow]

Posibles soluciones:

Liberar espacio manualmente: Antes de ejecutar tu contenedor, puedes intentar limpiar componentes no esenciales:

bash
docker system prune -a -f
sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/hostedtoolcache
Optimizar la imagen Docker: Considera reducir el tamaño de tu imagen eliminando archivos temporales, usando etapas multi-stage o eliminando dependencias no esenciales.

Alternativas de ejecución: Si el problema persiste, evalúa usar runners auto-hospedados con mayor capacidad de almacenamiento o dividir el proceso en jobs separados.

El error no está relacionado con cambios en tu código sino con ajustes recientes en la infraestructura de GitHub Actions que han reducido el margen de espacio disponible.

[sudostealth]

You’re using a GitHub Action that runs a Docker image, and it suddenly started failing with a ‘no space left on device’ error, even though nothing changed in your Docker setup. It happens when the action runs the docker run command. This likely means the GitHub runner is running out of storage. You may need to clean up unused Docker layers, reduce image size, or switch to a larger runner. Something on GitHub’s side may have changed, so checking storage usage or opening a support ticket could help too.

[sudostealth]

Simple solutions:

1. Clean Docker space before running your container
   docker system prune -af
2. Add this as a step before running your Docker command.
3. Remove unnecessary files in the workspace
   Sometimes the repository or build artifacts fill up the limited runner space.
4. Use a smaller Docker image
   If possible, slim down the image or split the build into smaller steps.
5. Use a larger runner (GitHub-hosted “runner with more disk” or self-hosted runner).
6. Check if GitHub made changes
   Sometimes GitHub updates runner images, which can reduce available disk space. Checking GitHub status or opening a support issue may help.

[iyawnnn]

This error usually happens because the GitHub Actions runner ran out of disk space, not because your Docker image changed. GitHub recently made some backend changes that reduced available free space on the default Ubuntu runners, and several workflows that use large Docker images have started failing with the same no space left on device error.

Your image (calypso-quartus-docker) is large, and Quartus itself requires a lot of disk space, so the runner hits the 14-GB space limit and fails when extracting layers.

How to fix or work around it:

1. Clean up unused Docker data before running your container
   Add this step before running your image:

- name: Free up disk space
  run: |
    docker system prune -af
    sudo rm -rf /usr/share/dotnet
    sudo rm -rf /opt/ghc
    sudo rm -rf /usr/local/lib/android

This usually frees 10–20 GB.

2. Use a larger runner (GitHub-hosted, paid)
   If disk usage is consistently high, you may need a runner with more than the default 14 GB of space:

• ubuntu-latest with larger runners (requires paid plan)
• or a self-hosted runner where you control disk size

3. Use actions/cache or split the workflow
   Reducing how much your script rebuilds each run can help avoid full disk usage.

4. Check your image size
   If possible, reduce layers or remove unnecessary tools from the Docker image. Even small optimizations help.

Nothing you changed caused the issue — it’s happening because your build now exceeds what GitHub-hosted runners can store.

[teiram]

Thanks for all your responses.
I was able to make it work again by following your recomendations, basically I added:

      - name: Reduce docker footprint
        run: sudo rm -rf /usr/share/dotnet /usr/local/lib/android /opt/hostedtoolcache
      - name: Reduce further
        run: docker system prune -a -f

Before my build actions, and that did the trick.
I don't use ubuntu:latest because at some point when latest moved away from 22-04 I started getting random failures in the build process. Moving back to 22-04 fixed that issue for me.