Product

Your Build Dependencies Now Come From Our Datacenter

By GetMac Team
5 min read
Your Build Dependencies Now Come From Our Datacenter

Every CI job starts by downloading things it already downloaded yesterday. brew install, npm ci, pod install, bundle install, pip install — before a single line of your code compiles, a runner has pulled hundreds of megabytes from package registries scattered across the internet.

We've moved that traffic inside our datacenter. Runners now fetch those packages from a cache sitting on the same network, a few switch hops away.

Why Consistency Beats Peak Speed

The obvious pitch would be "downloads are faster now". That's true, but it undersells the real problem and oversells the fix.

When we measured the same 77 MB package from the public internet, we got 6.4 MB/s on one attempt and 79 MB/s on another — same file, same runner, minutes apart. Public CDNs are fast when the file you want is warm at an edge near you, and slow when it isn't. You never find out which you're getting until the job runs.

The local cache served that same file at a steady ~110 MB/s, every single time, with variance measured in tenths of a percent. That number is bounded by the runner's own network link, not by the cache.

If your build times swing by minutes between runs with no change to your code, this is a large part of why. A predictable ten-minute build is worth more than a build that's sometimes six and sometimes eighteen.

Small Files Are Where It Really Shows

Bulk download speed isn't what dominates most dependency installs. Resolving a lockfile means hundreds of small metadata requests, and each one pays a full round trip to a server somewhere far away.

Fetching metadata for twenty popular npm packages took 121–157 ms per package from the public registry, and 29 ms per package from the local cache — roughly four to five times faster, and that gap compounds across every package in your dependency tree.

What's Cached

  • Homebrew — bottles and the formula index
  • npm — package tarballs and metadata
  • CocoaPods — the CDN trunk and podspecs
  • RubyGems.gem files and the dependency API
  • PyPI — wheels and the simple index
  • Debian — packages for our Linux runners

We only cache artifacts that are public and immutable — a published version of a package can never legitimately change its contents. Anything private, anything authenticated, and anything that moves stays well clear of the cache and goes straight to the origin.

Nothing To Change

There is no setting to enable, no action to add, and no change to your workflow file. Jobs use it automatically.

If the cache is ever unavailable, runners fall back to the public internet exactly as before. It makes your builds faster and steadier; it is never a new thing that can stop them.

There's a side benefit too: when a public registry has an outage — and they all do — recently used packages keep serving from our copy.

What's Next

Package downloads are the first step. The bigger prize is build artifacts: derived data, compiled dependencies, and the caches your workflow saves between runs. Those are usually far larger than your dependencies, and today they travel much further than they need to. We're working on that next.

Get Started

Nothing to do — it's rolling out across the fleet now and your jobs will pick it up automatically.

As always, we'd love to hear your feedback — join our Discord community to share thoughts or tell us which registries you'd like to see cached next.

— The GetMac Team

#product#performance#ci-cd#caching#runners
Your Build Dependencies Now Come From Our Datacenter | GetMac