Grab's engineering blog described moving its macOS continuous-integration fleet off a US cloud vendor onto a colocation cluster in Southeast Asia in November 2025 - bare metal with no virtualisation - with a projected saving of about 2.4 million dollars over three years. What made owning the hardware the cheaper answer here, and where would copying it be a mistake?
Show the full answer Hide the answer
The situation they were in
A mobile build fleet is an unusual workload: macOS builds must run on Mac hardware, so the rental market is thin and priced per machine-month rather than per second. Grab rented that capacity from a US vendor while its Git servers sat in Southeast Asia, which put a trans-Pacific hop in the middle of every checkout.
What they chose
A colocation cluster near their own Git infrastructure: four 42U racks holding more than 200 Mac minis, two power sources, mostly active/active networking, and bare metal rather than virtualisation - the write-up's stated reason being that virtualisation's benefits did not justify its cost and downside.
Why it fit their constraints
Three facts line up, and all three have to be true for the answer to come out this way.
- Utilisation. CI on a large mobile codebase is close to always-busy. Per-second elasticity buys nothing when the machines are hot all day, and per-invocation or per-hour pricing is a premium you pay for idle time you do not have.
- A thin supplier market. Their own total-cost comparison found colocation more cost-effective at their scale, and the gap widened against the vendors offering macOS compute in the region.
- Locality. Moving the builders next to the Git servers removed a round trip from every job. Reported gains were roughly 20-40% depending on use case: p50 pipeline time for the Grab app fell from 43 to 33 minutes and p50 app build time from 10.7 to 6.45 minutes.
What it cost them
A second operational surface: racks, spares, hands in a facility, power and network redundancy to design, and the patience to run the new cluster alongside the old one for a long period, comparing failure rates and job performance before ending the vendor contracts. The 2.4 million dollars is their own projection from their own cost model, not an audited result - treat it as the shape of the saving, not as a quote.
Where copying it would be a mistake - and when it is the wrong answer
The saving is a function of fleet size, workload flatness and whether you already have people who can run physical infrastructure. At 20 build machines the same arithmetic collapses: the fixed cost of racks, redundancy and a hardware on-call rota is paid whole, and it is larger than the rental bill it replaces.
Four conditions to test before anyone writes the proposal:
- Is the workload flat? If demand swings 10x daily, elasticity is worth the premium and owning peak capacity is the expensive choice.
- Can you amortise the fixed cost? Below roughly 100 machines there is not enough bill to spread the facility and staffing across.
- Is the supplier market genuinely thin? For ordinary x86 compute it is not, and the cloud premium over colocation is far smaller than it is for a licence-restricted platform.
- Can you afford a parallel run? A cut-over with no overlap period turns a cost decision into an availability incident.
Where none of those hold, the boring answer - keep renting and spend the engineering time on the product - is also the cheaper one.