You must serve similarity search over 200 million embeddings of 1024 dimensions with an in-memory HNSW index. Which estimate and which lever are right?

The assumptions, stated

200 million vectors, 1024 dimensions, 32-bit floats as produced by most embedding models, and an HNSW graph with a connectivity parameter M of 32, which is a common default. Neighbour lists are 32-bit identifiers.

The arithmetic, step by step

Vectors:

200,000,000 × 1024 dims × 4 bytes = 819,200,000,000 bytes ≈ 819 GB

Graph: HNSW keeps up to 2M neighbours at the base layer and M at higher layers, and higher layers hold a small fraction of nodes, so per node it is roughly 2M identifiers:

2 × 32 × 4 bytes = 256 bytes per node
200,000,000 × 256 bytes ≈ 51 GB

Total: about 870 GB, of which the vectors are 94%.

The number, with its range

820 to 900 GB for fp32 at M = 32. Changing M from 32 to 16 saves about 26 GB - 3% of the total - and costs recall, which is why the third option's lever is the wrong one even though its direction sounds plausible.

Which assumption dominates the error

Dimension and element width, jointly. They are the only terms with a factor-of-four lever, and nothing else in the estimate matters by comparison. An 1536-dimension model instead of 1024 takes the figure to about 1.28 TB; int8 scalar quantisation takes the vectors from 819 GB to 205 GB; binary quantisation takes them to about 26 GB, at which point the 51 GB graph is the larger term and the earlier arithmetic inverts.

Why the other options fail

• 80 GB is roughly the fp32 figure for 20 million vectors - an order-of-magnitude slip, and the one that produces a capacity plan for a tenth of the data.
• 8 TB with the graph dominating overstates the graph by two orders of magnitude. The graph is a few hundred bytes per node, not a few kilobytes.
• 200 GB with metadata dominating happens to be close to the int8 figure, with the wrong attribution. Index metadata is negligible at any configuration.

What the number rules in or out

It rules out a single commodity machine at fp32, and it rules out tuning connectivity as the answer. It rules in three real options, in order of what to try:

1. int8 scalar quantisation with full-precision rescoring of the top candidates. Four times less memory, and recall@10 typically within about a percentage point of fp32 when the top 100 to 200 candidates are rescored against the original vectors. The decision rule: if measured recall@10 at int8 is within your tolerance, take the four times saving - this is the highest-value change available and the one most often skipped.
2. Sharding across nodes by tenant or by a random partition with scatter-gather, which is required anyway for availability and for rebuild time. 870 GB is not one machine's problem to carry.
3. Disk-resident approximate search, which trades a few milliseconds of latency for an order of magnitude less memory and is the right answer when the corpus grows faster than the budget.

When this is the wrong estimate to make

Below roughly a million vectors none of it matters: a million 768-dimension float vectors is about 3 GB, fits anywhere, and the real questions are filtering semantics and freshness. It is also the wrong frame for a managed service priced by stored vectors or capacity units - compute in the vendor's units, while keeping the same ordering of levers, since dimension and precision usually drive their pricing too.

And it rules in one question before any of them: does the application need 200 million vectors in one index? A per-tenant index of 50,000 vectors is a different and much cheaper problem, and the single global index is frequently a default rather than a requirement.