Untrusted Weights and Deserialisation Risk
Why loading a model file can execute code, what safetensors changed, and the checks that belong in any pipeline that downloads weights from a public hub.
Downloading a model from a public hub and calling a load function is, in the general case, running someone else's code with your process's privileges. This is not a subtle vulnerability class; it is the documented behaviour of the serialisation format that dominated the ecosystem for years, and it is the most direct supply chain risk in machine learning.
Why pickle executes
PyTorch's default checkpoint format wraps Python's pickle, and pickle is not a data format. It is a small stack-based virtual machine whose opcodes include REDUCE, which calls a callable with arguments. Deserialising a pickle therefore runs whatever the serialiser encoded, and a crafted file can invoke os.system or any importable function during load.
There is no way to safely unpickle untrusted data, which the Python documentation states plainly. torch.load with weights_only=True restricts the allowed opcodes to a safe subset and is the correct setting; it became the default in PyTorch 2.6, and any pipeline pinned to an older version or overriding it is still exposed.
Safetensors
The safetensors format exists to remove the problem rather than mitigate it. A file is a JSON header describing tensor names, dtypes, shapes and byte offsets, followed by a contiguous block of raw tensor data. There is no code path that executes anything, because there is nothing in the format that could encode a call.
Two additional properties follow. Loading is zero-copy through memory mapping, which makes it substantially faster for large models than deserialising a pickle. And the header is parseable without reading the data, so a file's contents can be inspected before committing to load it.
The format is now the default for published weights on the major hubs, and preferring it is the single highest-value change a team can make to its model loading path.
What else arrives with a model
Weights are not the only artefact. A repository typically also contains configuration files, a tokeniser, and, for architectures not in the library, Python modules loaded through a remote-code mechanism. That mechanism executes arbitrary code by design, and enabling it is a decision to trust the repository as a code dependency rather than as a data dependency.
Tokeniser files can also carry risk, since some formats support custom normalisers and pre-tokenisers defined in ways that are not purely declarative.
When it breaks
Format alone is not provenance. A safetensors file cannot execute code and can still contain a backdoored model. Removing the deserialisation risk leaves the training-time risk entirely intact, and conflating the two produces a false sense of coverage.
Scanning is incomplete. Hub-side scanners detect known malicious pickle patterns and are a filter rather than a guarantee, since obfuscation and novel gadget chains evade signature matching.
Pinning by name is not pinning. A model reference by repository name resolves to whatever is current, so a compromised or updated repository silently changes what loads. Pinning to a commit revision, and verifying a hash, is what makes the dependency reproducible and tamper-evident.
The risk moves to the runtime. Loading a model into a serving process gives it that process's network access and credentials. Running untrusted model loading in an isolated environment, with no credentials and restricted egress, bounds the damage from the case where the earlier checks failed.
14 flashcards for this concept
Click a card to reveal the answer.