LLM (Large Language Model) model security is the practice of inspecting pre-trained AI model files for embedded malicious code before they load into an environment. Model files in formats like pickle, PyTorch, TensorFlow, and Keras can execute code the moment they're loaded, making them a supply-chain risk equivalent to running unvetted software.
TL;DR / Key Takeaways
- Pre-trained models downloaded from Hugging Face, Kaggle, or PyPI are executable code, not inert data
- According to JFrog's Software Supply Chain State of the Union 2026, 53% of organizations pull models directly from public registries, where researchers identified roughly 495 malicious models capable of credential theft and full system compromise
- Pickle-based formats (.pt, .pth, .bin, .pkl) execute arbitrary functions via the REDUCE instruction during loading, the same structural flaw carried by TensorFlow graphs and Keras Lambda layers
- Safetensors removes the execution engine entirely, but hundreds of thousands of legacy-format models remain in circulation, and a safe file in a repo doesn't guarantee the rest of that repo is safe
- MetaDefender Aether™ inspects model files across five analysis layers, catching threats like stacked pickle payloads that score clean on traditional scanners
Pre-Trained AI Models Are Executable Code Crossing a Trust Boundary
Building a large language model from scratch requires data, compute, and time most organizations don't have. Teams instead download pre-trained models from public hubs like Hugging Face, Kaggle, and PyPI. That reliance has created a supply chain attack vector that many security programs still overlook.
Organizations already vet the open-source libraries and container images they pull into their environments. Few extend that same scrutiny to the AI models their teams download, even though a model file is not simply a spreadsheet of numbers. Depending on its format, a model file can be a program that runs the moment it loads.
Hugging Face alone hosts well over a million models, most of which have never undergone a safety review. According to JFrog's Software Supply Chain State of the Union 2026, 53% of organizations now pull models directly from public registries, and researchers identified roughly 495 malicious models in those registries capable of credential theft, code execution, and full system compromise. An LLM model is executable software crossing a trust boundary.
How Model-Borne Threats Evolved From Theory to Evasion
The threat developed in stages: academic warnings, proofs of concept, real-world supply chain abuse, and evasion techniques designed to defeat scanners.
The Evolution of Model-Borne Threats, 2018–2026
Phase | Timeframe | What Changed | Representative Evidence |
Theory | 2018 | Academics warn that reused, pre-trained models from untrusted sources can be manipulated | Model-Reuse Attacks on Deep Learning Systems |
Proof of concept | 2023–2024 | Real malicious models surface; pickle-based formats shown to carry live payloads | star23/baller13, a model carrying a reverse shell, appeared on Hugging Face |
In the wild | 2024–2025 | Volume increases, with supply chain delivery through package registries | Alibaba-branded PyPI packages mark the arrival of LLM supply chain attacks |
Advanced evasion | 2024–2025 | Layered, stacked payloads engineered to defeat quick inspection | A stacked pickle proof of concept scored 0/70 on VirusTotal |
A safer format | 2023–2026 | A data-only format that cannot carry code is introduced, but legacy models remain widely used | Safetensors becomes the default for new model releases |
Pickle and PyTorch Files Execute Code the Moment They Load
Pickle is Python's native serialization format and was the standard way to save models to disk for years. PyTorch built its .pt, .pth, and .bin files on top of it, and many models were published in those formats. Safer options exist today, removed as abstract business language. A large share of the models still circulating on public hubs remain pickle-based, and teams load them every day.
Pickle stores more than data. It is an instruction stream that a virtual machine executes while loading the file and reconstructing Python objects. The stream can call arbitrary functions, so loading a pickle-based model can also run code.
Attackers exploit the pickle REDUCE instruction, which tells the loader to call a specified function with attacker-chosen arguments during unpickling. It can launch a reverse shell, drop a second-stage payload, or overwrite an SSH key while the model still loads and behaves like a legitimate file.
TensorFlow and Keras expose similar risks. Malicious TensorFlow graphs can smuggle file-write or network operators into the computation graph, while Keras Lambda layers can carry marshaled Python bytecode that deserializes on load. Each format allows a file to carry executable behavior. This risk is structural, so patching a single bug cannot remove it. That limitation forced the industry to find a safer format.
Real-World Incidents Turned the Theory into a Supply Chain Threat
Warnings about model-reuse attacks first appeared in 2018. Researchers later demonstrated that the risk worked in practice. In January 2024, a model named star23/baller13 appeared on Hugging Face with a reverse shell inside a PyTorch file. It could deliver remote access while still passing as a valid model.
By May 2025, the threat moved into classic supply chain territory. Attackers published PyPI packages named aliyun-ai-labs-snippets-sdk, ai-labs-snippets-sdk, and aliyun-ai-labs-sdk that impersonated an Alibaba AI toolkit and quietly loaded a malicious model during use. The packages stayed live for less than 24 hours yet were downloaded roughly 1,600 times; a reminder that a brief exposure window is more than enough when automated dependency resolution does the downloading.
Stacked Pickle Payloads Defeat Traditional Scanners
By 2024, attackers were targeting the scanners built to catch malicious models. The clearest example is the "Stacked Pickle" technique, which nests multiple pickle objects and spreads malicious instructions across the layers, then wraps them in compression and encoding.
Each layer appears harmless when examined alone, so scanners and manual reviews that stop at the surface find nothing. When the model loads, the layers unpack in sequence and reassemble the payload. A proof-of-concept sample built with this technique scored zero detections across every engine on VirusTotal.
A scanner that checks only the visible layer can miss a payload hidden deeper in the file. That is how the sample passed every engine.
Safetensors Removes the Execution Engine Entirely
Safetensors stores model weights without an execution engine. A file contains a compact metadata header describing each tensor's shape, data type, and byte offset, followed by the raw tensor bytes. It has no instruction stream, virtual machine, or equivalent to pickle's REDUCE, so the loader cannot be instructed to call a function.
A Safetensors file is inert data that the loader reads. Even a maliciously crafted file cannot execute code on load because the format provides no path for that code to run.
Why Safetensors Was Built, and Why It Spread
The problem with legacy formats was never the weights themselves. It was that the file could also carry executable instructions. Hugging Face, working with EleutherAI and Stability AI, built Safetensors as a purpose-made replacement that strips out that capability while preserving what teams need from a weights file.
Safetensors is also practical. It loads quickly through memory-mapped, zero-copy access, works across PyTorch, TensorFlow, JAX, and other frameworks, and has become the default for most new model releases. Those advantages helped it spread without a mandate.
An Independent Audit Backs the Safety Claim
The maintainers are not the only source behind the safety claim. The library is written in Rust, whose compiler prevents whole classes of parsing bugs. In 2023, Hugging Face, EleutherAI, and Stability AI jointly commissioned an independent audit of the format from Trail of Bits.
The audit found no critical flaw that could lead to arbitrary code execution. It identified a handful of specification imprecisions and a missing validation that had allowed polyglot files. The maintainers fixed and published all of them, then made Safetensors the default.
The record since then supports the audit's finding. Legacy formats have produced live incidents since 2024, while no code-execution attack against the Safetensors format itself has been demonstrated over the same period. Safetensors gives attackers no execution mechanism for the class of attack that makes pickle-based files dangerous. The remaining risk comes from legacy formats that are still widely used.
The Model Supply Chain Still Needs an Inspection Gate
The risk comes from whether a model file format can carry code alongside its weights. Pickle, PyTorch, TensorFlow, and Keras formats can; Safetensors cannot by design. Defenders need to treat executable model formats as a risk, inspect every model entering the environment, and steer teams toward safe formats wherever possible.
Millions of legacy-format models remain on public hubs, and teams still download them. A repository can also ship a pickle-based file alongside a safe one, so the presence of Safetensors weights does not make the entire repository safe. The model supply chain will need an inspection gate until legacy formats are retired.
In practice, that becomes a short, low-false-positive checklist:
- Prefer Safetensors, and treat legacy formats as requiring inspection before use. Any .pt, .pth, .bin, .pkl, .pb, or .h5 model from an unvetted publisher should be scanned before it's ever loaded
- Treat loading a legacy-format model as an execution event. A model load that spawns a shell, opens an outbound connection from a Python interpreter, or writes to a sensitive path is a behavioral signal that belongs in host telemetry the same way any code execution would
- Scan the whole artifact, not just the weights file. The unit of analysis is the model package as delivered, since a repository's format is a claim, not a guarantee, until something verifies it
How MetaDefender Aether's Five-Layer Pipeline Inspects LLM Model Files
MetaDefender Aether™, OPSWAT's unified zero-day detection solution, delivers 99.9% zero-day detection efficacy through a five-layer inspection pipeline. Each submission begins with file type analysis, which identifies the file instead of trusting its extension. Each analysis layer can resolve the verdict independently. Once a layer reaches a definitive answer, analysis stops, so most files avoid the deeper, more expensive stages.
- Threat Reputation. The file's hash and related indicators are checked against OPSWAT's global threat intelligence, drawing on more than 50 billion indicators. A hash already tied to a known malicious upload is blocked in under a second, and a known-good file clears just as quickly, so only unknown files move on.
- Static Analysis. Before anything runs, the file is evaluated by Predictive Alin AI alongside static analysis and antivirus engine scanning. Predictive Alin AI delivers machine-learning verdicts in milliseconds without sandbox detonation, trained on curated, privacy-safe enterprise datasets and continuously improved through a zero-day retraining loop fed by MetaDefender Aether's confirmed detections. For model files, this is where a suspicious pickle with recognizable payload opcodes gets caught before any emulation time is spent on it.
- Dynamic Analysis. Instruction-level emulation addresses the Stacked Pickle problem by revealing a file's behavior without booting a full virtual machine for every sample. It defeats the anti-VM checks and timing delays that bypass legacy sandboxes, completing analysis in seconds rather than minutes at up to 20x the speed and 100x the volume of traditional sandboxing tools.
- Threat Scoring. This layer combines the functions invoked, connections attempted, reputation signals, and other findings from the first three layers. It draws on more than 900 behavioral indicators to produce one actionable score and a similarity fingerprint for the final stage.
- Threat Hunting. Machine learning similarity search correlates the fingerprint with OPSWAT's threat intelligence database and MITRE-mapped behaviors to identify variants and campaigns. A repacked or renamed malicious model may have a new hash, but this layer can still detect its similarity to a known malicious ancestor.
The full chain runs automatically. It returns one consolidated verdict and an evidence report that connects each indicator to the behavior that produced it.
MetaDefender Aether Inspects Model Files at Every Entry Point
Models arrive through web downloads, email, file transfer platforms, and automated pipelines. An inspection gate must cover every route without forcing organizations to redesign the infrastructure that carries the traffic.
For network channels, MetaDefender Aether integrates through ICAP, the standard protocol that proxies, email gateways, and file transfer platforms already use to hand files to an inspection service. Where traffic never crosses a network device, the same analysis is reachable over a REST API instead.
- Web downloads. When a developer pulls a model from Hugging Face or Kaggle, the proxy or secure web gateway passes the download to MetaDefender Aether over ICAP, and a malicious pickle file is stopped before it ever reaches the endpoint.
- Email. Model files and AI tooling shared as attachments are extracted and analyzed through the same pipeline as any other attachment, closing the social engineering route around registry-level controls.
- File transfers. Models exchanged with partners and vendors, or moved between internal zones through managed file transfer, are scanned in transit, so the trust boundary holds even for files that never touch a public hub.
- Automated pipelines and registries. Build jobs and ML platforms that fetch models programmatically submit them through the API before anything is promoted to an internal registry, closing the exact route the Alibaba PyPI packages exploited, where dependency resolution does the downloading and no one ever sees a file to be suspicious of.
Every route uses the same inspection pipeline and verdict logic, leaving no unmonitored channel for an attacker to exploit.
Security Without Slowing Down Developers
Security teams often leave model risk unmanaged because banning public-hub downloads is impractical. MetaDefender Aether lets developers keep their existing workflows while the pipeline scans models in line at emulation speed. Blocked downloads include an evidence report explaining the verdict.
Learn how MetaDefender Aether inspects LLM model files before they cross your trust boundary.
Frequently Asked Questions
What is LLM model security?
LLM model security is the practice of inspecting pre-trained AI model files for embedded malicious code before they load into an environment. It treats models downloaded from public hubs as executable software crossing a trust boundary, not as inert data.
Why are pickle files dangerous for AI models?
A virtual machine executes a pickle file's instruction stream as the file loads. The REDUCE instruction lets an attacker call arbitrary functions during that process, so loading a pickle-based model can run attacker-controlled code without warning.
What is Safetensors, and how is it different from pickle?
Safetensors is a data-only file format for storing model weights, containing a metadata header and raw tensor bytes with no execution engine and nothing equivalent to pickle's REDUCE instruction. Where a pickle file is a program the loader runs, a Safetensors file is inert data the loader reads.
Can a Safetensors file still be malicious?
A Safetensors file cannot execute code on load because the format provides no execution path. However, a repository can include a legacy pickle-based file alongside safe Safetensors weights, so the whole repository still requires inspection.
How does MetaDefender Aether detect malicious LLM models?
MetaDefender Aether inspects model files through five layers: threat reputation, static analysis, dynamic analysis, threat scoring, and threat hunting. It stops most files before they reach the deeper, more expensive stages and can detect stacked pickle payloads that pass traditional single-layer scanners.
Inspect Every Model Before It Crosses Your Trust Boundary
Teams now rely on reused, pre-trained models, but every download from a public source raises the same question: does the file contain only data, or can it run code? MetaDefender Aether answers before the model loads, applying five layers of analysis to web downloads, email, file transfers, and automated pipelines.
