CVE-2025-33214DeserializationMachine LearningRCE

CVE-2025-33214: when an ML workflow becomes code on the server

NVTabular loaded workflows with cloudpickle without restricting reconstructed classes. If an untrusted artifact reaches the pipeline, the file executes with the identity of the ML process.

By Equipe VorvexPublished on August 26, 20264 min read

Model and workflow files often move through a company as if they were data: they enter through a bucket, pass through a registry, and end up in a job with access to GPUs, datasets, and cloud credentials. CVE-2025-33214 shows why that assumption is wrong. In NVIDIA NVTabular, loading a serialized workflow could execute code in the same process performing the load.

The flaw does not require breaking CUDA or exploiting the GPU. The boundary fails earlier, when a workflow.pkl file is deserialized with cloudpickle. If someone can replace, publish, or point to an artifact that another person or automation will load, its content stops being mere configuration and gains the authority of the identity running the pipeline.

Technical attack path diagram showing an untrusted machine learning workflow artifact moving from a shared model registry into an NVTabular loading process and then reaching cloud credentials, datasets, and compute infrastructure — The artifact crosses a trust boundary. The impact follows the permissions of the job that calls Workflow.load.

What CVE-2025-33214 confirms

NVIDIA assigned a CVSS 3.1 score of 8.8, high severity, with vector AV:N/AC:L/PR:N/UI:R/S:U/C:H/I:H/A:H. In practical terms, the input can arrive over a network, exploitation has low complexity and requires no prior privileges, but it depends on victim action: the malicious workflow must be loaded. Potential impact includes code execution, denial of service, information disclosure, and data tampering.

Affected product: NVIDIA NVTabular on Linux
Vulnerable component: Workflow loading
Version tested by GitHub Security Lab: v23.08.00
Official scope: all versions that do not include commit 5dd11f4
Classified cause: CWE-502, deserialization of untrusted data

The line where data becomes execution

Workflow.load resolves the path with fsspec and opens workflow.pkl. In the implementation analyzed by GitHub Security Lab, the object was passed directly to cloudpickle.load. Pickle is not a passive format: during reconstruction, it can import classes and call functions. Checking the extension or name, or merely calculating a hash without a trusted reference, does not change the nature of the file.

python

# Vulnerable implementation documented by GitHub Security Lab
workflow = cloudpickle.load(
    fs.open(fs.sep.join([path, "workflow.pkl"]), "rb")
)

# Fix in commit 5dd11f4
from nvtabular.tools.serialization import load as load_safe

workflow = load_safe(
    fs.open(fs.sep.join([path, "workflow.pkl"]), "rb")
)

The central change replaces unrestricted loading with an Unpickler that only accepts imports registered on an allowlist.

The fix adds a custom Unpickler and overrides find_class. When the pickle tries to reconstruct a class or function outside the allowlist, loading ends with an error. It is a small difference at the call site, but it changes the security question from 'what does this file want to import?' to 'was this import explicitly authorized?'.

Split technical diagram comparing unrestricted pickle deserialization that can invoke arbitrary functions with allowlisted deserialization that blocks an unapproved import at a security boundary — Before the fix, the artifact chose what the process imported. After the fix, imports outside the allowlist are blocked during deserialization.

The real attack vector is the artifact chain

CVSS records user interaction because a person or job must load the workflow. That does not limit the scenario to manual phishing. In ML environments, the action may be a notebook opened by a researcher, an automatic promotion between buckets, a validation job, or a service that receives a model path. The relevant test is to discover who can write or point to the artifact and which identity consumes it.

A registry or bucket allows publication by identities that should not influence production
The pipeline promotes artifacts without a signature, provenance record, or pinned hash
A shared notebook loads workflows from directories writable by other users
An inference or data preparation job runs with broader credentials than necessary
Logs record job execution but not the digest and exact origin of the loaded artifact

How to test without executing a malicious pickle

During a pentest, we do not need to place an execution payload in a shared environment to prove the risk. We first confirm the code revision and load point through static inspection. Then we map registry write and read permissions, the consumer identity, and promotion between environments. A safe proof can use an invalid, inert artifact to demonstrate that an untrusted origin reaches the loader, without including an opcode capable of calling the operating system.

bash

# Run in the NVTabular checkout used to build the image.
# These commands only read code and Git history.

rg -n 'cloudpickle\.load|load_safe' nvtabular/workflow/workflow.py

if git merge-base --is-ancestor 5dd11f4 HEAD; then
  echo 'fix 5dd11f4 is present in history'
else
  echo 'fix 5dd11f4 was not confirmed in this checkout'
fi

git rev-parse HEAD

Local, non-destructive triage. In a packaged image without Git metadata, compare the loader source and image digest with the artifact approved by the team.

Finding load_safe is useful evidence, but it does not complete the analysis. The production binary or container must match the reviewed code, and the list of allowed classes also deserves inspection. In deserialization, every newly authorized class expands the surface available during reconstruction.

Remediation and containment

The official guidance is to update to a branch that includes commit 5dd11f4 or later. NVIDIA does not identify a fixed numbered release in the bulletin, so checking only the package version may be insufficient. Teams need to validate the included commit, the approved container image, or equivalent confirmation from the maintainer of the distribution in use.

Update NVTabular to code containing 5dd11f4 or later and rebuild derived images
Invalidate workflows of unknown origin and republish artifacts from a trusted pipeline
Require an immutable digest, signature, and provenance before production promotion
Separate artifact publishers from the identities that authorize execution in privileged environments
Reduce credentials, mounts, and network access for the process that loads workflows
Record digest, origin, publisher identity, and consumer identity for every load

The lesson for every ML pipeline

CVE-2025-33214 is specific to NVTabular, but the pattern is broader: serialized models, checkpoints, and workflows can contain behavior, not only weights or configuration. If the ingestion process trusts the file extension and the execution process trusts the loader, the entire chain delegates execution to whoever controls the artifact.

A pentest of this environment must follow the complete object path, from publication to runtime. The decisive question is not only 'is the package patched?', but 'who can make this privileged process load what?'. The connection between storage, identity, automation, and runtime is where a deserialization flaw becomes infrastructure compromise.

← Back to blog