Penetration Testing AI Products: A Due-Diligence Lens

As part of technical due diligence, one of the first questions I ask about an acquisition target is simple: if this company ships AI, can it be broken? Security posture is a direct input to valuation and integration risk, and AI products introduce failure modes that traditional application security checklists miss. Below is the lens I use when assessing an AI product's robustness, the same lens I applied running penetration tests against production AI systems.

Why security is a due-diligence concern, not just an engineering one

A vulnerable AI product is a liability that surfaces after close: data-breach exposure, regulatory risk, and remediation cost that eats into the value-creation thesis. Assessing it early means the risk is priced in, not discovered later.

Key Threats: OWASP Top 10, applied to AI systems

The OWASP Top 10 enumerates the most critical risks for web applications, and most of them apply directly to AI products. A few that consistently matter in diligence:

1. Injection Attacks

Malicious input can manipulate AI models or the databases behind them. A common example is SQL injection in an API endpoint:

import sqlite3

def get_user_data(user_id):
    conn = sqlite3.connect("database.db")
    cursor = conn.cursor()
    query = f"SELECT * FROM users WHERE id = {user_id}"  # Vulnerable to SQL injection
    cursor.execute(query)
    return cursor.fetchall()

Fix: Use parameterized queries.

query = "SELECT * FROM users WHERE id = ?"
cursor.execute(query, (user_id,))

2. Insecure Model APIs

Attackers can exploit model endpoints with adversarial inputs or malformed requests:

from flask import Flask, request, jsonify
import torch
import model

app = Flask(__name__)

@app.route("/predict", methods=["POST"])
def predict():
    data = request.json["input"]
    result = model.predict(torch.tensor(data))  # No validation
    return jsonify({"prediction": result})

Fix: Enforce strict input validation and authentication.

if not isinstance(data, list) or any(not isinstance(i, (int, float)) for i in data):
    return jsonify({"error": "Invalid input"}), 400

3. Data Leakage

Models trained on sensitive data can inadvertently expose private information. Attackers use model inversion to reconstruct training data:

# An overfitted model leaking user data
print(model.predict([user_sensitive_input]))

Fix: Apply differential-privacy techniques and avoid overfitting to specific data patterns.

What this means for a deal

When these controls are absent, the finding is not "the code needs work"; it is a quantified risk with a remediation cost and timeline that belongs in the diligence report. Addressing injection attacks, insecure model APIs, and data leakage before or immediately after close is part of protecting the investment and clearing the path for the AI value-creation plan.