LLM API Security: Securing OpenAI, Anthropic, and Claude Integrations

Integrating LLM APIs into production applications introduces a category of security concerns that most teams underestimate. The issues go beyond "protect your API key" — they include financial exposure from unbounded usage, regulatory risk from PII leakage to third-party AI providers, prompt injection via user-supplied content, and liability from model outputs that downstream systems trust uncritically.

This guide covers the full security lifecycle of an LLM API integration: key management, rate limiting and cost controls, input sanitization, output validation, and PII handling.

API Key Management

Never Hardcode Keys

The most common and most damaging mistake is embedding API keys directly in source code or configuration files checked into version control.

# BAD — never do this
client = OpenAI(api_key="sk-proj-abc123...")

# GOOD — load from environment
import os
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

Even a private repository is not safe: ex-employees retain access, CI systems log environment variables, and secrets in git history persist after deletion.

Use a secrets manager in production:

import boto3

def get_api_key(secret_name: str) -> str:
    client = boto3.client("secretsmanager", region_name="us-east-1")
    response = client.get_secret_value(SecretId=secret_name)
    secret = json.loads(response["SecretString"])
    return secret["api_key"]

openai_key = get_api_key("prod/openai/api-key")

AWS Secrets Manager, Google Secret Manager, HashiCorp Vault, and Doppler all work well for this. The key principle: API keys should never appear in code, logs, or version control.

Scope API Keys to Minimum Permissions

OpenAI's project-scoped API keys (released in 2024) let you restrict keys to specific capabilities. Use them:

• Create separate keys per environment (dev/staging/prod)
• Create separate keys per service or microservice
• Restrict keys to only the models and endpoints each service needs
• Set spending limits per key in the provider dashboard

# In your application bootstrap, validate key scope
def validate_api_key_permissions():
    try:
        # Test with a minimal API call
        models = openai.models.list()
        allowed_models = {m.id for m in models.data}
        required_models = {"gpt-4o", "text-embedding-3-small"}
        if not required_models.issubset(allowed_models):
            raise RuntimeError(f"API key missing access to required models")
    except openai.AuthenticationError:
        raise RuntimeError("Invalid API key")

Key Rotation

Rotate LLM API keys on a schedule and immediately upon any suspected exposure:

# Automated rotation script (run monthly via cron or CI)
#!/bin/bash
NEW_KEY=$(openai-cli keys create --name "prod-$(date +%Y%m)" --format json | jq -r .key)
aws secretsmanager put-secret-value \
  --secret-id prod/openai/api-key \
  --secret-string "{\"api_key\": \"$NEW_KEY\"}"

# Verify new key works before deleting old one
python verify_api_key.py

# Delete old key only after verification
openai-cli keys delete $OLD_KEY_ID

Rate Limiting and Cost Controls

LLM APIs charge per token. Without controls, a single malicious user or a runaway loop can cost thousands of dollars in minutes. This is a form of financial denial-of-service.

Per-User Rate Limiting

Implement rate limiting at multiple granularities:

from redis import Redis
from datetime import datetime

redis_client = Redis(host="localhost", port=6379)

class LLMRateLimiter:
    def __init__(self):
        self.limits = {
            "requests_per_minute": 10,
            "tokens_per_day": 50_000,
            "requests_per_day": 200,
        }

    def check_rate_limit(self, user_id: str, estimated_tokens: int) -> bool:
        now = datetime.utcnow()
        minute_key = f"llm:rpm:{user_id}:{now.strftime('%Y%m%d%H%M')}"
        day_key = f"llm:rpd:{user_id}:{now.strftime('%Y%m%d')}"
        token_key = f"llm:tokens:{user_id}:{now.strftime('%Y%m%d')}"

        pipe = redis_client.pipeline()
        pipe.incr(minute_key)
        pipe.expire(minute_key, 60)
        pipe.incr(day_key)
        pipe.expire(day_key, 86400)
        pipe.incrby(token_key, estimated_tokens)
        pipe.expire(token_key, 86400)
        results = pipe.execute()

        rpm, _, rpd, _, tokens, _ = results

        if rpm > self.limits["requests_per_minute"]:
            raise RateLimitError("Too many requests per minute")
        if rpd > self.limits["requests_per_day"]:
            raise RateLimitError("Daily request limit reached")
        if tokens > self.limits["tokens_per_day"]:
            raise RateLimitError("Daily token limit reached")

        return True

Hard Token Limits

Always set max_tokens on every API call. Never omit it — without this parameter, the model will generate until it naturally stops, which for verbose models can be thousands of tokens:

response = openai.chat.completions.create(
    model="gpt-4o",
    messages=messages,
    max_tokens=1024,  # Always set this
    temperature=0.7,
)

Set max_tokens based on your use case, not as a safety backstop. A customer support bot should not generate 4,000-token responses.

Cost Alerting

Configure budget alerts in your provider's dashboard. Supplement with application-level tracking:

import datadog

def track_llm_cost(model: str, prompt_tokens: int, completion_tokens: int, user_id: str):
    # Approximate cost per model (update as pricing changes)
    COST_PER_1K = {
        "gpt-4o": {"input": 0.0025, "output": 0.01},
        "gpt-4o-mini": {"input": 0.00015, "output": 0.0006},
        "claude-3-5-sonnet-20241022": {"input": 0.003, "output": 0.015},
    }

    rates = COST_PER_1K.get(model, {"input": 0.01, "output": 0.03})
    cost = (prompt_tokens / 1000 * rates["input"]) + (completion_tokens / 1000 * rates["output"])

    datadog.statsd.increment("llm.requests", tags=[f"model:{model}", f"user:{user_id}"])
    datadog.statsd.gauge("llm.cost_usd", cost, tags=[f"model:{model}", f"user:{user_id}"])

    # Alert if single request is suspiciously expensive
    if cost > 0.50:
        alert(f"High-cost LLM request: ${cost:.2f} for user {user_id}")

Input Sanitization

Limit Input Length

Establish maximum input lengths and enforce them before the API call, not after:

MAX_USER_INPUT_CHARS = 4000  # roughly 1000 tokens
MAX_CONTEXT_CHARS = 12000

def validate_input(user_message: str) -> str:
    if len(user_message) > MAX_USER_INPUT_CHARS:
        raise ValueError(f"Input too long: {len(user_message)} chars (max {MAX_USER_INPUT_CHARS})")
    return user_message.strip()

Excessively long inputs can be used to dilute the system prompt's influence (reducing its proportional weight in the context window) and to inflate your token costs.

Detect Injection Attempts

import re

INJECTION_PATTERNS = [
    r"ignore (all |previous |prior |the )?(above |)instructions",
    r"disregard (the |your |all )(previous |prior |above |)instructions",
    r"(you are|act as|pretend (you are|to be))\s+(?!a helpful)",
    r"system\s*prompt",
    r"reveal\s+(your|the)\s+(system\s*)?prompt",
    r"jailbreak",
    r"DAN\s+mode",
]

def screen_for_injection(text: str) -> dict:
    matches = []
    for pattern in INJECTION_PATTERNS:
        if re.search(pattern, text, re.IGNORECASE):
            matches.append(pattern)

    return {
        "flagged": len(matches) > 0,
        "patterns_matched": matches,
        "risk_level": "high" if len(matches) > 1 else ("medium" if matches else "low"),
    }

Sanitize for Downstream Use

When model output will be inserted into HTML, SQL, shell commands, or other interpreters, treat it as untrusted user input:

import html
import bleach

def sanitize_model_output_for_html(output: str) -> str:
    # Allow limited markdown-derived HTML
    allowed_tags = ["p", "ul", "ol", "li", "strong", "em", "code", "pre"]
    allowed_attrs = {}

    # First escape, then selectively allow safe tags
    return bleach.clean(output, tags=allowed_tags, attributes=allowed_attrs)

Output Validation

Schema Validation for Structured Outputs

When you request JSON or structured data from the model, validate it against a schema before using it:

from pydantic import BaseModel, ValidationError
from typing import Optional

class ProductRecommendation(BaseModel):
    product_id: str
    reason: str
    confidence_score: float  # 0.0-1.0

    @validator("confidence_score")
    def score_must_be_valid(cls, v):
        if not 0.0 <= v <= 1.0:
            raise ValueError("confidence_score must be between 0 and 1")
        return v

def parse_recommendation(model_output: str) -> Optional[ProductRecommendation]:
    try:
        data = json.loads(model_output)
        return ProductRecommendation(**data)
    except (json.JSONDecodeError, ValidationError) as e:
        logger.warning(f"Invalid model output: {e}")
        return None

Semantic Content Checks

For applications where model output affects business logic, add semantic guardrails:

class OutputGuardrails:
    PROHIBITED_CONTENT = [
        "competitor_name",
        "internal_pricing",
        "employee_names",
    ]

    def check_output(self, output: str, context: dict) -> dict:
        issues = []

        # Check for prohibited content
        for term in self.PROHIBITED_CONTENT:
            if term.lower() in output.lower():
                issues.append(f"Output contains prohibited term: {term}")

        # Check for unexpected URLs
        urls = re.findall(r'https?://\S+', output)
        for url in urls:
            domain = urlparse(url).netloc
            if domain not in ALLOWLISTED_DOMAINS:
                issues.append(f"Output contains non-allowlisted URL: {url}")

        # Check output length is reasonable
        if len(output) > 10000:
            issues.append("Output suspiciously long")

        return {"passed": len(issues) == 0, "issues": issues}

PII Scrubbing

Sending personal data to third-party LLM providers raises GDPR, CCPA, and HIPAA compliance concerns. Your users' data becomes subject to OpenAI's or Anthropic's data handling policies unless you're on an enterprise plan with a Data Processing Agreement.

Detect PII Before Sending

import re
from presidio_analyzer import AnalyzerEngine

analyzer = AnalyzerEngine()

PII_PATTERNS = {
    "email": r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b',
    "ssn": r'\b\d{3}-\d{2}-\d{4}\b',
    "phone": r'\b(\+\d{1,2}\s?)?\(?\d{3}\)?[\s.-]?\d{3}[\s.-]?\d{4}\b',
    "credit_card": r'\b\d{4}[\s-]?\d{4}[\s-]?\d{4}[\s-]?\d{4}\b',
}

def detect_pii(text: str) -> list[dict]:
    """Use Microsoft Presidio for comprehensive PII detection."""
    results = analyzer.analyze(text=text, language="en")
    return [
        {"type": r.entity_type, "start": r.start, "end": r.end, "score": r.score}
        for r in results if r.score > 0.7
    ]

def redact_pii(text: str) -> tuple[str, list[dict]]:
    """Redact PII and return redacted text plus a log of what was removed."""
    pii_found = detect_pii(text)
    redacted = text

    # Process in reverse order to preserve character positions
    for item in sorted(pii_found, key=lambda x: x["start"], reverse=True):
        placeholder = f"[{item['type']}]"
        redacted = redacted[:item["start"]] + placeholder + redacted[item["end"]:]

    return redacted, pii_found

Build a PII Gateway

For applications where PII flows regularly, build a preprocessing layer:

class PIIGateway:
    def __init__(self, llm_client):
        self.llm = llm_client
        self.pii_map = {}  # token -> original value, per-request

    def prepare_message(self, text: str) -> str:
        """Replace PII with reversible tokens before sending to LLM."""
        redacted, pii_items = redact_pii(text)
        return redacted

    def restore_response(self, response: str, pii_map: dict) -> str:
        """Restore tokens in model output back to original values if needed."""
        for token, original in pii_map.items():
            response = response.replace(token, original)
        return response

Enterprise API Tiers

If you require PII in prompts (e.g., healthcare applications), use enterprise tiers that include zero data retention agreements:

• OpenAI Enterprise: Zero data retention, SOC 2 Type II, BAA available for HIPAA
• Anthropic Claude for Enterprise: No training on your data, DPA available
• Azure OpenAI: Data stays in your Azure tenant, HIPAA-eligible, no training on inputs

Self-hosted models (Ollama, vLLM, LM Studio) eliminate third-party data sharing entirely but require your own infrastructure and security controls.

Logging and Audit Trails

Log every LLM interaction for security review, but do so carefully:

def log_llm_interaction(
    request_id: str,
    user_id: str,
    model: str,
    prompt_tokens: int,
    completion_tokens: int,
    flagged: bool,
    duration_ms: int,
):
    # Log metadata, NOT the actual prompt content (which may contain PII)
    logger.info("llm_request", extra={
        "request_id": request_id,
        "user_id": user_id,
        "model": model,
        "prompt_tokens": prompt_tokens,
        "completion_tokens": completion_tokens,
        "flagged": flagged,
        "duration_ms": duration_ms,
    })

Store full prompt/response content separately in a high-security log store with strict access controls and a defined retention period. In GDPR jurisdictions, define your legal basis for retaining conversation logs and ensure your retention period is proportionate.

Summary

Securing LLM API integrations requires controls at every layer:

Layer	Control
Key management	Secrets manager, rotation, scope restriction
Rate limiting	Per-user RPM/RPD/token limits in Redis
Cost controls	max_tokens on every call, spend alerts
Input validation	Length limits, injection screening
PII handling	Presidio redaction, enterprise DPA
Output validation	Pydantic schemas, semantic guardrails
Audit	Metadata logging, content logging with retention policy

None of these controls are optional in production. The combination of financial exposure, data privacy obligations, and prompt injection risk makes LLM API security a first-class concern, not an afterthought.