SafePrompt
Prompt injection detection API by Reboot Media, Inc.
Built against OWASP LLM01Benchmark re-run every 6 hoursOpen-source SDK · MIT

LangChain

SafePrompt ships an official LangChain callback handler for both runtimes: safeprompt-langchain on PyPI and @safeprompt.dev/langchain on npm. Add it to any chain or agent and every prompt is screened before it reaches your LLM. With agents it also screens tool outputs, which is where indirect prompt injection hides.

The JS/TS guide is below. Jump to Python.

Install

npm install @safeprompt.dev/langchain

Peer dependency: @langchain/core ^0.3.0. You will also need a SafePrompt API key from the dashboard.

Quick Start

Pass a SafePromptCallbackHandler in the callbacks array of any chain. It fires before each LLM call and throws SafePromptBlockedError when a prompt is unsafe, so you handle it exactly like any other error from chain.call().

import { LLMChain } from 'langchain/chains';
import { ChatOpenAI } from '@langchain/openai';
import { PromptTemplate } from '@langchain/core/prompts';
import {
  SafePromptCallbackHandler,
  SafePromptBlockedError,
} from '@safeprompt.dev/langchain';

const chain = new LLMChain({
  llm: new ChatOpenAI({ model: 'gpt-4o-mini' }),
  prompt: PromptTemplate.fromTemplate('Answer: {input}'),
  callbacks: [
    new SafePromptCallbackHandler({
      apiKey: process.env.SAFEPROMPT_API_KEY!,
      userIP: req.ip, // end-user IP from your web framework
    }),
  ],
});

try {
  const { text } = await chain.call({ input: userInput });
  return res.json({ text });
} catch (err) {
  if (err instanceof SafePromptBlockedError) {
    return res.status(400).json({
      error: 'Prompt blocked for safety',
      threats: err.result.threats,
    });
  }
  throw err;
}

Configuration

new SafePromptCallbackHandler({
  apiKey: process.env.SAFEPROMPT_API_KEY!,
  userIP: '203.0.113.1',                   // REQUIRED — end-user IP

  provider: 'https://api.safeprompt.dev',  // default
  mode: 'balanced',                        // 'fast' | 'balanced' | 'strict'
  enforcement: 'block',                    // 'block' | 'log'
  onProviderError: 'fail-closed',          // 'fail-closed' | 'fail-open'
  sampleRate: 1.0,                         // 0..1 — fraction of prompts to screen

  onBlock: (prompt, result) => {
    console.warn('[safeprompt] blocked', result.threats);
  },
  onError: (prompt, err) => {
    console.error('[safeprompt] provider error', err.message);
  },
});

onProviderError decides what happens if SafePrompt itself is unreachable: fail-closed blocks the request, fail-open lets it through and fires onError. Choose deliberately for your risk profile.

Roll Out in Log Mode First

Set enforcement: 'log' to run the handler without ever aborting a chain. You still get onBlock events for anything it would have blocked, so you can watch real traffic in your logs or the SafePrompt dashboard, tune custom lists and sensitivity, then flip to enforcement: 'block' once you are confident.

new SafePromptCallbackHandler({
  apiKey: process.env.SAFEPROMPT_API_KEY!,
  userIP: req.ip,
  enforcement: 'log', // observe only — never throws
  onBlock: (prompt, result) => {
    logger.warn({ threats: result.threats }, 'would block');
  },
});

Agents & Indirect Injection

When you attach the handler to a LangChain agent, it also screens tool outputsthe moment a tool returns content that will be fed back to the LLM. This is the key defense against indirect prompt injection: instructions hidden inside a web page, a retrieved document, or an API response that the agent is about to read. Direct user input and untrusted tool output are both screened by the same handler, no extra wiring.

import { AgentExecutor, createReactAgent } from 'langchain/agents';
import { SafePromptCallbackHandler } from '@safeprompt.dev/langchain';

const executor = new AgentExecutor({
  agent: await createReactAgent({ llm, tools, prompt }),
  tools,
  callbacks: [
    new SafePromptCallbackHandler({
      apiKey: process.env.SAFEPROMPT_API_KEY!,
      userIP: req.ip,
    }),
  ],
});

Python

The safeprompt-langchain package exposes SafePromptCallbackHandler, a BaseCallbackHandler you pass in the callbacks list of any chain or agent.

Install

pip install safeprompt-langchain

Quick start

from langchain_core.prompts import ChatPromptTemplate
from langchain_openai import ChatOpenAI
from safeprompt_langchain import SafePromptCallbackHandler, SafePromptBlockedError

handler = SafePromptCallbackHandler(
    api_key="sp_live_...",   # get one at https://safeprompt.dev
    user_ip=request_ip,      # end-user IP from your web framework
)

chain = ChatPromptTemplate.from_template("Answer: {input}") | ChatOpenAI(model="gpt-4o-mini")

try:
    result = chain.invoke({"input": user_input}, config={"callbacks": [handler]})
    print(result.content)
except SafePromptBlockedError as err:
    # The prompt never reached the model.
    return "That request looked unsafe and was not sent."

api_key and user_ip are both required. The end-user IP is what powers abuse correlation across requests, so passing a real one matters.

Options

SafePromptCallbackHandler(
    api_key="sp_live_...",
    user_ip="203.0.113.1",                  # REQUIRED, end-user IP

    provider="https://api.safeprompt.dev",  # default
    mode="balanced",                        # "fast" | "balanced" | "strict"
    enforcement="block",                    # "block" raises, "log" only reports
    on_provider_error="fail-closed",        # "fail-closed" | "fail-open"
    sample_rate=1.0,                        # fraction of calls validated
    timeout=30.0,                           # seconds
    on_block=None,                          # callable(text, result)
    on_error=None,                          # callable(exc)
)

Roll out in log mode first

Set enforcement="log" to see what would have been blocked without blocking anything. Nothing is rejected, and on_block still fires, so you can measure the real false-positive rate against your own traffic before enforcing.

handler = SafePromptCallbackHandler(
    api_key="sp_live_...",
    user_ip=request_ip,
    enforcement="log",
    on_block=lambda text, result: logger.warning("would have blocked: %s", result),
)

Fail-closed by default

If SafePrompt itself cannot be reached, on_provider_error="fail-closed" raises rather than letting an unchecked prompt through. That is the default on purpose. Set "fail-open" only if availability matters more to you than the check.

Next Steps