llmverify - LLM Checker npm Package
Local-first LLM output monitoring, risk scoring, and classification for Node.js.
Built for teams shipping AI features who need guardrails that work without sending data to third parties.
The Problem
You're shipping AI features. Your LLM outputs need validation before they reach users-prompt injection detection, PII redaction, hallucination risk scoring. The standard options are:
- •Cloud APIs that require sending your data to third parties
- •Complex ML pipelines that add latency and infrastructure overhead
- •Manual regex that's brittle and doesn't scale
Most teams end up with a patchwork of solutions-or skip validation entirely and hope for the best.
What llmverify Does
A single npm package that handles the common LLM safety checks. No cloud dependencies. No ML infrastructure. Just import and use.
Prompt Injection Detection
PII Redaction
Hallucination Risk Scoring
Runtime Health Monitoring
JSON Repair & Validation
Model-Agnostic Adapters
Quick Start
npm install llmverifyimport { verify, isInputSafe, redactPII } from 'llmverify';
// Verify AI output safety
const result = await verify({ content: aiOutput });
if (result.risk.level === 'critical') {
console.log('Block this content');
}
// Check user input for prompt injection
if (!isInputSafe(userInput)) {
throw new Error('Potential attack detected');
}
// Redact PII before displaying
const { redacted } = redactPII(aiOutput);
console.log(redacted); // "Contact [REDACTED] at [REDACTED]"Limitations
llmverify uses heuristics, not AI. It provides risk indicators, not ground truth.
All results include confidence intervals, methodology explanations, and explicit limitations. This is a guardrail layer, not a replacement for human review on high-stakes decisions.
Compliance Alignment
llmverify provides technical evaluation and monitoring checks that can support selected activities within the AI Governance Execution Framework, implementing baseline checks for:
OWASP LLM Top 10
Security
NIST AI RMF
Risk Management
EU AI Act
Compliance
ISO 42001
AI Management
Privacy Guarantee
What We Do
- • Zero network requests
- • Zero telemetry
- • Zero data collection
- • Open source-verify yourself
What We Never Do
- • Train on your data
- • Share with third parties
- • Track without consent
- • Phone home for any reason
Run tcpdump while using it-you'll see zero network traffic.
Why I Built This
I've spent years building AI governance frameworks for large-scale enterprise deployments. I've seen what happens when teams ship AI features without proper guardrails-and I've seen the compliance overhead that comes with enterprise-grade solutions.
Most teams don't need a full ML pipeline for basic safety checks. They need something that works out of the box, runs locally, and doesn't require a PhD to configure.
llmverify is that tool. It's not perfect-the limitations section is honest about that-but it covers the 80% case for teams who need to ship safely without overengineering.
How It Works
Install the Package
Run npm i llmverify in your project. Zero dependencies, zero config required.
Import & Wrap
Import llmverify and pass your LLM output through the verify function. Works with OpenAI, Anthropic, or any LLM provider.
Configure Checks
Enable the checks you need: prompt injection detection, PII redaction, hallucination risk scoring, JSON validation. All run locally.
Monitor Results
Get structured risk scores and flags for every LLM response. Block, log, or alert based on your own thresholds.
Use Cases
Chatbot Safety
Detect prompt injection attacks in user messages before they reach your LLM. Block jailbreak attempts and keep your chatbot within its intended behavior.
RAG Pipeline Validation
Validate LLM outputs in retrieval-augmented generation pipelines. Check for hallucination risk when the LLM generates answers from your knowledge base.
AI Agent Guardrails
Add safety checks to autonomous AI agents before they take actions. Verify outputs for PII leaks, prompt injection, and hallucination before executing tool calls.
Customer Support AI Monitoring
Monitor LLM-powered customer support responses for PII leaks and hallucination risk. Ensure your AI assistant never exposes sensitive customer data or fabricates answers.
Frequently Asked Questions
Is llmverify an AI model?
No. llmverify is a deterministic rule-based library. It uses pattern matching, regex, and heuristic scoring - not a neural network. This means results are reproducible, fast, and require no GPU or model inference.
Does it send data anywhere?
No. llmverify makes zero network requests. All processing happens in your Node.js process. You can verify this with tcpdump or any network monitor - there is zero telemetry, zero data collection, zero phone-home behavior.
What LLM providers are supported?
llmverify is provider-agnostic. It works with any LLM output - OpenAI, Anthropic, Google, Mistral, local models via Ollama, or any custom LLM. You pass the output text to llmverify, and it returns structured verification results.
How accurate is prompt injection detection?
llmverify uses pattern-based detection which catches common injection patterns (ignore previous instructions, role-play attacks, encoding tricks). It is not 100% - novel or sophisticated attacks may evade detection. It covers the 80% case for teams who need basic guardrails without deploying a separate ML model.
Can I use it in production?
Yes. llmverify is MIT licensed and production-ready. It is used in production by teams shipping AI features. The API is stable and backward-compatible. However, it should be used as one layer in a defense-in-depth strategy, not your only safety measure.
How does it compare to Langfuse or guardrails AI?
Langfuse is an observability platform that requires a server and database. guardrails AI is a Python library with ML-based validation. llmverify is a zero-dependency npm package that runs locally with deterministic checks. Choose llmverify for lightweight, local-first guardrails. Choose Langfuse for full observability. Choose guardrails AI for ML-based validation in Python.