Your proactive agent engineer

LangSmith Engine finds agent issues, writes prompt and code fixes, and continuously monitors for regressions.

Get started
Request a demo

Helping top teams ship great agents

Use cases in production

Three ways to build,
from full harness to full control

Each package solves a different problem. Choose whereyou want to start, then compose the rest ofthestack around it.

Automatically detect agent issues

Engine analyzes LangSmith traces for failures, and red-teams your agent to find issues that haven’t appeared in production yet. Every issue is tracked in a single queue and sorted by severity, with evidence from your traces to validate bad behavior.

Engine can:

  • Identify and cluster issues with your agents from production traces
  • Prioritize issues based on severity and your preferences
  • Red team your agent to find issues before they hit production

Review, approve, and ship fixes fast

For each issue, Engine summarizes the failure mode, writes the prompt or code fix, and confirms that it works without introducing regressions. If you connect your codebase, Engine can open a GitHub PR with the proposed change, ready for your team to review and merge.

Engine can:

  • Write prompt and code changes based on production failures
  • Validate its fixes against your agent
  • Open GitHub PRs for review

Prevent issues from coming back

Engine stays on guard even after an issue is resolved. It automatically monitors your agent for issue recurrences, and surfaces dataset examples for your offline evals.

Engine can:

  • Monitor for regressions of previously-resolved issues
  • Reopen recurring issues for engineering review
  • Recommend production examples for evaluation datasets

Resources for LangSmith Engine

LangSmith Engine v2: red teaming and automated testing
New in LangSmith Engine: >2x better issue detection
Get started with LangSmith Engine

FAQs for LangSmith Engine

What model do you use for LangSmith Engine?

LangSmith Engine uses LLMs provided through LangChain. We do not support bringing your own model for Engine so that we can control the full Engine experience and deliver the best results for customers. Please read more about our subprocessors here and our notice to customers here.

What can I share with our security team on using Engine?

All model providers operate under zero data retention and are contractually prohibited from training or fine-tuning on your data.

How is Engine priced?

LangSmith Engine is a standalone agent that consumes LangChain Compute Units while it works. LCUs are a normalized unit of work that account for compute, storage, memory, and LLM usage.

Usage depends on the number of traces Engine analyzes, the depth of analysis required, and the amount of work it performs. To learn more about pricing, visit our pricing page.

Ready to add a proactive agent engineer to your team?

Start building
Get a demo