LangSmith Observability

Instrument your LLM application, investigate traces, and monitor performance in production with LangSmith.

LangSmith Observability provides full visibility into your LLM application: from individual traces to production-wide performance metrics.

LangSmith works with many frameworks and providers. Browse available integrations to connect your stack including OpenAI, Anthropic, CrewAI, Vercel AI SDK, Pydantic AI, and more.

Get started

Set up tracing

Add tracing to your app in minutes with environment variables, framework integrations, or the SDK.

Trace a RAG application

Follow a step-by-step tutorial to instrument a retrieval-augmented generation app from start to finish.

Investigate and monitor

View traces

Filter, export, share, and compare traces via the UI or API.

Monitor performance

Build dashboards and set alerts to track quality and catch issues early.

Configure automations

Automate workflows with rules, webhooks, and online evaluations.

Collect feedback

Annotate outputs and gather user feedback using queues or inline annotation.

Find and fix failures with Engine

Automatically detect recurring issues in your traces, diagnose their root cause, and resolve them with LangSmith Engine.

For terminology and core concepts, refer to Observability concepts.

Usage and billing

Data retention

This section covers how data retention works and how it’s priced in LangSmith.

Why retention matters

• Privacy: Many data privacy regulations, such as GDPR in Europe or CCPA in California, require organizations to delete personal data once it’s no longer necessary for the purposes for which it was collected. Setting retention periods aids in compliance with such regulations.
• Cost: LangSmith charges less for traces that have low data retention. For more information, learn how to enforce spend limits.

Plan your retention tiers before you start sending traces. Changes apply to new traces only—existing traces keep their original tier. See Change project-level default retention.

How it works

LangSmith has two tiers of traces based on Data Retention with the following characteristics:

	Base	Extended
Price	See pricing page	See pricing page
Retention Period	14 days	400 days

Enterprise customers can customize the extended retention period per workspace. Changes apply to new traces only—existing traces are unaffected. See Customize extended retention policy.

Data deletion after retention ends

After the specified retention period, traces are no longer accessible in the tracing project UI or via the API. All user data associated with the trace (e.g. inputs and outputs) is deleted from our internal systems within a day thereafter. Some metadata associated with each trace may be retained indefinitely for analytics and billing purposes.

Data retention auto-upgrades

Auto upgrades can have an impact on your bill. Please read this section carefully to fully understand your estimated LangSmith tracing costs.

When you use certain features with base tier traces, their data retention will be automatically upgraded to extended tier. This will increase both the retention period, and the cost of the trace.

The complete list of scenarios in which a trace will upgrade when:

• Feedback is added to any run on the trace (or any trace in the thread), whether through manual annotation, automatically with an online evaluator, or programmatically via the SDK.
• An annotation queue receives any run from the trace.
• An automation rule matches any run within a trace.

Why auto-upgrade traces?

We have two reasons behind the auto-upgrade model for tracing:

1. We think that traces that match any of these conditions are fundamentally more interesting than other traces, and therefore it is good for users to be able to keep them around longer.
2. We philosophically want to charge customers an order of magnitude lower for traces that may not be interacted with meaningfully. We think auto-upgrades align our pricing model with the value that LangSmith brings, where only traces with meaningful interaction are charged at a higher rate.

If you have questions or concerns about our pricing model, please feel free to contact support via support.langchain.com and let us know your thoughts!

How does data retention affect downstream features?

• Annotation Queues, Run Rules, and Feedback: Traces that use these features will be auto-upgraded.
• Monitoring: The monitoring tab will continue to work even after a base tier trace’s data retention period ends. It is powered by trace metadata that exists for >30 days, meaning that your monitoring graphs will continue to stay accurate even on base tier traces.
• Datasets: Datasets have an indefinite data retention period. Restated differently, if you add a trace’s inputs and outputs to a dataset, they will never be deleted. We suggest that if you are using LangSmith for data collection, you take advantage of the datasets feature.

Billing model

Billable metrics

On your LangSmith invoice, you will see two metrics that we charge for:

• LangSmith Traces (Base Charge)
• LangSmith Traces (Extended Data Retention Upgrades).

The first metric includes all traces, regardless of tier. The second metric just counts the number of extended retention traces.

Why measure all traces + upgrades instead of base and extended traces?

A natural question to ask when considering our pricing is why not just show the number of base tier and extended tier traces directly on the invoice?

While we understand this would be more straightforward, it doesn’t fit trace upgrades properly. Consider a base tier trace that was recorded on June 30, and upgraded to extended tier on July 3. The base tier trace occurred in the June billing period, but the upgrade occurred in the July billing period. Therefore, we need to be able to measure these two events independently to properly bill our customers.

If your trace was recorded as an extended retention trace, then the base and extended metrics will both be recorded with the same timestamp.

Cost breakdown

The Base Charge for a trace is .05¢ per trace. We priced the upgrade such that an extended retention trace costs 10x the price of a base tier trace (.50¢ per trace) including both metrics. Thus, each upgrade costs .45¢.

Rate limits

LangSmith has rate limits which are designed to ensure the stability of the service for all users.

To ensure access and stability, LangSmith will respond with HTTP Status Code 429 indicating that rate or usage limits have been exceeded under the following circumstances:

Temporary throughput limit over a 1 minute period at our application load balancer

This 429 is the result of exceeding a fixed number of API calls over a 1 minute window on a per service key or PAT basis. The start of the window will vary slightly—it is not guaranteed to start at the start of a clock minute—and may change depending on application deployment events.

After the max events are received we will respond with a 429 until 60 seconds from the start of the evaluation window has been reached and then the process repeats.

This 429 is thrown by our application load balancer and is a mechanism in place for all LangSmith users independent of plan tier to ensure continuity of service for all users.

Method	Endpoints	Limit	Window
DELETE	/sessions*	30	1 minute
POST OR PATCH	/runs*	5000	1 minute
GET	/runs/:id	30	1 minute
POST	/feedbacks*	5000	1 minute
*	*	2000	1 minute

The LangSmith SDK takes steps to minimize the likelihood of reaching these limits on run-related endpoints by batching up to 100 runs from a single session ID into a single API call.

Plan-level hourly trace event limit

This 429 is the result of reaching your maximum hourly events ingested and is evaluated in a fixed window starting at the beginning of each clock hour in UTC and resets at the top of each new hour.

An event in this context is the creation or update of a run. If a run is created and then subsequently updated in the same hourly window, that counts as 2 events against this limit.

This is thrown by our application and varies by plan tier, with organizations on our Startup/Plus and Enterprise plan tiers having higher hourly limits than our Free and Developer Plan Tiers which are designed for personal use.

Plan	Limit	Window
Developer (no payment on file)	50,000 events	1 hour
Developer (with payment on file)	250,000 events	1 hour
Startup/Plus	500,000 events	1 hour
Enterprise	Custom	Custom

Plan-level hourly trace data ingest limit

This 429 is the result of reaching the maximum amount of data ingested across your trace inputs, outputs, and metadata and is evaluated in a fixed window starting at the beginning of each clock hour in UTC and resets at the top of each new hour.

Typically, inputs, outputs, and metadata are sent on both run creation and update events. If a run is created at 2.0MB and updated to 3.0MB in the same hourly window, that counts as 5.0MB of storage against this limit.

This is thrown by our application and varies by plan tier, with organizations on our Startup/Plus and Enterprise plan tiers having higher hourly limits than our Free and Developer Plan Tiers which are designed for personal use.

Plan	Limit	Window
Developer (no payment on file)	500MB	1 hour
Developer (with payment on file)	2.5GB	1 hour
Startup/Plus	5.0GB	1 hour
Enterprise	Custom	Custom

Plan-level monthly unique traces limit

This 429 is the result of reaching your maximum monthly traces ingested and is evaluated in a fixed window starting at the beginning of each calendar month in UTC and resets at the beginning of each new month.

This is thrown by our application and applies only to the Developer Plan Tier when there is no payment method on file.

Plan	Limit	Window
Developer (no payment on file)	5,000 traces	1 month

Self-configured monthly usage limits

This 429 is the result of reaching your usage limit as configured by your organization admin and is evaluated in a fixed window starting at the beginning of each calendar month in UTC and resets at the beginning of each new month.

This is thrown by our application and varies by organization based on their configured settings.

Maximum runs per trace

Each trace is limited to a maximum of 25,000 runs. Once the trace reaches this limit, LangSmith will reject any additional runs that you send for that trace.

Run query endpoint

The POST /runs/query endpoint has additional per-tenant rate limits based on query parameters. See Query traces using the SDK for details.

Handling 429s responses in your application

Since some 429 responses are temporary and may succeed on a successive call, if you are directly calling the LangSmith API in your application we recommend implementing retry logic with exponential backoff and jitter.

For convenience, LangChain applications built with the LangSmith SDK has this capability built-in.

It is important to note that if you are saturating the endpoints for extended periods of time, retries may not be effective as your application will eventually run large enough backlogs to exhaust all retries.

If that is the case, we would like to discuss your needs more specifically. Please contact support via LangSmith Support with details about your applications throughput needs and sample code and we can work with you to better understand whether the best approach is fixing a bug, changes to your application code, or a different LangSmith plan.

Usage limits

LangSmith lets you configure usage limits on tracing. Note that these are usage limits, not spend limits, which mean they let you limit the quantity of occurrences of some event rather than the total amount you will spend.

LangSmith lets you set two different monthly limits, mirroring our Billable Metrics discussed in the aforementioned data retention guide:

• All traces limit
• Extended data retention traces limit

These let you limit the number of total traces, and extended data retention traces respectively.

Properties of usage limiting

Usage limiting is approximate, meaning that we do not guarantee the exactness of the limit. In rare cases, there may be a small period of time where additional traces are processed above the limit threshold before usage limiting begins to apply.

Side effects of extended data retention traces limit

The extended data retention traces limit has side effects. If the limit is already reached, any feature that could cause an auto-upgrade of tracing tiers becomes inaccessible. This is because an auto-upgrade of a trace would cause another extended retention trace to be created, which in turn should not be allowed by the limit. Therefore, you can no longer:

1. match run rules
2. add feedback to traces
3. add runs to annotation queues

Each of these features may cause an auto upgrade, so we shut them off when the limit is reached.

Updating usage limits

Usage limits can be updated from the Settings page under Usage and Billing. Limit values are cached, so it may take a minute or two before the new limits apply.

Related content

• Tutorial on how to enforce spend limits

To set up a LangSmith instance, visit the Platform setup section to choose between cloud, hybrid, or self-hosted. All options include observability, evaluation, prompt engineering, and deployment.

---

Connect these docs to Claude, VSCode, and more via MCP for real-time answers.

Edit this page on GitHub or file an issue.