> For the complete documentation index, see [llms.txt](https://docs.interactive.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.interactive.ai/improvement/datasets.md). # Datasets A dataset is a fixed set of test cases. Each item holds an input, the expected output, and metadata, so the same inputs can be replayed against a new prompt, a different model, or your own code, and the results compared against what came before. ### Dataset Items There are **three ways to fill a dataset**. New Item adds one by hand, Upload CSV loads them in bulk, and Add to Datasets on a trace in Observability turns a real interaction into a test case, carrying its input, output, and metadata across.
Opening an item shows its input, expected output, and metadata alongside every run that has used it, which is how you tell whether one particular case has got better or worse over time. ### Experiments An experiment runs your system over every item in the dataset and records what came out. There are two kinds. A **prompt experiment** is configured here with no code. Pick a context and the version of it to test, a provider and model, and the evaluators that should score the results. This is what lets you hold the dataset still and vary exactly one thing, a prompt version or a model, so that any difference in the output is attributable. The prompt's variables take their values from the dataset by name, so a prompt containing `{{input}}` receives each item's input.
A [**custom experiment** ](/sdk/experiments.md)runs through the SDK instead, for when the thing under test is your own code rather than a prompt and a model. Evaluators can be attached to the dataset itself as well as to a single experiment, so every run gets scored the same way without setting it up each time. Attached this way their variables map to fields from the experiment runs rather than from traces. ### Runs and comparison **Every experiment produces a run**. The **Runs tab** charts latency and average model cost across all of them, so a change that improved outputs while doubling the cost is visible before you read a single result. Any score in the project can be added as a chart of its own, which is how you watch quality move from one run to the next rather than only speed and spend.
Opening a run lists each item with its output beside the expected output, and links to the trace that produced it, so a disappointing result leads straight to the execution behind it.
Compare takes two or more runs and gives each one its own column next to the item's input and expected output, carrying the output that run produced along with its latency, cost, and any scores. That side-by-side is the point of holding the dataset still in the first place.
--- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.interactive.ai/improvement/datasets.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.