Key Concepts
Bulk Run Test
Bulk Run Test is a large-scale test feature that verifies whether an Ability produces consistent results across a wide range of input values. A regular Ability Test is enough for a quick functional check, but before deploying to production you should also verify the accuracy, structure, and quality of results using a large volume of input data. This matters especially for Abilities that use an LLM node, since output format can vary even for the same input structure — Bulk Run Test lets you catch this before launch. Beyond verification, you can also use this feature to batch-process large volumes of data with a given Ability.
This guide walks you through preparing an Ability version and the required storages, creating a Bulk Run, and checking the results.
Before You Begin
The Ability you want to test must already be complete. If you're not familiar with Data Structure and Data Storage, see 🔗Using Data Storage first — it makes the rest of this guide easier to follow.
Step 1: Create a Version of the Ability to Test
Once the Ability you want to test is complete, create a new version. Bulk Run Test can only run on an Ability that has a registered version.
Step 2: Set Up Input and Output Storage
Bulk Run Test automatically processes multiple input values and their results, so you need two separate storages — one for input values and one for results.
In the Data Structure tab, click Create Data Structure to add a data structure.
In the Data Storage tab, click Create Storage to register the data structure as a storage.
Storage type | Purpose | Note |
|---|---|---|
Input storage | Stores the input data passed to the Ability | Must match the Ability's input variable structure |
Output storage | Stores the Ability's execution results | Must match the Ability's output variable structure |
Step 3: Create a Bulk Run
Click Create Bulk Run and select the Ability and version you want to test. Enter a name and description for the test, then click Create.
Step 4: Map Storage Data
Configure Input Storage
In the Input Storage setup, select the input storage you created earlier. Map the Ability's input variables to the storage's fields, then click Save.
Configure Output Storage
In the Output Storage setup, select the output storage you created earlier. Map the Ability's output variables to the storage's fields, then click Save.
Variable names can differ, but the data types must match when mapping storage.
Step 5: Run the Test and Check Results
Click the Start Test button, active in the top-right corner, to run the test.
Once the test is complete, you can check the full results in the Results tab. You can review the result data directly or download it as a CSV file for external validation.
Next Steps
Setup is complete.
You can now use Bulk Run Test to verify that an Ability performs consistently across a range of input values.
🔗What Is Storage? — See how Bulk Run Test relates to Data Storage, Data Structure, and Slipbox.
🔗Using Data Storage — Revisit how to create a Data Structure and Data Storage.
Frequently Asked Questions
What is Bulk Run Test?
Bulk Run Test is a large-scale test feature that verifies whether an Ability produces consistent results across a wide range of input values. It runs the Ability once for each row in an input Data Storage, then stores the results in a separate output Data Storage.
When should I use Bulk Run Test?
Use it before deploying to production, when you need to verify the accuracy, structure, and quality of results across a large volume of input data. It's especially useful for verifying Abilities that use an LLM node, since output format can vary even for the same input structure. Beyond verification, you can also use it to batch-process large volumes of data with a given Ability.
Does Bulk Run Test require Data Storage?
Yes. Bulk Run Test runs on top of Data Storage, so you need an input storage for your input values and an output storage for your results. Both storages require a Data Structure to be defined first, and the input and output storages must be two separate storages.