Key Concepts
Bulk data testing capabilities
A Bulk Run Test is a feature that verifies whether an Ability produces consistent results across various input values.
While standard testing using the Ability Test is sufficient for basic function checks, before deploying to production, you should use large sets of input data to validate the accuracy, structure, and quality of results.
This is especially important when using LLM nodes, as output formats may vary even when the input structure remains the same — making bulk run testing a critical step in quality assurance.
Additionally, this feature can also be used for batch processing large datasets using a specific Ability, beyond just testing purposes.
Accuracy Verification: Ensures that the Ability generates consistent outputs across diverse inputs.
Quality Control: Allows you to pre-check the structure and quality of responses from LLM-based Abilities.
Automated Bulk Processing: Automatically processes repetitive input data, saving significant time.
Preparing for Bulk Run Tests
1) Create an Ability Version
Once your Ability is complete, create a new Version.
Bulk Run Tests can only be executed on Abilities that have an existing version.
2) Create Data Storages
Because Bulk Run Tests automatically process multiple inputs and outputs, you need two Storages — one for input data and one for output data.
In the Data Structure tab, click +Create Data Structure to add a new structure.
Then, in the Structured Data tab, click +Create Storage to register the structure as a storage.
Storage Type | Purpose | Notes |
|---|---|---|
Input Storage | Stores the input data to be passed to the Ability | Must match the structure of the Ability’s Input variables |
Output Storage | Stores the execution results of the Ability | Must match the structure of the Ability’s Output variables |
How to Execute
1) Create a Bulk Run
Create Test
Click the
+Create Bulk Runbutton and select the Ability and version you want to test.Enter a name and description for the test, then click
Create.
2) Map Storage Data
Input Storage Setup
In Input Storage Setup, select the pre-created Input Storage.
Map the Ability’s input variables to the corresponding storage fields, then click
Save.
Output Storage Setup
In Output Storage Setup, select the pre-created Output Storage.
Map the Ability’s output variables to the storage fields, then click
Save.
When configuring storages, variable names may differ, but data types must match exactly.
Run Test
Click the
Start Testbutton at the top right to begin the test.
View Results
After the test is complete, go to the
Execution Resulttab to view the full results.You can open result data directly or download it as a CSV file for external validation and review.