- Overview
- Getting Started with UiPath Agents
- Getting Started with UiPath Agents using LangGraph
- Introduction
- Set up your environment
- Build the agent
- Evaluate the agent
- Connect to Studio Web
- Building a Low-Code Agent in Studio Web
- Adding Tools to Your UiPath Agent
Run evaluation traces locally and review results flowing into Studio Web.
With the agent running locally and the entry points registered, you are ready to run it and build an evaluation set.
Step 6 - Run the agent locally
Run the agent with the sample input file your coding agent created:
uip codedagent run agent --file input.json
uip codedagent run agent --file input.json
You should see the agent classify the request and return a tier with reasoning.
You can also pass input inline. These examples use Bash single-quote syntax; if you are in PowerShell, use --file with a JSON file instead:
uip codedagent run agent '{"description": "Clear the rats out of the inn cellar"}'
uip codedagent run agent '{"description": "Clear the rats out of the inn cellar"}'
Try a few different inputs to verify the classifications make sense:
uip codedagent run agent '{"description": "Slay the ancient red dragon terrorizing the countryside"}'
uip codedagent run agent '{"description": "Slay the ancient red dragon terrorizing the countryside"}'
Step 7 - Create evaluation tests
Evaluations test how well your agent performs across a range of inputs. The uipath-agents skill includes the complete evaluation framework reference: evaluator types, eval set schema, directory structure conventions, and best practices like using gpt-4.1 (not mini) for LLM judge evaluators. Your coding agent uses this to produce correct evaluator configs and test sets from a short prompt.
Ask your coding agent:
Create an evaluation set for the intake classifier agent with 5 test cases:
1. A clearly trivial request (e.g., delivering a letter)
2. A standard request (e.g., escort a caravan)
3. A heroic request (e.g., clear a goblin stronghold)
4. A legendary request (e.g., slay a dragon)
5. An edge case with ambiguous difficulty
Use both a semantic similarity evaluator (to check the output) and a
trajectory evaluator (to check the agent's reasoning path).
Include evaluator config files in evaluations/evaluators/ and the eval
set in evaluations/eval-sets/. Use gpt-4.1-2025-04-14 as the model in
the evaluator configs. Each evaluator config must include a populated
defaultEvaluationCriteria - use {"expectedOutput": {}} for the
semantic evaluator and {"expectedAgentBehavior": ""} for the
trajectory evaluator. Empty {} fails schema validation.
Create an evaluation set for the intake classifier agent with 5 test cases:
1. A clearly trivial request (e.g., delivering a letter)
2. A standard request (e.g., escort a caravan)
3. A heroic request (e.g., clear a goblin stronghold)
4. A legendary request (e.g., slay a dragon)
5. An edge case with ambiguous difficulty
Use both a semantic similarity evaluator (to check the output) and a
trajectory evaluator (to check the agent's reasoning path).
Include evaluator config files in evaluations/evaluators/ and the eval
set in evaluations/eval-sets/. Use gpt-4.1-2025-04-14 as the model in
the evaluator configs. Each evaluator config must include a populated
defaultEvaluationCriteria - use {"expectedOutput": {}} for the
semantic evaluator and {"expectedAgentBehavior": ""} for the
trajectory evaluator. Empty {} fails schema validation.
Step 8 - Run evaluations
Run the evaluation set locally:
uip codedagent eval agent evaluations/eval-sets/smoke-test.json --workers 3 --output-file eval-results.json
uip codedagent eval agent evaluations/eval-sets/smoke-test.json --workers 3 --output-file eval-results.json
The evaluation framework runs each test case through your agent and scores the results.
| Score | What it measures |
|---|---|
| Semantic similarity | How closely the agent's output matches the expected output |
| Agent trajectory | Whether the agent took the expected reasoning path |
Expect trajectory scores of 0.0 on this agent. Trajectory evaluators judge the agent's reasoning path: which tools it called, in what order, and how it routed between nodes. This classifier makes no tool calls and has a single node, so every test case scores 0.0. For this single-node classifier, lean on semantic similarity scores.
For semantic similarity, scores above 0.8 are generally solid. Review eval-results.json to see how your agent performed.
After you connect to Studio Web in the next step, running uip codedagent eval run from the CLI uploads results to Studio Web automatically; they appear in the Evaluation Sets tab under Runs.
The Studio Web Run Evals button is not the same thing. That button triggers a cloud robot execution requiring Python runtime support — a more involved setup outside the scope of this lab. Use uip codedagent eval run from the CLI instead; results appear in Studio Web either way.
With local evaluation results confirmed, you are ready to connect the project to Studio Web in the next section.