What an agent is
Each agent has:| Property | What it controls |
|---|---|
| Name | What customers see in the UI (“Hi, I’m Nova”) |
| Greeting | First message on webchat |
| Status | live (answers automatically) or paused (only drafts) |
| System prompt | The core instructions — tone, scope, rules, what to refuse |
| Languages | Which languages it’s allowed to reply in |
| Model | The LLM the agent runs on (e.g. claude-sonnet-4, gpt-4o-mini) — set per agent via the LLM router |
| Temperature | How creative vs deterministic (low = safer, high = more varied) |
| Confidence threshold | The minimum self-rated confidence (0.0–1.0) for the agent to send a reply itself. Below it, the conversation hands off to a teammate. Default is 0.70. |
| Auto-send | When on, the agent sends replies it’s confident about; when off, it only drafts |
| Knowledge sources | Which crawls, files, snippets, Q&As, and Shopify feeds it can draw from |
| Tools | What it’s allowed to do — look up Shopify orders, call your custom HTTP endpoints, hand off, etc. |
| Avatar | Profile image shown in the widget header and inbox |
How an agent actually replies
When a message arrives on a channel an agent is attached to:- Keloa picks the best knowledge passages for this question using semantic search.
- The system prompt + passages + conversation history go to the LLM.
- The model drafts a reply. If it wants to use a tool (look up order, hand off), it calls it.
- Reply goes back to the customer and into the inbox.
Grounded answers, not hallucinations
The agent only answers from your knowledge sources and the conversation itself. If it doesn’t know, it says so — or hands off to a human via a flow. You’ll never catch it inventing a policy you don’t have. If the agent is guessing when it shouldn’t, that’s almost always fixable by:- Adding the answer to your knowledge (a Q&A pair is fastest).
- Tightening the system prompt — “If you don’t find it in the knowledge, say you’ll route to a teammate and hand off.”
- Raising the confidence threshold so borderline replies escalate instead of sending.
Confidence threshold
Every reply gets a self-rated confidence score between 0 and 1. The agent’s confidence threshold (default0.70) is the line below which the agent stops sending and escalates to a human via the configured handoff flow. Raise it to be stricter, lower it to be more aggressive.
Confidence appears next to the AI badge on every assisted message in the inbox, and the distribution is plotted in Agent performance so you can see whether the threshold is calibrated.
Model providers
Keloa routes each agent through either Anthropic (Claude) or OpenAI (GPT) automatically based on the model selected. Both providers run with zero-retention flags so prompts and completions are never used for training.One agent or several?
Start with one. Split when you notice:- The system prompt is getting conflicting rules (“be playful on pre-sales, strict on refunds”).
- Different channels need different personalities (formal on email, casual on Instagram DM).
- You want different knowledge scopes (only product pages for pre-sales, only policy pages for returns).
Next
Create an agent
The step-by-step for a new agent.
Write a system prompt
The single highest-leverage thing you can edit.
Tools
What your agent can do beyond answering.
Custom HTTP tool
Wire any of your own endpoints into the agent’s tool belt.
Test your agent
Chat with an agent before letting it loose on customers.