If you use resources in agents, it’s important to understand how Cogniti agents use them.
Importantly: Resources are not considered in their entirety when the agent generates a response.
Resources are documents that can be provided to assist Cogniti agents in creating more contextualised and accurate responses.
When a user types in a prompt, Cogniti uses this approach:
Step 1 – The user’s prompt is used to perform a text search over the agent’s resources. This search looks for textual and semantic similarity. For example, if the user’s prompt is “What do chickens eat?”, Cogniti will search through its resources to find snippets (basically, paragraphs of text) that match that prompt.
The system message is not used to search over the agent’s resources. Even if you tell the agent in the system message to use the resources, this does not mean it will.
Step 2 – The top few snippets that are found (ranked according to relevance) are then fed into the large language model (e.g. GPT-4), along with the system message, and the user’s prompt. The large language model then generates the response (‘completion’) for the user.
There are a few implications for using resources in Cogniti agents:
– Adding resources does slow down completions slightly
– Only snippets of the resources are sent to the large language model for consideration, not the full resource
If you have information that is critical to the function of your agent, ensure you put that information into the system message.