Skip to main content

Arc Agents

Welcome to Arc Agents, a kotlin based framework for writing AI powered applications.

Agents​

An Arc Agent is a collection of Large Language Model prompts, filters, events, readers and other services that combined can perform a complex task.

Goal and Instructions​

Each Agent should have a single goal or task they are designed to complete from which they should not deviate.

For example, a joke agent's goal is to tell jokes. It should not be possible to use the joke agent to do anything else, such as, maths homework.

Example of instructions for a weather agent:

# Goal

You are a professional weather agent. You provide weather data to your users.

How an Agent arrives at its goal is defined by the instructions given to it.

Example of instructions for a weather agent:

# Instructions

- If the user asks a question that is not related to weather,
  simply reply I cant help you
- Be kind and friendly to the user.
- Inform the user that the weather data is not real-time data.
- Use the get_weather function to get the weather data.

Both the goal and instructions of an Agent are defined within an Agent's System Prompt.

See Defining Agents for more details.

Knowledge​

Knowledge or Knowledge Injection refers to supplying Agents with additional information that they require perform their tasks.

For example, a Travel Agent may need to know the current Holiday packages that are available.

There are 2 ways to dynamically inject knowledge into an Agent, Readers and Functions. Both mechanisms enable Agents to access real-time data.

Static Knowledge can simple be added to the System Prompt of the Agent.

Filters​

The purpose of filters is to transform, augment, validate or filter the input and output of Agents.

Input filters, for example, can be used to anonymize sensitive data before handing it to the Agent LLM. And Output filters can be used to detect Hallucinations generated by the Agent LLM.

See Filters for more details.

Tools​

Tools enable Agents to access external services and data. Tools are a powerful way to extend the capabilities of an Agent. Enabling Agents to not only retrieve data, but to also execute real-world actions, such as turning on a light or even booking a flight.

For example, a Weather Agent may use a Weather Tool to get the weather data. Whereas a Travel Agent may use the Booking Tool to book flights.

Tools are often implemented as Functions. See Functions for more details.

Memory​

Memory is the ability of an Agent to store and retrieve information.

There are 2 types of memory:

  • LONG_TERM - information stored in long term memory is persistent and can be accessed by the Agent at any time.
  • SHORT_TERM - information stored in short term memory is only available for the duration of a conversation.

Each information stored in memory is associated with an owner, usually the user defined in the conversation.

The Arc Agent Framework declares the following interface for memory:

interface Memory {

    /**
     * Store a value in LONG_TERM memory.
     * @param owner The owner of the memory. For example, the user id.
     * @param key The key to store the value under.
     * @param value The value to store. If null, the value is removed from memory.
     */
    fun storeLongTerm(owner: String, key: String, value: Any?)

    /**
     * Store a value in SHORT_TERM memory.
     * @param owner The owner of the memory. For example, the user id.
     * @param key The key to store the value under.
     * @param value The value to store. If null, the value is removed from memory.
     * @param sessionId The session id to store the value under.
     */
    fun storeShortTerm(owner: String, key: String, value: Any?, sessionId: String)

    /**
     * Fetch a value from memory.
     * @param owner The owner of the memory. For example, the user id.
     * @param key The key to fetch the value for.
     * @param sessionId The session id to fetch the value for. Only used if the value was stored under SHORT_TERM memory.
     * @return The value stored under the key, or null if no value is stored.
     */
    fun fetch(owner: String, key: String, sessionId: String? = null): Any?
}

Each application wanting to use memory must provide an implementation of the Memory interface. See the chapter on Memory for available implementations.

Eventing​

Eventing is a great way to decouple communication between different parts of an application.

The Arc Agent Framework provides the following interfaces for enabling eventing between your application and components of the Arc Agent Framework:

  /**
 * EventPublisher interface.
 */
fun interface EventPublisher {
    fun publish(event: Event)
}

/**
 * Add EventHandlers to the implementation of this event to receive events.
 */
fun interface EventListeners {
    fun add(handler: EventHandler<out Event>)
}

Out of the box, components of the Arc Framework will publish various events. The events are useful for monitoring the system.

Custom events can be easily published throughout the Arc Agent DSL using the emit function.

  agent {
    name = "..."
    description = "..."
    prompt { "..." }
    filterInput {
        emit(MyCustomEvent())
    }
}

See the Eventing for more details.

Implementation Details​

The following highlights some of the main components that implement the Arc Agent Concept.

ChatAgent​

The ChatAgent is the main implementation of the Arc Agent interface.

As the name suggests, ChatAgents are agents that conduct a conversation with the client/user. They take a Conversation object as input and outputs the Conversation containing a new message. See the Conversations section for more details on conversations.

AgentProvider​

The AgentProvider provides agents to other components in your application. Agents can be loaded from different sources, for example, from kotlin scripts or Spring beans. The AgentProvider consolidates all these agents loaders in the application. Therefore, each application should have at most one AgentProvider.

fun interface AgentProvider {

    fun getAgents(): List<Agent<*, *>>
}

When using the spring boot starter, the CompositeAgentProvider is automatically configured as the default AgentProvider. The CompositeAgentProvider combines the agents of multiple AgentLoaders and any Agents injected directly into the Spring Context.

AgentLoader​

AgentLoaders load agents from a specific source. Unlike the AgentProvider, an application may have multiple AgentLoaders.

fun interface AgentLoader {

    fun getAgents(): List<Agent<*, *>>
}

The main AgentLoader of the Arc Agent Framework is the ScriptAgentLoader that loads agents from Kotlin scripts.

Agents can also be loaded as Spring Beans. See Spring Agent Beans

Conversations​

The Conversation object represents a conversation between a client and an Arc Agent, such as a ChatAgent. It contains the entire transcript of the conversation, both client and agent messages, plus some metadata about the conversation.

Each Conversation object must be associated with a User which acts as the owner of the conversation. This to prevent a conversation from being accessed by unauthorized users/clients. Furthermore, the User id may also be used as a key to store user/client specific data.

Messages are usually be one of 2 types:

  • UserMessage - messages sent by the user/client.
  • AssistantMessage - messages generated by the Agent.

SystemMessages, messages that contain instructions for the Agent, are usually not store in the conversation transcript.