ChatModelAgent

ChatModelAgent Overview

import "github.com/cloudwego/eino/adk"

What is ChatModelAgent

ChatModelAgent is the core Agent implementation of Eino ADK — it uses ChatModel as the decision maker, Tools as the action space, and autonomously drives problem solving through a ReAct Loop.

For a complete introduction to ChatModelAgent concepts, ReAct Loop, and the Middleware system, see: ChatModelAgent Introduction

ReAct Loop

When Tools are configured, ChatModelAgent executes in a ReAct loop:

Reason: Call the ChatModel, which decides the next action
Action: The model returns a ToolCall request
Act: Execute the corresponding Tool
Observation: Inject the Tool result into the context and start a new loop iteration

The loop continues until the model determines no further Tool calls are needed. Without Tools configured, it degrades to a single ChatModel call.

Configuration

TypedChatModelAgentConfig

type TypedChatModelAgentConfig[M MessageType] struct {
    Name        string
    Description string
    Instruction string

    Model       model.BaseModel[M]    // Required. Must support model.WithTools when using Tools

    ToolsConfig ToolsConfig
    GenModelInput TypedGenModelInput[M]

    Exit          tool.BaseTool         // NOT RECOMMENDED
    OutputKey     string                // NOT RECOMMENDED
    MaxIterations int                   // Default 20

    Handlers          []TypedChatModelAgentMiddleware[M]
    Middlewares        []AgentMiddleware  // Legacy compatibility

    ModelRetryConfig    *TypedModelRetryConfig[M]
    ModelFailoverConfig *ModelFailoverConfig[M]
}

// Default alias
type ChatModelAgentConfig = TypedChatModelAgentConfig[*schema.Message]

Field Descriptions

Field	Description
Name	Agent name. Required when used as an AgentTool
Description	Agent capability description. Required when used as an AgentTool
Instruction	System Prompt. Supports {Key} placeholders; the default GenModelInput renders them using SessionValues
Model	Required. Type model.BaseModel[M] ; must support model.WithTools when using Tools
ToolsConfig	Tool configuration, see below for details
GenModelInput	Custom input transformation. By default, uses Instruction as System Message + f-string rendering
MaxIterations	Maximum ReAct loop iterations; exits with error when exceeded. Default 20
Handlers	Interface-based Middleware ( TypedChatModelAgentMiddleware[M] ), recommended
Middlewares	Struct-based Middleware ( AgentMiddleware ), legacy compatibility
ModelRetryConfig	Retry strategy for failed model calls
ModelFailoverConfig	Switch to a backup model on failure. Requires configuring GetFailoverModel and ShouldFailover

💡 The default GenModelInput uses pyfmt rendering, so { and } in Messages are treated as placeholders. To output these characters literally, escape them with {{ and }}.

ToolsConfig

type ToolsConfig struct {
    compose.ToolsNodeConfig

    ReturnDirectly     map[string]bool  // Tool names that return directly after execution
    EmitInternalEvents bool             // Forward AgentTool internal events
}

ReturnDirectly: When a matching Tool is executed, the Agent exits immediately without calling the model again. If multiple Tools match, the first one is used
EmitInternalEvents: When a sub-Agent is called via AgentTool, its events are forwarded in real-time to the parent Agent’s event stream

Constructors

func NewChatModelAgent(ctx context.Context, config *ChatModelAgentConfig) (*ChatModelAgent, error)
func NewTypedChatModelAgent[M MessageType](ctx context.Context, config *TypedChatModelAgentConfig[M]) (*TypedChatModelAgent[M], error)

Middleware (ChatModelAgentMiddleware)

Interface Definition

type TypedChatModelAgentMiddleware[M MessageType] interface {
    BeforeAgent(ctx context.Context, runCtx *ChatModelAgentContext) (context.Context, *ChatModelAgentContext, error)
    AfterAgent(ctx context.Context, state *TypedChatModelAgentState[M]) (context.Context, error)

    BeforeModelRewriteState(ctx context.Context, state *TypedChatModelAgentState[M], mc *TypedModelContext[M]) (context.Context, *TypedChatModelAgentState[M], error)
    AfterModelRewriteState(ctx context.Context, state *TypedChatModelAgentState[M], mc *TypedModelContext[M]) (context.Context, *TypedChatModelAgentState[M], error)

    WrapModel(ctx context.Context, m model.BaseModel[M], mc *TypedModelContext[M]) (model.BaseModel[M], error)

    WrapInvokableToolCall(ctx context.Context, endpoint InvokableToolCallEndpoint, tCtx *ToolContext) (InvokableToolCallEndpoint, error)
    WrapStreamableToolCall(ctx context.Context, endpoint StreamableToolCallEndpoint, tCtx *ToolContext) (StreamableToolCallEndpoint, error)
    WrapEnhancedInvokableToolCall(ctx context.Context, endpoint EnhancedInvokableToolCallEndpoint, tCtx *ToolContext) (EnhancedInvokableToolCallEndpoint, error)
    WrapEnhancedStreamableToolCall(ctx context.Context, endpoint EnhancedStreamableToolCallEndpoint, tCtx *ToolContext) (EnhancedStreamableToolCallEndpoint, error)
}

type ChatModelAgentMiddleware = TypedChatModelAgentMiddleware[*schema.Message]

Embed *BaseChatModelAgentMiddleware to only override the methods you need:

type MyMiddleware struct {
    *adk.BaseChatModelAgentMiddleware
}

func (m *MyMiddleware) BeforeModelRewriteState(
    ctx context.Context,
    state *adk.ChatModelAgentState,
    mc *adk.ModelContext,
) (context.Context, *adk.ChatModelAgentState, error) {
    // Custom logic
    return ctx, state, nil
}

Hook Points

Hook	Timing	Modifiable Content
BeforeAgent	Before Agent runs (once only)	Instruction, Tools, ReturnDirectly, ToolSearchTool
AfterAgent	After Agent completes successfully	Read final state (no modification)
BeforeModelRewriteState	Before each model call	Messages, ToolInfos, DeferredToolInfos (persisted to state)
AfterModelRewriteState	After each model call	Messages (including model response), ToolInfos (persisted to state)
WrapModel	Wrap model calls	Retry, failover, event emission (do not modify Messages)
WrapToolCall	Wrap tool calls	Permission checks, logging, output rewriting

💡 The state returned by BeforeModelRewriteState is persisted by the framework to the agent’s internal state. Therefore, modifications in this hook (such as compressing Messages or filtering ToolInfos) affect all subsequent iterations.

Core Types

ChatModelAgentContext (BeforeAgent Parameter)

type ChatModelAgentContext struct {
    Instruction    string
    Tools          []tool.BaseTool
    ReturnDirectly map[string]bool
    ToolSearchTool *schema.ToolInfo  // Model's native ToolSearch capability
}

ChatModelAgentState (BeforeModel/AfterModel Parameter)

type TypedChatModelAgentState[M MessageType] struct {
    Messages          []M
    ToolInfos         []*schema.ToolInfo         // Tool list passed to the model
    DeferredToolInfos []*schema.ToolInfo         // Server-side deferred tool list
}

type ChatModelAgentState = TypedChatModelAgentState[*schema.Message]

ModelContext (WrapModel Parameter)

type TypedModelContext[M MessageType] struct {
    Tools               []*schema.ToolInfo          // Deprecated: use state.ToolInfos
    ModelRetryConfig    *TypedModelRetryConfig[M]
    ModelFailoverConfig *ModelFailoverConfig[M]
}

type ModelContext = TypedModelContext[*schema.Message]

Execution Order

Model call chain (outer to inner):

AgentMiddleware.BeforeChatModel
BeforeModelRewriteState
failover wrapper (built-in)
retry wrapper (built-in)
event sender wrapper (built-in)
WrapModel (first registered = outermost)
callback injection (built-in)
Actual model call
AfterModelRewriteState
AgentMiddleware.AfterChatModel

Tool call chain (outer to inner):

event sender (built-in)
ToolsConfig.ToolCallMiddlewares
AgentMiddleware.WrapToolCall
WrapToolCall (first registered = outermost)
callback injection (built-in)
Actual tool call

AgentAsTool

Wrap a sub-Agent as a Tool, allowing the parent Agent to call it autonomously via ToolCall:

subAgent, _ := adk.NewChatModelAgent(ctx, &adk.ChatModelAgentConfig{
    Name:        "researcher",
    Description: "Search and summarize information",
    Model:       chatModel,
    // ...
})

agentTool := adk.NewAgentTool(ctx, subAgent)

parentAgent, _ := adk.NewChatModelAgent(ctx, &adk.ChatModelAgentConfig{
    // ...
    ToolsConfig: adk.ToolsConfig{
        ToolsNodeConfig: compose.ToolsNodeConfig{
            Tools: []tool.BaseTool{agentTool},
        },
    },
})

Generic version: adk.NewTypedAgentTool[M](ctx, agent, options...)

Options: WithFullChatHistoryAsInput() (pass full chat history), WithAgentInputSchema(schema) (custom input schema)

ModelRetry

When configured, ChatModel calls are automatically retried on failure. When an error occurs during streaming, the current stream is still returned via AgentEvent, and consuming the MessageStream yields a WillRetryError:

agent, _ := adk.NewChatModelAgent(ctx, &adk.ChatModelAgentConfig{
    // ...
    ModelRetryConfig: &adk.ModelRetryConfig{
        // Retry strategy configuration
    },
})

// Handle WillRetryError when consuming the event stream
stream := event.Output.MessageOutput.MessageStream
for {
    msg, err := stream.Recv()
    if err == io.EOF {
        break
    }
    if err != nil {
        var willRetry *adk.WillRetryError
        if errors.As(err, &willRetry) {
            log.Printf("Attempt %d failed, retrying...", willRetry.RetryAttempt)
            break // Wait for the next event
        }
        break
    }
    displayChunk(msg)
}

ModelFailover

When configured, the agent switches to a backup model on failure:

agent, _ := adk.NewChatModelAgent(ctx, &adk.ChatModelAgentConfig{
    Model: primaryModel,
    ModelFailoverConfig: &adk.ModelFailoverConfig{
        GetFailoverModel: func(ctx context.Context, err error) (model.BaseModel[*schema.Message], error) {
            return backupModel, nil
        },
        ShouldFailover: func(err error) bool {
            return true // Decide whether to failover based on error type
        },
    },
})

Cancel

A runtime cancellation capability introduced in v0.9. See Agent Cancel and TurnLoop for details.

cancelOpt, cancelFn := adk.WithCancel()
iter := runner.Run(ctx, messages, cancelOpt)

// Cancel later (CancelMode supports bitmask combination)
handle := cancelFn(adk.CancelAfterChatModel | adk.CancelAfterToolCalls)
handle.Wait() // Wait for cancellation to complete

ChatModel Failover Guide

Feedback

Was this page helpful?

Please tell us how we can improve.

Last modified May 18, 2026: docs(eino): sync english translations (ffdf777e)