# Tool Search Pattern: Middleware Filtering

Demonstrate dynamic tool discovery using middleware filtering.

When agents have many tools (30-50+), providing all tools upfront causes:

1. **Context bloat**: Tool definitions consume tokens
2. **Selection errors**: LLM struggles to choose the right tool

**Approach**: Register all tools at creation, but use middleware to filter
which tools are shown to the LLM based on `tool_search` results.

**Comparison with interrupt pattern** (`tool-search-interrupt.ipynb`):

| Aspect | Filter Pattern | Interrupt Pattern |
|--------|---------------|-------------------|
| Complexity | Simpler | More complex |
| Token saving | Yes | Yes |
| Tool registration | All upfront | Dynamic |
| Use case | Known tool set | Truly dynamic tools |

## Setup


```python
import re
from collections.abc import Callable, Sequence
from typing import Any

import rich
from dotenv import load_dotenv
from langchain.agents import create_agent
from langchain.agents.middleware.types import (
    AgentMiddleware,
    AgentState,
    ModelRequest,
    ModelResponse,
)
from langchain_anthropic import ChatAnthropic
from langchain_core.tools import BaseTool, tool
from langgraph.graph.state import CompiledStateGraph

load_dotenv()
```




    True



## Define Tool Registry

Create a registry of all available tools. All tools are registered upfront,
but only shown to LLM after discovery.


```python
@tool
def get_weather(city: str) -> str:
    """Get current weather for a city."""
    return f"Weather in {city}: Sunny, 22°C"


@tool
def get_forecast(city: str, days: int = 3) -> str:
    """Get weather forecast for a city."""
    return f"{days}-day forecast for {city}: Sunny → Cloudy → Rain"


@tool
def send_email(to: str, subject: str, body: str) -> str:
    """Send an email to a recipient."""
    return f"Email sent to {to}: {subject}"


@tool
def read_emails(folder: str = "inbox") -> str:
    """Read emails from a folder."""
    return f"3 unread emails in {folder}"


@tool
def create_calendar_event(title: str, date: str) -> str:
    """Create a calendar event."""
    return f"Event '{title}' created on {date}"


@tool
def list_calendar_events(date: str) -> str:
    """List calendar events for a date."""
    return f"Events on {date}: Meeting at 10am, Lunch at 12pm"


# Tool registry: name -> tool
TOOL_REGISTRY: dict[str, BaseTool] = {
    t.name: t
    for t in [
        get_weather,
        get_forecast,
        send_email,
        read_emails,
        create_calendar_event,
        list_calendar_events,
    ]
}

rich.print("Available tools:", list(TOOL_REGISTRY.keys()))
```


    Available tools:
    ['get_weather', 'get_forecast', 'send_email', 'read_emails', 'create_calendar_event', 'list_calendar_events']



## Tool Search Tool

Returns tool names matching a regex pattern. The middleware will use this
to track which tools have been discovered.


```python
@tool
def tool_search(pattern: str) -> list[dict[str, str]]:
    """Search for available tools by regex pattern.

    Args:
        pattern: Regex pattern to match against tool names and descriptions.

    Returns:
        List of matching tools with name and description.
    """
    matches = []
    for name, t in TOOL_REGISTRY.items():
        if re.search(pattern, name, re.IGNORECASE) or re.search(pattern, t.description, re.IGNORECASE):
            matches.append({"name": name, "description": t.description})
    return matches
```

## Filter Middleware

Tracks discovered tools via `tool_search` results and filters which tools
are shown to LLM. All tools are registered, but only discovered ones are visible.


```python
import json

from langchain_core.messages import ToolMessage
from langgraph.prebuilt.tool_node import ToolCallRequest
from langgraph.types import Command


def get_tool_names(tools: Sequence[BaseTool | dict[str, Any]]) -> list[str]:
    """Extract tool names from a list of tools."""
    return [t.name if isinstance(t, BaseTool) else t.get("name", "?") for t in tools]


class ToolSearchFilterMiddleware(AgentMiddleware[AgentState[Any], None]):
    """Middleware that filters tools based on tool_search results.

    - Intercepts tool_search results to track discovered tools
    - Filters model requests to only show tool_search + discovered tools
    - Saves tokens by hiding undiscovered tools from LLM
    """

    def __init__(self, tool_registry: dict[str, BaseTool]):
        self.tool_registry = tool_registry
        self.discovered_tools: set[str] = set()

    def wrap_model_call(
        self,
        request: ModelRequest,
        handler: Callable[[ModelRequest], ModelResponse],
    ) -> ModelResponse:
        """Filter tools to only show tool_search + discovered tools."""
        # Build filtered tool list
        filtered: list[BaseTool | dict[str, Any]] = []
        for t in request.tools:
            if isinstance(t, BaseTool):
                # Always include tool_search, plus discovered tools
                if t.name == "tool_search" or t.name in self.discovered_tools:
                    filtered.append(t)
            else:
                # Keep dict tools as-is
                filtered.append(t)

        rich.print(f"[dim]Visible tools: {get_tool_names(filtered)}[/dim]")
        request = request.override(tools=filtered)
        return handler(request)

    def wrap_tool_call(
        self,
        request: ToolCallRequest,
        handler: Callable[[ToolCallRequest], ToolMessage | Command[Any]],
    ) -> ToolMessage | Command[Any]:
        """Intercept tool_search results to track discovered tools."""
        result = handler(request)

        # Check if this was a tool_search call
        tool_name = request.tool.name if isinstance(request.tool, BaseTool) else ""
        if tool_name == "tool_search" and isinstance(result, ToolMessage):
            # Parse results and track discovered tools
            try:
                tools = json.loads(str(result.content))
                for t in tools:
                    if isinstance(t, dict) and t.get("name"):
                        name = t["name"]
                        if name in self.tool_registry:
                            self.discovered_tools.add(name)
            except json.JSONDecodeError:
                pass
            rich.print(f"[yellow]Discovered: {self.discovered_tools}[/yellow]")

        return result
```

## Create Agent

Register ALL tools upfront (including tool_search). The middleware will
filter which ones are shown to the LLM.


```python
middleware = ToolSearchFilterMiddleware(TOOL_REGISTRY)
model = ChatAnthropic(model="claude-sonnet-4-5-20250929")

# Register ALL tools - middleware will filter visibility
all_tools: list[BaseTool] = [tool_search, *TOOL_REGISTRY.values()]

agent: CompiledStateGraph[Any] = create_agent(
    model=model,
    tools=all_tools,
    system_prompt="You are a helpful assistant. Use tool_search to find available tools.",
    middleware=[middleware],
)

rich.print(f"Agent created with {len(all_tools)} tools (only tool_search visible initially)")
```


    Agent created with 7 tools (only tool_search visible initially)



## Test: Dynamic Tool Discovery

Ask about weather - agent will search for tools, discover weather tools,
then use them in subsequent model calls.


```python
for chunk in agent.stream(
    {"messages": [{"role": "user", "content": "What's the weather in Tokyo?"}]},
):
    rich.print("chunk =", chunk)
```


    Visible tools: ['tool_search']




    chunk =
    {
        'model': {
            'messages': [
                AIMessage(
                    content=[
                        {
                            'text': "I'll search for weather-related tools to help you get the weather information for 
    Tokyo.",
                            'type': 'text'
                        },
                        {
                            'id': 'toolu_01UaPpkB5V7taAvZAAQu4ktS',
                            'input': {'pattern': 'weather'},
                            'name': 'tool_search',
                            'type': 'tool_use'
                        }
                    ],
                    additional_kwargs={},
                    response_metadata={
                        'id': 'msg_01E6e1A8LnkFzquYTTMEMzbT',
                        'model': 'claude-sonnet-4-5-20250929',
                        'stop_reason': 'tool_use',
                        'stop_sequence': None,
                        'usage': {
                            'cache_creation': {'ephemeral_1h_input_tokens': 0, 'ephemeral_5m_input_tokens': 0},
                            'cache_creation_input_tokens': 0,
                            'cache_read_input_tokens': 0,
                            'input_tokens': 619,
                            'output_tokens': 72,
                            'server_tool_use': None,
                            'service_tier': 'standard'
                        },
                        'model_name': 'claude-sonnet-4-5-20250929',
                        'model_provider': 'anthropic'
                    },
                    id='lc_run--195757b7-51c6-4f80-81fb-fb130ca92547-0',
                    tool_calls=[
                        {
                            'name': 'tool_search',
                            'args': {'pattern': 'weather'},
                            'id': 'toolu_01UaPpkB5V7taAvZAAQu4ktS',
                            'type': 'tool_call'
                        }
                    ],
                    usage_metadata={
                        'input_tokens': 619,
                        'output_tokens': 72,
                        'total_tokens': 691,
                        'input_token_details': {
                            'cache_read': 0,
                            'cache_creation': 0,
                            'ephemeral_5m_input_tokens': 0,
                            'ephemeral_1h_input_tokens': 0
                        }
                    }
                )
            ]
        }
    }




    Discovered: {'get_weather', 'get_forecast'}




    chunk =
    {
        'tools': {
            'messages': [
                ToolMessage(
                    content='[{"name": "get_weather", "description": "Get current weather for a city."}, {"name": 
    "get_forecast", "description": "Get weather forecast for a city."}]',
                    name='tool_search',
                    id='b660b4ae-5cad-4b3c-bea9-f3431f63af76',
                    tool_call_id='toolu_01UaPpkB5V7taAvZAAQu4ktS'
                )
            ]
        }
    }




    Visible tools: ['tool_search', 'get_weather', 'get_forecast']




    chunk =
    {
        'model': {
            'messages': [
                AIMessage(
                    content=[
                        {'text': 'Now let me get the current weather for Tokyo:', 'type': 'text'},
                        {
                            'id': 'toolu_01KuNpZzpV8R9Uor3q5Jc1Ak',
                            'input': {'city': 'Tokyo'},
                            'name': 'get_weather',
                            'type': 'tool_use'
                        }
                    ],
                    additional_kwargs={},
                    response_metadata={
                        'id': 'msg_01SgbGKF99Aoqicy1Z2MnPce',
                        'model': 'claude-sonnet-4-5-20250929',
                        'stop_reason': 'tool_use',
                        'stop_sequence': None,
                        'usage': {
                            'cache_creation': {'ephemeral_1h_input_tokens': 0, 'ephemeral_5m_input_tokens': 0},
                            'cache_creation_input_tokens': 0,
                            'cache_read_input_tokens': 0,
                            'input_tokens': 868,
                            'output_tokens': 64,
                            'server_tool_use': None,
                            'service_tier': 'standard'
                        },
                        'model_name': 'claude-sonnet-4-5-20250929',
                        'model_provider': 'anthropic'
                    },
                    id='lc_run--42b7ab8c-0ac5-4300-b207-fab44fcae6db-0',
                    tool_calls=[
                        {
                            'name': 'get_weather',
                            'args': {'city': 'Tokyo'},
                            'id': 'toolu_01KuNpZzpV8R9Uor3q5Jc1Ak',
                            'type': 'tool_call'
                        }
                    ],
                    usage_metadata={
                        'input_tokens': 868,
                        'output_tokens': 64,
                        'total_tokens': 932,
                        'input_token_details': {
                            'cache_read': 0,
                            'cache_creation': 0,
                            'ephemeral_5m_input_tokens': 0,
                            'ephemeral_1h_input_tokens': 0
                        }
                    }
                )
            ]
        }
    }




    chunk =
    {
        'tools': {
            'messages': [
                ToolMessage(
                    content='Weather in Tokyo: Sunny, 22°C',
                    name='get_weather',
                    id='8bd4c0fa-910b-42fa-86d6-1c9f859a7418',
                    tool_call_id='toolu_01KuNpZzpV8R9Uor3q5Jc1Ak'
                )
            ]
        }
    }




    Visible tools: ['tool_search', 'get_weather', 'get_forecast']




    chunk =
    {
        'model': {
            'messages': [
                AIMessage(
                    content="The current weather in Tokyo is **Sunny** with a temperature of **22°C** (approximately 
    72°F). It's a beautiful day there!",
                    additional_kwargs={},
                    response_metadata={
                        'id': 'msg_01EHAkKUAsDViXSnrLzFvShr',
                        'model': 'claude-sonnet-4-5-20250929',
                        'stop_reason': 'end_turn',
                        'stop_sequence': None,
                        'usage': {
                            'cache_creation': {'ephemeral_1h_input_tokens': 0, 'ephemeral_5m_input_tokens': 0},
                            'cache_creation_input_tokens': 0,
                            'cache_read_input_tokens': 0,
                            'input_tokens': 955,
                            'output_tokens': 36,
                            'server_tool_use': None,
                            'service_tier': 'standard'
                        },
                        'model_name': 'claude-sonnet-4-5-20250929',
                        'model_provider': 'anthropic'
                    },
                    id='lc_run--98ba7313-9079-44b7-a807-7ef45b938899-0',
                    usage_metadata={
                        'input_tokens': 955,
                        'output_tokens': 36,
                        'total_tokens': 991,
                        'input_token_details': {
                            'cache_read': 0,
                            'cache_creation': 0,
                            'ephemeral_5m_input_tokens': 0,
                            'ephemeral_1h_input_tokens': 0
                        }
                    }
                )
            ]
        }
    }



## Token Comparison

Notice the input tokens in each model call:
- First call: Only `tool_search` visible → fewer tokens
- Second call: `tool_search` + discovered tools → slightly more tokens

Without filtering, ALL 7 tools would be sent every time.

## Test: Accumulated Discovery

Discovered tools persist across calls. Let's ask about email too.


```python
rich.print(f"Currently discovered: {middleware.discovered_tools}")

for chunk in agent.stream(
    {"messages": [{"role": "user", "content": "Check my emails"}]},
):
    rich.print("chunk =", chunk)

rich.print(f"Now discovered: {middleware.discovered_tools}")
```


    Currently discovered: {'get_weather', 'get_forecast'}




    Visible tools: ['tool_search', 'get_weather', 'get_forecast']




    chunk =
    {
        'model': {
            'messages': [
                AIMessage(
                    content=[
                        {
                            'text': "I don't have access to a tool that can check your emails. Let me search for 
    available tools to see what I can help you with.",
                            'type': 'text'
                        },
                        {
                            'id': 'toolu_01UjQzaC4Kr45oKkbMurVuJV',
                            'input': {'pattern': 'email'},
                            'name': 'tool_search',
                            'type': 'tool_use'
                        }
                    ],
                    additional_kwargs={},
                    response_metadata={
                        'id': 'msg_01T6j1BMsmgvnQfZsnz7vsGS',
                        'model': 'claude-sonnet-4-5-20250929',
                        'stop_reason': 'tool_use',
                        'stop_sequence': None,
                        'usage': {
                            'cache_creation': {'ephemeral_1h_input_tokens': 0, 'ephemeral_5m_input_tokens': 0},
                            'cache_creation_input_tokens': 0,
                            'cache_read_input_tokens': 0,
                            'input_tokens': 739,
                            'output_tokens': 83,
                            'server_tool_use': None,
                            'service_tier': 'standard'
                        },
                        'model_name': 'claude-sonnet-4-5-20250929',
                        'model_provider': 'anthropic'
                    },
                    id='lc_run--52856d1c-685d-4029-a24e-f0430acfc49e-0',
                    tool_calls=[
                        {
                            'name': 'tool_search',
                            'args': {'pattern': 'email'},
                            'id': 'toolu_01UjQzaC4Kr45oKkbMurVuJV',
                            'type': 'tool_call'
                        }
                    ],
                    usage_metadata={
                        'input_tokens': 739,
                        'output_tokens': 83,
                        'total_tokens': 822,
                        'input_token_details': {
                            'cache_read': 0,
                            'cache_creation': 0,
                            'ephemeral_5m_input_tokens': 0,
                            'ephemeral_1h_input_tokens': 0
                        }
                    }
                )
            ]
        }
    }




    Discovered: {'read_emails', 'send_email', 'get_weather', 'get_forecast'}




    chunk =
    {
        'tools': {
            'messages': [
                ToolMessage(
                    content='[{"name": "send_email", "description": "Send an email to a recipient."}, {"name": 
    "read_emails", "description": "Read emails from a folder."}]',
                    name='tool_search',
                    id='6058ead1-f410-4c28-8bc4-0d6e986bb541',
                    tool_call_id='toolu_01UjQzaC4Kr45oKkbMurVuJV'
                )
            ]
        }
    }




    Visible tools: ['tool_search', 'get_weather', 'get_forecast', 'send_email', 'read_emails']




    chunk =
    {
        'model': {
            'messages': [
                AIMessage(
                    content=[
                        {
                            'text': 'Great! I do have a tool to read your emails. Let me check your inbox for you.',
                            'type': 'text'
                        },
                        {
                            'id': 'toolu_012aHtYzFeCQ8tPm8vAcgpsU',
                            'input': {},
                            'name': 'read_emails',
                            'type': 'tool_use'
                        }
                    ],
                    additional_kwargs={},
                    response_metadata={
                        'id': 'msg_01X9vmWHueHErUpW2S3y2uon',
                        'model': 'claude-sonnet-4-5-20250929',
                        'stop_reason': 'tool_use',
                        'stop_sequence': None,
                        'usage': {
                            'cache_creation': {'ephemeral_1h_input_tokens': 0, 'ephemeral_5m_input_tokens': 0},
                            'cache_creation_input_tokens': 0,
                            'cache_read_input_tokens': 0,
                            'input_tokens': 1007,
                            'output_tokens': 57,
                            'server_tool_use': None,
                            'service_tier': 'standard'
                        },
                        'model_name': 'claude-sonnet-4-5-20250929',
                        'model_provider': 'anthropic'
                    },
                    id='lc_run--e18d98cd-f34c-4dac-b2d1-74b9a816ebe9-0',
                    tool_calls=[
                        {
                            'name': 'read_emails',
                            'args': {},
                            'id': 'toolu_012aHtYzFeCQ8tPm8vAcgpsU',
                            'type': 'tool_call'
                        }
                    ],
                    usage_metadata={
                        'input_tokens': 1007,
                        'output_tokens': 57,
                        'total_tokens': 1064,
                        'input_token_details': {
                            'cache_read': 0,
                            'cache_creation': 0,
                            'ephemeral_5m_input_tokens': 0,
                            'ephemeral_1h_input_tokens': 0
                        }
                    }
                )
            ]
        }
    }




    chunk =
    {
        'tools': {
            'messages': [
                ToolMessage(
                    content='3 unread emails in inbox',
                    name='read_emails',
                    id='6209854f-86da-4558-9148-4ea47bb66a15',
                    tool_call_id='toolu_012aHtYzFeCQ8tPm8vAcgpsU'
                )
            ]
        }
    }




    Visible tools: ['tool_search', 'get_weather', 'get_forecast', 'send_email', 'read_emails']




    chunk =
    {
        'model': {
            'messages': [
                AIMessage(
                    content='You have **3 unread emails** in your inbox.',
                    additional_kwargs={},
                    response_metadata={
                        'id': 'msg_019pCNWLUhUxsfC5zteJM8LX',
                        'model': 'claude-sonnet-4-5-20250929',
                        'stop_reason': 'end_turn',
                        'stop_sequence': None,
                        'usage': {
                            'cache_creation': {'ephemeral_1h_input_tokens': 0, 'ephemeral_5m_input_tokens': 0},
                            'cache_creation_input_tokens': 0,
                            'cache_read_input_tokens': 0,
                            'input_tokens': 1083,
                            'output_tokens': 16,
                            'server_tool_use': None,
                            'service_tier': 'standard'
                        },
                        'model_name': 'claude-sonnet-4-5-20250929',
                        'model_provider': 'anthropic'
                    },
                    id='lc_run--e5dfaea7-7c65-4179-905b-11a86d796288-0',
                    usage_metadata={
                        'input_tokens': 1083,
                        'output_tokens': 16,
                        'total_tokens': 1099,
                        'input_token_details': {
                            'cache_read': 0,
                            'cache_creation': 0,
                            'ephemeral_5m_input_tokens': 0,
                            'ephemeral_1h_input_tokens': 0
                        }
                    }
                )
            ]
        }
    }




    Now discovered: {'read_emails', 'send_email', 'get_weather', 'get_forecast'}



## Conclusion

The filter pattern is simpler than the interrupt pattern because:

1. **No agent recreation** - Same agent instance throughout
2. **No interrupt/resume** - Normal execution flow
3. **State in middleware** - `discovered_tools` persists naturally

Trade-off: All tools must be registered upfront. For truly dynamic tools
(e.g., loaded from external sources), use the interrupt pattern.