AI Tool Integration for Agents: The Architecture Guide

The first one takes an afternoon. Slack, one endpoint, one token, a function that posts a message and hands back a timestamp. Done before lunch. The sixth takes a sprint, and by then not one line of your agent loop has changed. AI tool integration is the work that gets estimated once and paid for every week after, because the thing that grows is not the number of tools. It is the number of independent surfaces you have to keep alive.

Day 14 defined a single tool: name, description, schema, executor, result, error. Clean. This is what happens when you need five of them, wired to real systems, running for real users, at 3am when a provider rotates a certificate.

Why AI tool integration cost grows faster than tool count

One agent, one tool, one wiring. Add four more and the naive read is five afternoons of work. Wrong in both directions. Some of it was cheaper because you copied the HTTP client. Most of it was dearer, because each wiring drags six independent concerns behind it, none of them shared with the tool you wired last week.

Then someone ships a second agent. A nightly summarizer, an eval harness, a support bot that needs the CRM read the main agent already has. If each agent talks to tools directly you have agents times tools wirings. Three agents and five tools is fifteen. Times six concerns is ninety things that can rot quietly until a customer finds one.

The M times N framing is old and not specific to AI. It just arrives faster here, because an agent makes it very cheap to want another tool. Direct wiring costs M times N. A boundary that every agent speaks and every tool implements costs M plus N. At six agents and twenty tools that is one hundred and twenty versus twenty six, which is the difference between a sprint and a team.

The six axes where tools diverge

Integrations do not compose because no two providers agree on anything below the word "API".

1. API shape

Slack's Web API is method style: you POST to https://slack.com/api/chat.postMessage and the operation lives in the path, not the verb. GitHub is resource oriented REST where the verb carries meaning. Notion rejects any request without a Notion-Version header. Some internal services are gRPC, some payment APIs still want form encoded bodies. Request construction is per provider and the differences are load bearing.

2. Authentication

Five providers can mean five genuinely different mechanisms, each failing in its own way.

• Static API key. A secret in a header. Never expires on its own, not tied to a user, rotated when a human decides. Easiest to build, worst blast radius, because one key usually means every tenant shares an identity.
• OAuth 2.0 user tokens. An access token that expires, a refresh token, and a per user record you store. Refresh fails with 400 {"error": "invalid_grant"} when access was revoked or the refresh token rotated out from under you. Two concurrent calls that both see an expired token will both refresh, and with rotating refresh tokens one wins while the other permanently kills the credential.
• Service accounts. Sign a JWT with a private key, exchange it for an access token, no user in the loop. Google Workspace layers domain wide delegation on top, so the same call returns different data depending on who is impersonated.
• Request signing. AWS SigV4 signs a canonical request per call, so nothing caches, and the signature depends on your clock. A few minutes of skew gives 403 SignatureDoesNotMatch, which reads like a credential problem and is an NTP problem.
• Mutual TLS. The credential lives in the handshake, not your code. Certificate distribution, a trust store, expiry monitoring, and a failure your HTTP library surfaces nowhere near a 401.

Four questions separate these: where the secret lives, how long it lives, whose identity it carries, and what happens when it dies. No two of the five answer all four alike.

3. Schemas and pagination

Timestamps are ISO 8601 here, epoch seconds there, epoch milliseconds next door, and Slack's ts looks like a float but is a message identifier you must never do arithmetic on. Missing values are null in one API and an absent key in the next, which matters the second you write row["assignee"]["name"].

Pagination is worse because it fails silently. Nothing crashes. You return page one and the agent reports that the customer has four orders. Four conventions are in wide use: a cursor in the body like Slack's response_metadata.next_cursor or Notion's next_cursor with has_more, a cursor in a header like GitHub's Link with rel="next", a page token like Google Calendar's nextPageToken, and plain offset and limit, where rows shift under you mid scan.

4. Credentials as a lifecycle

Authentication is the mechanism. Credentials are the operations problem, and separate work. Where is the secret stored, and is it encrypted with a key you actually rotate. Is it per tenant, per user, or one shared key pretending to be per user. Can you rotate without a deploy. What happens when a user disconnects an integration and forty queued jobs are holding a dead token. Who can read the secret, and is that read audited. None of it is tool logic.

5. Deployment and runtime surface

"Add a tool" sounds like adding a file. Sometimes it is adding a container. A vendor SDK pulls native dependencies that pin your Python version. A browser tool needs headless Chromium, a gigabyte of memory and a process supervisor. An internal service sits inside a VPC, so the agent needs network placement, not a token.

6. Error semantics

Slack commonly returns 200 OK with {"ok": false, "error": "invalid_auth"} in the body. Branch on status_code alone and that call is logged as a success. GitHub returns 401 {"message": "Bad credentials"}. An OAuth refresh returns 400 invalid_grant. AWS returns 403 SignatureDoesNotMatch. Rate limits arrive as 429 with Retry-After, or as GitHub's 403 with X-RateLimit-Remaining: 0.

No agent reasons well across five taxonomies. It will retry a permission error forever and give up on a rate limit that would have cleared in thirty seconds.

One shape for every tool call

All of it collapses if you refuse to let provider shapes past a single boundary. The rule is unglamorous: the agent never sees a raw provider response.

from dataclasses import dataclass
from enum import Enum
from typing import Any, Protocol
 
 
class ErrorKind(str, Enum):
    BAD_INPUT = "bad_input"        # the agent sent something invalid
    AUTH = "auth"                  # credential missing, expired or rejected
    PERMISSION = "permission"      # credential is valid, scope is not
    NOT_FOUND = "not_found"        # the target does not exist
    RATE_LIMIT = "rate_limit"      # slow down and come back
    UNAVAILABLE = "unavailable"    # provider is having a bad day
 
 
@dataclass(frozen=True)
class ToolError:
    kind: ErrorKind
    message: str                   # written for the model, not for a log
    retry_after: float | None = None
    provider_code: str | None = None
 
 
@dataclass(frozen=True)
class ToolCall:
    tool: str
    args: dict
    tenant_id: str                 # who this call runs as
    actor_id: str                  # which user inside that tenant
    idempotency_key: str | None = None
 
 
@dataclass(frozen=True)
class ToolResult:
    ok: bool
    data: Any = None
    error: ToolError | None = None
    next_cursor: str | None = None
 
 
class Tool(Protocol):
    name: str
    description: str
    input_schema: dict
    writes: bool
 
    def invoke(self, call: ToolCall) -> ToolResult: ...

Four things do real work. Identity travels on the call instead of being read from ambient state. ToolError carries a kind the agent can branch on rather than a string it has to parse. retry_after is a number, not a header. And next_cursor is uniform, so pagination is one pattern regardless of the provider's convention.

The adapter base is a template method with four hooks, one per axis that actually differs.

import httpx
 
 
class HTTPTool:
    """Base for any tool that is an HTTP call into somebody else's service."""
 
    name: str
    provider: str                  # which credential to resolve
    base_url: str
    writes: bool = False
 
    def __init__(self, credentials, client: httpx.Client):
        self.credentials, self.client = credentials, client
 
    # per provider hooks
    def build_request(self, call: ToolCall) -> httpx.Request: ...
 
    def authorize(self, request: httpx.Request, secret: str) -> httpx.Request:
        request.headers["Authorization"] = f"Bearer {secret}"
        return request
 
    def classify(self, response: httpx.Response) -> ToolError | None:
        """Return None only if this response is genuinely a success."""
        ...
 
    def parse(self, response: httpx.Response) -> tuple[Any, str | None]:
        return response.json(), None          # (data, next_cursor)
 
    # the part nobody overrides
    def invoke(self, call: ToolCall) -> ToolResult:
        secret = self.credentials.resolve(call.tenant_id, self.provider)
        request = self.authorize(self.build_request(call), secret)
        try:
            response = self.client.send(request, timeout=20.0)
        except (httpx.TimeoutException, httpx.TransportError) as exc:
            return ToolResult(False, error=ToolError(
                ErrorKind.UNAVAILABLE, f"Could not reach the provider: {exc!r}"))
 
        error = self.classify(response)
        if error:
            return ToolResult(False, error=error)
        data, cursor = self.parse(response)
        return ToolResult(True, data=data, next_cursor=cursor)

A Slack adapter then says exactly one interesting thing: a 200 is not a success.

class SlackPostMessage(HTTPTool):
    name, provider, writes = "slack_post_message", "slack", True
    base_url = "https://slack.com/api"
 
    ERRORS = {"invalid_auth": ErrorKind.AUTH, "token_revoked": ErrorKind.AUTH,
              "missing_scope": ErrorKind.PERMISSION,
              "channel_not_found": ErrorKind.NOT_FOUND}
 
    def build_request(self, call: ToolCall) -> httpx.Request:
        return self.client.build_request(
            "POST", f"{self.base_url}/chat.postMessage", json=call.args)
 
    def classify(self, response: httpx.Response) -> ToolError | None:
        if response.status_code == 429:
            return ToolError(ErrorKind.RATE_LIMIT, "Slack is rate limiting this workspace.",
                             retry_after=float(response.headers.get("Retry-After", 30)))
        if response.status_code >= 500:
            return ToolError(ErrorKind.UNAVAILABLE, "Slack returned a server error.")
 
        body = response.json()
        if body.get("ok"):
            return None                       # the only real success path
 
        code = body.get("error", "unknown")
        return ToolError(self.ERRORS.get(code, ErrorKind.BAD_INPUT),
                         f"Slack rejected the call: {code}.", provider_code=code)

Every provider quirk now lives in one method on one class. The agent loop, the retry policy, the logging and the tracing get written once and never touched again when tool seven shows up.

Credentials resolve per call, never at import time

Here is the most common production bug in multi-tenant agent systems.

import os
 
SLACK_TOKEN = os.environ["SLACK_TOKEN"]   # do not do this

It works perfectly in development, where there is one tenant and it is you. In production every tenant's agent posts into whatever workspace that token belongs to. There is no crash. There is user A's data arriving in user B's channel, discovered by user B.

The credential is a function of the call, not of the process.

import threading, time
 
 
class CredentialResolver:
    """store.get returns a Credential carrying .secret and .expires_at."""
 
    def __init__(self, store, refreshers: dict):
        self.store = store                    # encrypted and audited, not a dict
        self.refreshers = refreshers          # provider -> callable
        self._locks: dict[tuple[str, str], threading.Lock] = {}
        self._guard = threading.Lock()
 
    def resolve(self, tenant_id: str, provider: str) -> str:
        cred = self.store.get(tenant_id, provider)
        if cred is None:
            raise MissingCredential(tenant_id, provider)
        if not self._expiring(cred):
            return cred.secret
 
        # Serialize refreshes. Two threads refreshing a rotating refresh token
        # invalidate each other, and the loser fails silently much later.
        with self._lock_for((tenant_id, provider)):
            cred = self.store.get(tenant_id, provider)   # another worker may have won
            if not self._expiring(cred):
                return cred.secret
            fresh = self.refreshers[provider](cred)
            self.store.put(tenant_id, provider, fresh)
            return fresh.secret
 
    def _lock_for(self, key):
        with self._guard:
            return self._locks.setdefault(key, threading.Lock())
 
    @staticmethod
    def _expiring(cred, skew: float = 60.0) -> bool:
        return cred.expires_at is not None and cred.expires_at - skew <= time.time()

The double check inside the lock stops a herd of simultaneous refreshes and lets another worker's refresh win. The sixty second skew refreshes before expiry rather than after a failed call. MissingCredential is explicit, because a missing credential must never quietly fall back to a default identity. And the secret never reaches a prompt, a log line or a trace attribute. Log the tenant, the provider and the credential ID.

Normalizing errors into six kinds

Once every adapter classifies into the same six kinds, retry policy is a lookup table instead of a per provider argument.

The split that pays for itself immediately is auth versus permission. Both look credential shaped and need opposite handling. An expired token is fixable by your system without bothering anyone. A missing OAuth scope needs a human to reconsent, and retrying it a hundred times burns quota for nothing.

Error messages are prompts, so write them for the model. Tell the agent "invalid arguments" and it reformats the channel ID and tries again. Tell it "that channel does not exist in this workspace" and it lists channels.

Retries that do not double charge your customer

With normalized errors, one retry wrapper covers every tool.

import random, time
 
 
def call_with_retry(tool: Tool, call: ToolCall, max_attempts: int = 3) -> ToolResult:
    for attempt in range(1, max_attempts + 1):
        result = tool.invoke(call)
        if result.ok:
            return result
 
        retryable = result.error.kind in (ErrorKind.RATE_LIMIT, ErrorKind.UNAVAILABLE)
        if tool.writes and not call.idempotency_key:
            retryable = False                 # a blind write retry is a duplicate
        if not retryable or attempt == max_attempts:
            return result
 
        delay = result.error.retry_after
        if delay is None:
            delay = min(2 ** attempt, 30) * random.uniform(0.5, 1.5)
        time.sleep(delay)

It honours Retry-After rather than guessing, because guessing shorter than the provider asked is how a soft limit becomes a hard ban. It jitters the backoff, because ten workers on a clean exponential schedule hit the provider in a synchronized wave forever. And it refuses to retry a write that cannot prove it is safe.

That last rule matters most. Retrying a read is free. If send_email times out after the provider already accepted the message, the naive retry sends it twice, and a timed out charge charges twice. The fix is an idempotency key bound to the logical action, generated once when the agent decides to act and reused on every attempt. Stripe takes it as an Idempotency-Key header and replays the original response. Many APIs do not, so keep your own durable record keyed on that value. An in-process dictionary does not count, because the retry that matters is the one after the pod restarted.

Versioning, or surviving somebody else's Tuesday

Every tool you integrate is a dependency that breaks on a schedule you do not control. Notion pins behaviour through a mandatory Notion-Version header. Stripe pins to the API version your account was created with and lets you override per request. Plenty of smaller vendors change a response field on a Tuesday and tell nobody.

Pin explicitly. Send the version header on every request, hardcoded in the adapter. An unpinned integration means the provider picks your deploy schedule.

Parse defensively. parse extracts the four fields you actually use into your own type. If a provider adds twelve fields, nothing happens. If it removes one you use, you get a clean error in one file instead of a KeyError buried in a prompt.

Contract test against the real thing. One nightly job per provider, hitting a sandbox with a real credential, asserting the shape your adapter expects. Highest value test in the tool layer, usually the last one written.

Then put the maintenance tax in the estimate. Twenty integrations, each shipping a breaking change roughly once a year, is a forced change every two or three weeks, forever, on somebody else's calendar.

Before you add tool number six

• Every tool returns the same result type and the agent never sees a raw provider payload
• Provider quirks live in exactly one adapter method
• The credential resolves per call from the tenant on the call, never captured at import
• Refresh is serialized per tenant and provider, and a missing credential raises instead of falling back
• Secrets never appear in prompts, arguments, logs or traces
• Errors classify into a fixed set of kinds, auth and permission stay apart, and messages tell the model what to do next
• Retries honour Retry-After, carry jitter, and never fire blind on a write
• Idempotency records survive a restart
• Pagination normalizes to one cursor field and API versions are pinned in every request
• A nightly contract test asserts each provider's response shape
• Every call is logged with tenant, actor, tool, duration, outcome and error kind

Eleven lines. Tick them all and tool twelve costs about what tool two did, which is the whole goal.

What a standard interface would have to solve

Everything above is a boundary layer you built yourself. Every team that gets past five tools builds one, they all build a slightly different one, and none of them compose. Your Slack adapter is worthless to the team down the hall, and their calendar tool is worthless to you.

The problem is not new and not specific to AI. Hardware answered with USB. Databases answered with ODBC. Editors had M editors times N languages and answered with LSP. Operating systems answered with POSIX. Every time the answer had the same shape: one contract at the boundary, many implementations behind it, wiring count falling from M times N to M plus N.

So what would that contract have to specify for AI agents? Based on everything that diverged above, at minimum:

• Discovery. An agent asks a provider what tools exist and gets back names, descriptions and schemas, with no human pasting definitions into a codebase.
• A uniform call envelope. One request shape and one result shape, so the agent loop is written once.
• Authentication as a separate concern. Caller identity travels with the call and tool logic never touches the secret.
• A shared error vocabulary. Enough structure to decide retry versus stop without knowing who answered.
• Transport independence. The same contract whether a tool runs beside the agent or across a network.
• Capability negotiation. Both sides declare what they support, so an old client and a new server do not break each other.

That is a specification, not a library. Written out as a list, it stops looking like a wish and starts looking like something somebody has probably already built.

Somebody has. That is the next post.