Custom AI

Open in the app: AI Assistants

Introduction

Custom AI lets you connect Bread Crumbs Bot to your own AI endpoint. When a message is processed by an AI with type: custom, Bot sends a request to your endpoint and uses your response as the assistant reply.

Custom AI supports text, audio, and document messages. Each message in the request includes a type field. For media messages, content is a public URL to the uploaded file. Your endpoint can return text or media replies the same way.

Communication model

AI assistant integration uses request/response.

• Bread Crumbs sends a request with the conversation context/input
• Your AI endpoint processes it
• Your AI returns a response payload directly

This model is synchronous from Bread Crumbs point of view: one request in, one AI response back.

AI type

Use custom in AI creation operation.

Integration properties

Message types

Each item in messages[] includes:

For inbound channel media, Bread Crumbs downloads the file and stores a URL in content. Captions and filenames may arrive as separate type: text messages.

Conversation history sent to your endpoint includes recent text, audio, and document messages (up to five), each with its type.

Special keywords

In case the agent returns a text response containing one of the following keywords, Bot will handle them specially:

• #HANDOVER#: Handover the conversation to an agent.
• #END#: End the conversation.

You can also control this behavior with a structured command field in your custom AI response:

• command: "HANDOVER"
• command: "END"

For audio and document replies, use the command field instead of embedding #END# / #HANDOVER# in the media URL.

How it works

When a message is processed by an AI with type: custom, Bot sends a request to your endpoint and uses your response as the assistant reply. So it will wait for your endpoint to process the message with AI and return the response. In case your endpoint returns a text response with special keywords (or returns a command), Bot will handle them specially as described in the Special keywords section and will remove markers from the response text. So you can return something like this:

{
  "text": "I will forward this conversation to a live agent. Please wait a while...",
  "command": "HANDOVER"
}

Request payload example

Bot sends the message to your custom AI endpoint as POST <integrationProperties.endpoint> with the following JSON payload example:

{
  "externalClientId": "905301749170",
  "clientId": "6728c77dbba874b293e60000",
  "conversationId": "6728c86bf893860f458e0000",
  "workspaceId": 12,
  "integrationId": "6743884f784839ad14770000",
  "messages": [
    {
      "role": "user",
      "type": "text",
      "content": "Hello"
    },
    {
      "role": "assistant",
      "type": "text",
      "content": "Hello John! How can I help you today?"
    },
    {
      "role": "user",
      "type": "document",
      "content": "https://storage.example.com/whatsapp/ext-client/invoice.pdf"
    },
    {
      "role": "user",
      "type": "text",
      "content": "Please review this invoice"
    },
    {
      "role": "user",
      "type": "audio",
      "content": "https://storage.example.com/whatsapp/ext-client/voice-note.mp3"
    }
  ]
}

Your endpoint should fetch or process media URLs when type is audio or document (for example transcription, summarization, or document parsing).

Authorization header

We are using the API key to authorize the request to your endpoint. And to prevent unauthorized access to your endpoint if someone else tries to use it. The bot will send the API key as x-api-key header, you will need to validate it in your endpoint to ensure that the request is coming from the bot.

x-api-key: <integrationProperties.apiKey>

Expected response from your endpoint

Your endpoint should return JSON with reply content in text and/or content. If type is omitted, Bread Crumbs treats the reply as text.

Supported reply types:

Text reply example

{
  "text": "Hello! How can I help you today?",
  "type": "text",
  "command": "END",
  "usage": {
    "inputTokens": 42,
    "outputTokens": 18,
    "totalTokens": 60,
    "reasoningTokens": 0,
    "cachedInputTokens": 0
  },
  "traceId": "trace_123",
  "runId": "run_123",
  "finishReason": "stop"
}

Document reply example

{
  "type": "document",
  "content": "https://your-cdn.example.com/generated/quote.pdf"
}

Audio reply example with handover

{
  "type": "audio",
  "content": "https://your-cdn.example.com/generated/reply.mp3",
  "command": "HANDOVER"
}

Media replies must use URLs that supported channels can fetch (same model as inbound media). Bread Crumbs delivers the file to the customer on WhatsApp and other connected channels using the URL you return.

Create custom AI (GraphQL)

mutation createAi($input: AICreateInput!) {
  createAi(input: $input) {
    id
    workspaceId
    name
    type
    integrationProperties
    createdAt
  }
}
 
input AICreateInput {
  workspaceId: Int!
  integrationProperties: JSON!
  name: String!
  type: AIType!
}

Next is an example of the input for the createAi mutation in JSON format:

{
  "input": {
    "workspaceId": 12,
    "name": "My Custom AI",
    "type": "custom",
    "integrationProperties": {
      "endpoint": "https://your-ai.example.com/generate",
      "apiKey": "your-custom-ai-key"
    }
  }
}

Related pages in the app

• AI Assistants
• Channels