voicebot@v3

AI agent on the phone, no API key needed.

Versionvoicebot@v3
Runs ininbound-call and your own branches

Parameters

NameTypeRequiredDefaultDescription
voiceda-DK-Christel, da-DK-Jeppe, de-AT-Ingrid… (70 values) or stringSee Voices & Audio
accepts ${{ }}
languageaf-ZA, am-ET, ar-AE… (140 values)Language for Speech Recognition and Text-to-Speech. See Voices & Audio
accepts ${{ }}
engineobjectyesThe LLM that drives the conversation.
engine.name"OpenAI"yesAI provider. Only OpenAI is supported.
engine.authstringOptional: your own OpenAI API key, to have OpenAI bill the tokens on your account (any of its models then works). Without it, nothing to set up: Callr bills the tokens, and engine.model must be one of the OpenAI models listed in the AI section of your rates (Billing > Pricing). Required with url.
1..255 characters
engine.modelstringModel to use. Default: the default model of your account when it is an OpenAI model, else gpt-6.1-terra. voicebot only supports OpenAI: with an Anthropic or Mistral default model, gpt-6.1-terra answers. The -pro models are refused: too slow for a call. gpt-6-astra and gpt-6.1-sol need api: responses.
1..255 characters
engine.urlstringBase URL of an OpenAI-compatible API, instead of the OpenAI one. Requires auth, the API key of that endpoint (no API key is needed with OpenAI only). Most of them only have chat completions: set api: chat.
engine.apiresponses, chat"responses"OpenAI API used: responses (/v1/responses, the new API) or chat (/v1/chat/completions). Use chat with an OpenAI-compatible API (url) that has no responses API. With chat, a reasoning model calls the tools only without reasoning, and gpt-6-astra and gpt-6.1-sol do not work.
engine.paramsobjectGeneration settings sent to the model.
engine.params.temperaturenumberSampling temperature: lower is more predictable, higher is more creative. Not sent by default: the model chooses. A reasoning model refuses it while reasoning (reasoningEffort low and above, gpt-6-astra, gpt-6.1-sol).
0..2 · accepts ${{ }}
engine.params.maxCompletionTokensinteger0Maximum number of tokens of an answer, reasoning included; 0: no limit. With api: responses, 0 or at least 16.
0.. · accepts ${{ }}
engine.params.reasoningEffortnone, low, medium, high, xhighReasoning effort of a reasoning model. Each level adds latency before the bot speaks: none is the best for a call. By default none for gpt-5.5, gpt-5.6-*, gpt-6-luna, gpt-6-sol and gpt-6.1-terra, low for gpt-6-astra and gpt-6.1-sol (their minimum), not sent for the others. Refused by the gpt-4 models; only none with api: chat. While reasoning (low and above), temperature is refused by the model (not sent).
accepts ${{ }}
pleaseWaitobjectSentence said while the bot is preparing its answer (a slow model or a tool call), so that the caller does not wait in silence.
pleaseWait.enabledbooleanfalseaccepts ${{ }}
pleaseWait.delayBeforenumber4Seconds without an answer before a waiting sentence is said.
0..30 · accepts ${{ }}
pleaseWait.delayAfternumber2Seconds of silence after the waiting sentence, so that the answer does not follow it at once.
0..10 · accepts ${{ }}
pleaseWait.sentencesarray of stringWaiting sentences: one of them, picked at random, is said.
1..10 items · accepts ${{ }}
interruptionsobjectWhat the words of the caller do while the bot speaks. Whatever the mode, the model gets the conversation as the caller heard it: a sentence of the bot is in the history once played (cut to the words heard when interrupted), the words of the caller where they said them, and a sentence never heard is dropped.
interruptions.modeinterrupt, ignore, wait"interrupt"What happens when the caller talks while the bot speaks. See Interruptions
accepts ${{ }}
interruptions.minWordsinteger3The bot is cut off once the caller said more than this number of words (mode interrupt). Fewer words while the bot speaks are an acknowledgement ("yes", "ok"): not answered.
1..10 · accepts ${{ }}
initialMessagesarray of objectyesConversation the bot starts from: system (or developer) messages are its instructions, assistant messages are said to the caller when the action starts.
1..32 items · accepts ${{ }}
initialMessages[].rolesystem, developer, assistant, useryesAuthor of the message.
initialMessages[].contentstringyesText of the message.
1..65536 characters
toolsarray of objectHTTP endpoints the bot can call during the conversation (function calling). The response body is given to the model.
0..10 items
tools[].namestringyesName of the tool, as the model sees it.
1..32 characters
tools[].type"url"Type of tool. Only url (an HTTP call) is supported.
tools[].urlstringyesURL called when the model uses the tool.
accepts ${{ }}
tools[].methodGET, POST, PUT, DELETE, PATCH"GET"HTTP method. With GET, the parameters are sent as query arguments; otherwise as a JSON body.
tools[].headersobjectHTTP headers added to the request. Callr-RunID is always added.
tools[].descriptionstringyesWhat the tool does and when to use it: the model reads it to decide when to call the tool.
1..1024 characters
tools[].paramsarray of objectParameters of the tool, filled by the model.
0..10 items
tools[].params[].namestringyesName of the parameter.
1..32 characters
tools[].params[].typestring, number, boolean, object, array, nullyesJSON type of the parameter.
tools[].params[].descriptionstringyesWhat the parameter holds: the model reads it to fill the value.
1..1024 characters

Every parameter that accepts ${{ }} can be computed from an expression.

How it works

Voicebot is an AI agent on the phone: the words of the caller are recognized, the model answers, and its answer is spoken sentence by sentence.

voicebot@v3 gives the model the conversation as the caller heard it, with the right timing:

  • a sentence of the bot is in the history once it was played, cut to the words the caller heard when they interrupted it;
  • the words of the caller are placed where they said them;
  • a sentence the caller never heard is dropped.

When the caller talks over the very beginning of a sentence ("yes, go on"), the bot says it again from its beginning, unless the model answers instead. The result has a dated transcript.

Nothing to set up: without engine.auth, no API key is needed and Callr bills the tokens per model (see the AI section of your rates, Billing > Pricing). engine.model must then be one of the OpenAI models listed in the AI section of your rates (Billing > Pricing). Without engine.model, the default model of your account answers when it is an OpenAI one, else gpt-6.1-terra (voicebot only supports OpenAI). When the model asked is no longer available (deprecated by OpenAI), the closest model answers instead.

Your own OpenAI API key (optional): with engine.auth, OpenAI bills the tokens on your account, and any of its models works.

The model is called with the OpenAI Responses API (engine.api: responses, the default), which every recent model supports with tools. With an OpenAI-compatible API (engine.url) that only has chat completions, set engine.api: chat. The API sets the reasoning effort the model needs (none when it can, for the lowest latency).

You can also call external tools with URLs. The bot will call the URL with the parameters you specify. The URL should return a JSON object.

For a fast response, use a Flash voice, such as en-US-Olivia:MAI-Voice-2.1-Flash. See Voices & Audio.

Interruptions

interruptions.modeWhat happens when the caller talks while the bot speaks
interrupt (default)More than minWords words cut the bot off. Fewer words are an acknowledgement and the bot goes on. When the caller talks over the very beginning of a sentence, the bot says it again from its beginning, unless the model answers instead.
ignoreThe caller is not heard while the bot speaks (half-duplex), and is heard again once the bot is silent. Useful on a speakerphone, where the bot could hear itself.
waitThe bot is never cut off. What the caller said is answered once the bot is silent.

Migrate from voicebot@v2

voicebot@v2 still works and is unchanged. To move to voicebot@v3:

  • change the action name to voicebot@v3;
  • engine.auth and engine.model are now optional: keep engine.auth to have OpenAI bill the tokens on your account, or remove it to have Callr bill them (no API key needed);
  • without engine.model, the default model of your account answers when it is an OpenAI one, else gpt-6.1-terra.

Migrate from voicebot@v1

voicebot@v1 still works and is unchanged. To move to voicebot@v3:

  • change the action name to voicebot@v3;
  • interruptions replaces stopSpeaking: replace stopSpeaking.minWords with interruptions.minWords;
  • replace stopSpeaking.enabled: false with interruptions.mode: wait, the closest behavior;
  • rename pleaseWait.delay to pleaseWait.delayBefore, and set pleaseWait.delayAfter if needed;
  • engine.params.temperature is no longer sent by default (it was 0.2 in voicebot@v1): set it only if you need it;
  • with an OpenAI-compatible engine.url that only has chat completions, set engine.api: chat (the Responses API is used by default).

initialMessages, tools, voice, language, the built-in tools and the result work as in voicebot@v1; the result also has the transcript.

Examples

Half-duplex

The caller is not heard while the bot speaks (for example on a speakerphone, where the bot could hear itself).

- action: voicebot@v3
  params:
    initialMessages:
      - role: system
        content: You are a smart AI agent on the phone. Make short replies
      - role: assistant
        content: Hello there. How can I help?
    engine:
      name: OpenAI # no API key needed, the default model of your account
    interruptions:
      mode: ignore

Short example, no API key needed

- action: voicebot@v3
  params:
    initialMessages:
      - role: system
        content: You are a smart AI agent on the phone. Make short replies
      # all assistant initial messages will be said by the bot
      - role: assistant
        content: Hello there. How can I help?
    engine:
      name: OpenAI # the only supported engine for now
      # no auth: no API key needed, the tokens are billed by Callr
      # no model: the default model of your account

There are three built-in tools (functions)

  • dial(phone_number): to ask the bot to transfer the call to a human
  • hangup: to ask the bot to hang up the call
  • set_value(name, value): to store a value and retrieve it in the result

Here is an example use of built-in tools:

- action: voicebot@v3
  params:
    initialMessages:
      - role: system
        content: You are a smart AI agent on the phone. Make short replies
      - role: system
        content: >
          If the call wants to speak to a human, call the "dial" function with phone_number='+15559824200'.
          If the user says goodbye or if the conversation is over, call the hangup function.
      - role: assistant
        content: Hello there. How can I help?
    engine:
      name: OpenAI
      model: gpt-6.1-terra
  result: $bot

- if: $bot?.exitTool === 'dial'
  then:
    - action: dial@v2
      params:
        targets:
          - number: ${{ $bot?.payload }}
            timeout: 30

- if: $bot?.exitTool === 'hangup'
  then:
    - action: play@v3
      params:
        tts:
          text: Thank you, goodbye!
    - action: hangup@v1

Full example with tools

variables:
  $model: gpt-6.1-terra
  $url: "https://api.example.com"

branches:
  inbound-call:
    actions:
      - action: voicebot@v3
        params:
          initialMessages:
            - role: system
              content: You are an AI agent on the phone. Make short replies
            - role: system
              content: >
                If the call wants to speak to a human, call the "dial" function with phone_number='+15559824200'.
                If the user says goodbye or if the conversation is over, call the hangup function.
            - role: assistant
              content: Hello there. How can I help?
          engine:
            name: OpenAI
            model: ${{ $model }}
            auth: ${{ secrets.OPENAI_KEY }} # your own OpenAI key, optional
            params:
              maxCompletionTokens: 0 # default value
          pleaseWait:
            enabled: true
            delayBefore: 2.5
            delayAfter: 1
            sentences:
              - Please wait
          tools:
            - name: search_knowledge_base
              url: ${{ $url }}/search_knowledge_base
              description: >
                Search into the knowledge base (RAG). Do not call this function when
                looking for the next available slot.
              method: GET # GET is the default if you do not specify a method
              params: # parameters will be filled by the LLM
                # in case of a GET, they will be sent as query args
                - name: query
                  type: string
                  description: User query
            - name: get_next_available_slot
              url: ${{ $url }}/get_next_available_slot?from=${{ encodeURIComponent(call.fromNumber) }}
              description: >
                Returns the next available slot.
            - name: book_slot
              url: ${{ $url }}/book_slot
              description: >
                Books the slot.
              method: POST
              headers:
                # you can specify custom HTTP headers
                X-CallerID: ${{ call.fromNumber }}
                Authorization: Bearer ${{ secrets.API_TOKEN }}
              params:
                # those parameters will be sent as a JSON body (because method != GET)
                # and filled by the LLM
                - name: name
                  type: string
                  description: Firstname and lastname
                - name: object
                  type: string
                  description: Type of appointment
                - name: slot
                  type: string
                  description: ISO 8601 date-time "YYYY-MM-DDTHH:MM:SSZ"
        result: $bot

Result

Set result: $myVariable to keep it. Result of voicebot@v1, voicebot@v2 and voicebot@v3

FieldTypeDescription
okbooleantrue when the bot ended the conversation with a tool (exitTool)
timeoutbooleantrue when the conversation did not end in time
errorstringOnly present on error: hangup when the caller hung up, canceled, or the error message
exitTooldial, hangupBuiltin tool the bot called to end the conversation
payloadstringArgument of the exit tool: the phone number for dial, the reason for hangup
messagesarray of objectHistory of the conversation, as given to the model: as the caller heard it
valuesobjectValues saved by the bot with the builtin tool set_value(name, value)
totalTokensintegerNumber of tokens used
transcriptarray of objectvoicebot@v2 and v3. Everything said, dated: the sentences of the bot, played or not, and the words of the caller, answered or not
transcript[].tnumberSeconds since the start of the action (when it was said)
transcript[].roleassistant, user, toolWho said it
transcript[].textstringThe sentence, the words, or the tool called with its arguments
transcript[].statusstringSee the table below
transcript[].heardstringThe words of a cut sentence the caller heard
transcript[].replayOfintegerThe id of the cut sentence said again
transcript[].idintegerIdentifier of a sentence of the bot
statusMeaning
playedThe sentence was played
playingThe sentence was playing when the action ended
cutThe caller interrupted the sentence (heard is what they heard)
droppedThe sentence was never played
fillerA pleaseWait sentence, not in the history
turnThe words of the caller, answered by the model
backchannelA few words while the bot spoke, not answered
ignoredWords while the bot spoke, with interruptions.mode ignore
calledA tool called by the model
{
  "ok": true,
  "timeout": false,
  "exitTool": "hangup",
  "payload": "the caller has their appointment",
  "messages": [
    { "role": "assistant", "content": "Hello, how can I help you?" },
    { "role": "user", "content": "I'd like an appointment on Tuesday." },
    { "role": "assistant", "content": "Let me check." },
    { "role": "user", "content": "Yes, I'm listening." },
    { "role": "assistant", "content": "I can offer you 5pm." }
  ],
  "totalTokens": 2345,
  "transcript": [
    { "t": 0.02, "role": "assistant", "text": "Hello, how can I help you?", "status": "played", "id": 2 },
    { "t": 2.1, "role": "user", "text": "I'd like an appointment on Tuesday.", "status": "turn" },
    { "t": 4.5, "role": "assistant", "text": "Let me check.", "status": "played", "id": 4 },
    { "t": 5.3, "role": "assistant", "text": "I can offer you 5pm.", "status": "cut", "heard": "I", "id": 5 },
    { "t": 5.4, "role": "user", "text": "Yes, I'm listening.", "status": "turn" },
    { "t": 7.2, "role": "assistant", "text": "I can offer you 5pm.", "status": "played", "replayOf": 5, "id": 7 }
  ]
}

Older versions

Older versions keep working in existing scenarios. Use the version above in new scenarios.

  • voicebot@v2: AI agent, with the history of what was really said on the phone
  • voicebot@v1: AI agent. Replaced by voicebot@v2

Did this page help you?