Multi-Channel AI Agents

Callr Actions now lets you build intelligent agents that work across multiple channels — voice calls, SMS, and APIs — using real-time speech recognition, LLMs like OpenAI, and seamless integrations with external systems.

These agents can:

  • 📞 Answer voice calls like a smart receptionist
  • 💬 Chat via SMS
  • 🔗 Integrate with any backend via fetch@v3
  • 🧠 Understand natural language, summarize intent, and automate follow-up

🧠 What is an AI Agent?

A Multi-Channel AI Agent is a flow powered by:

  • voicebot@v3: a conversational agent for phone calls, no API key needed
  • 🔌 external tools (calendars, CRMs, databases)
  • 📡 real-time follow-up via SMS, email, or HTTP
  • 🧾 optional post-call summarization with llm@v1, an AI model with no API key needed

You define the assistant's behavior, tone, and tools. The rest is handled by the AI.


✅ Example Capabilities

With just YAML and a few HTTP APIs, your AI Agent can:

Use CaseExample Action
📆 Check availabilityQuery a calendar API via fetch@v3
🗓️ Book appointmentsPost a booking using structured JSON schema
🤝 Transfer to a humanUse dial@v2 with a fallback number
💬 Summarize the conversationUse llm@v1 on $bot.messages
📩 Notify your teamSend an email@v1 report
📲 Follow up with the callerSend a sms@v1 confirmation
📁 Enrich leads in your CRMPush metadata using fetch@v3 to your backend

🧪 Example Scenario: Real Estate AI Agent

This sample AI agent answers missed calls for a real estate agency. It can:

  • Qualify the caller's intent (buy/sell/inquire)
  • Capture property info (location, type, price)
  • Offer calendar availability via cal.com
  • Book appointments directly
  • Transfer to a human when requested
  • Send SMS to the caller and email to the team
description: >
  A smart voicemail capable of qualifying a lead
  and scheduling an appointment (cal.com).

compat:
  version: "2025-09-01"

variables:
  # cal.com username, event type and time zone
  # (the API key is the secret CALCOM_API_KEY of your account)
  $cal:
    username: <cal.com-username>
    eventTypeSlug: <cal.com-event-type-slug>
    eventTypeId: <cal.com-event-type-id>
    timeZone: Europe/Paris

  # voicebot prompt
  $messages:
    - role: developer
      content: >
        You are the intelligent answering machine for the TOP REAL ESTATE agency,
        located at 128 rue La Boétie, 75008, Paris.

        The agency's hours are:
          - Monday to Friday from 9am to 5pm
          - Saturday from 10am to 4pm
          - excluding public holidays

        The agency manager is Jean-Michel Mal-Logé. The staff are:
        - Sarah Pelle
        - Sarah Croche

        Your goal is to qualify the call. For example, the caller might be calling to:
        - Sell a property
        - Buy a property
        - Follow up on an ongoing file
        - Schedule an appointment
        - Inquire about the agency's hours

        If they call regarding a property or to schedule an appointment, ask questions to
        qualify their request and the property: location, area, desired price, etc.
        You must qualify their request as precisely as possible.

        If you are asked to call back, say that you will forward the information to the agency.
        If an appointment is requested, you can call the function "get_calendar_slot" to check
        availability. When listing available times to the caller, provide a maximum of 3 options.

        The booking is arranged over the phone with you. Once the caller confirms a slot,
        book it with "book_slot" and tell them that they will receive a confirmation.

        - Make sure to be humorous and witty!
        - Ensure that all your responses are short and simple. Use colloquial language.
        - This is a spoken conversation. Do not reply in markdown.
        - Do not use emojis (due to voice synthesis).

        You have the following functions (tools) available:

        - dial: to transfer the call to a human. phone_number = '+15559820800'. You can use it at the caller's request.
        - hangup: when the conversation is over, you can hang up.
        - get_calendar_slot: checks availability in the calendar.
        - book_slot: books the slot the caller has chosen.
    - role: developer
      content: >
        Today is ${{ now.toISOString() }}.
        Caller number: ${{ call.fromNumber }}.
    # all messages with role "assistant" will be spoken by the bot at the start of the call
    - role: assistant
      content: >
        Hello, I am Maria, the virtual assistant of TOP REAL ESTATE agency.
        How can I help you?

  # to extract information at the end of the call
  # (the shape of the answer is the jsonSchema of the llm@v1 action below)
  $ai: >
    Here is the conversation of a call handled by the AI assistant.
    Summarize it and extract the requested information.

  # set at the end of the call (see the hangup branch)
  $aiResult: null
  $short: null
  $long: null
  $sms: null

  $emails:
    - [email protected]

defaults:
  language: en-US

branches:
  inbound-call:
    actions:
      - action: voicebot@v3
        params:
          pleaseWait:
            # if the bot takes more than 2s (configurable) to respond,
            # the following phrases will be spoken
            enabled: true
            delayBefore: 2
            sentences:
              - please hold on
              - one moment please
              - okay, one moment please
          initialMessages: ${{ $messages }}
          engine:
            name: OpenAI
            # no API key needed: Callr bills the tokens per model.
            # No model: the default model of your account when it is an OpenAI one
            # (else gpt-6.1-terra). Your own OpenAI API key is optional:
            # auth: ${{ secrets.OPENAI_KEY }}
            # An OpenAI-compatible endpoint needs its own key (auth) and its url:
            # url: https://openai.compatible.endpoint
            # api: chat # when it has no Responses API
            params:
              maxCompletionTokens: 0 # default

          # the tools defined here are HTTP endpoints that will be called directly by the bot
          tools:
            - name: get_calendar_slot
              url: https://api.cal.com/v2/slots?username=${{ $cal.username }}&eventTypeSlug=${{ $cal.eventTypeSlug }}&timeZone=${{ $cal.timeZone }}
              description: Returns a list of available slots
              method: GET
              headers:
                Authorization: Bearer ${{ secrets.CALCOM_API_KEY }}
                cal-api-version: "2024-09-04"
              # the parameters are automatically filled by the bot
              params:
                - name: start
                  type: string
                  description: ISO8601 date-time (start).
                - name: end
                  type: string
                  description: ISO8601 date-time (end).

            - name: book_slot
              url: https://api.cal.com/v2/bookings
              description: Books a slot for an appointment
              headers:
                Authorization: Bearer ${{ secrets.CALCOM_API_KEY }}
                cal-api-version: "2024-08-13"
              method: POST
              params:
                # each parameter must be defined with a JSON SCHEMA
                # https://platform.openai.com/docs/guides/function-calling?api-mode=chat#defining-functions
                # https://json-schema.org/
                - name: attendee
                  type: object
                  description: caller information
                  additionalProperties: false
                  required:
                    - phoneNumber
                    - name
                    - timeZone
                    - email
                  properties:
                    phoneNumber:
                      type: string
                      description: Phone number in E.164
                    name:
                      type: string
                      description: Caller name. Use phone number if empty.
                    timeZone:
                      type: string
                      description: IANA timezone, use ${{ $cal.timeZone }}
                    email:
                      type: string
                      description: use noreply+(phoneNumber)@example.com
                - name: start
                  type: string
                  description: Slot ISO8601 date-time start
                - name: eventTypeId
                  type: number
                  description: Use ${{ $cal.eventTypeId }}
        result: $bot

      # if the bot wishes to call the builtin tool "dial",
      # the number to call is in $bot.payload
      - if: $bot?.exitTool === 'dial' && $bot?.payload.match(/^\+/)
        then:
          - action: dial@v2
            params:
              targets:
                - number: ${{ $bot?.payload }}
                  timeout: 30

      # if the bot wishes to hang up
      - if: $bot?.exitTool === 'hangup'
        then:
          - action: play@v3
            params:
              tts:
                text: Goodbye, see you soon
          - action: hangup@v1

  hangup:
    actions:
      # the conversation history (in OpenAI format)
      # is in $bot.messages
      # nothing to summarize if the voicebot did not run (no history)
      - if: $bot?.messages?.length
        then:
          # we write the whole history as one text message for the AI
          - action: javascript@v1
            params:
              code: |
                const lines = $bot.messages.map(m => {
                  const calls = (m.tool_calls ?? []).map(c => 'calls ' + c.function.name + ' ' + c.function.arguments)
                  return m.role + ': ' + [m.content, ...calls].filter(Boolean).join(' ')
                })
                const $messages = [{ role: 'user', content: lines.join('\n') }]
                return {
                  $messages
                }

          # we call the AI in one-shot: the history is the message, $ai the
          # instructions, and the jsonSchema is the shape of the answer
          - action: llm@v1
            params:
              system: ${{ $ai }}
              messages: ${{ $messages }}
              jsonSchema:
                type: object
                required: [short, long, booked, booked_date, sms]
                additionalProperties: false
                properties:
                  short:
                    type: string
                    description: summary of the conversation in a few words
                  long:
                    type: string
                    description: summary of the conversation in a few sentences
                  booked:
                    type: boolean
                    description: true if an appointment is booked, false otherwise
                  booked_date:
                    type: [string, "null"]
                    description: ISO 8601 date and time of the appointment if one is booked, else null
                  sms:
                    type: string
                    description: an SMS for the caller, with the appointment date and time if one is booked
              timeout: 60
            result: $aiResult

          # we read the JSON answer (already parsed), with a fallback if the AI failed
          - action: javascript@v1
            params:
              code: |
                const res = ($aiResult?.ok && $aiResult?.json) || {}
                return {
                  $short: res.short || 'no summary',
                  $long: res.long || 'The AI summary is not available. See the call in your Callr logs.',
                  $sms: res.sms || null
                }

          # we send a summary via email
          - action: email@v1
            params:
              subject: Missed call from ${{ call.fromNumber }} - ${{ $short }}
              text: |
                Missed call from ${{ call.fromNumber }}

                ${{ $long }}

                Regards.
              to: ${{ $emails }}

          # if the caller is on a mobile phone, also send them a summary
          - if: $sms && call.fromType.match(/MOBILE/)
            then:
              - action: sms@v1
                params:
                  to:
                    - ${{ call.fromNumber }}
                  body: ${{ $sms }}
                  nature: ALERTING

🔌 External Tools via voicebot@v3

Your AI Agent can call external services using HTTP. The AI will fill the params automatically.

tools:
  - name: get_calendar_slot
    method: GET
    url: https://api.example.com/calendar/slots
    headers:
      Authorization: Bearer ${{ secrets.CALENDAR_API_KEY }}
    params:
      - name: start
        type: string
        description: ISO8601 date
      - name: end
        type: string
        description: ISO8601 date
🧠

You define which tools are available, and the AI learns how to use them.


🧾 Post-call Summarization

After the call ends, you can process the full $bot.messages history with llm@v1 (no API key needed, structured output with jsonSchema) to extract:

  • Short + long summaries
  • Appointment details
  • Lead qualification info
  • A custom SMS to send back to the caller

You can send this data to your team or backend in real time via:

  • email@v1
  • sms@v1
  • fetch@v3

💼 Industry Use Cases

IndustryWhat Your Agent Can Do
🏠 Real EstateQualify buyers/sellers, book showings, push to CRM
🏥 HealthcareIntake patient intent, suggest time slots, follow up by SMS
⚖️ LegalDetect case type, triage leads, forward urgent calls
📞 SupportHandle missed calls, collect issue summaries, route tickets
📆 ServicesOffer appointment times, take bookings, auto-confirm

📣 What Makes It Multi-Channel?

  • ✅ Voice over phone (inbound or API-triggered)
  • ✅ Outbound calls with TTS + AI summarization
  • ✅ SMS notifications to caller or staff
  • ✅ Emails for internal reporting
  • ✅ Backend API integrations


🛠️ Requirements

To start using Multi-Channel AI Agents, you’ll need:

  • A Callr account with access to Callr Actions
  • No API key needed for the AI: voicebot@v3 and llm@v1 run without one, and Callr bills the tokens per model (the AI section of your rates, Billing > Pricing in the Console). Your own OpenAI API key is optional
  • A secret for each key of your other services, such as CALCOM_API_KEY for cal.com in the example above (Secrets button of the Actions page)
  • Optionally: access to your external systems via API

Did this page help you?