Multi-Channel AI Agents ✨

Callr Actions now lets you build intelligent agents that work across multiple channels — voice calls, SMS, and APIs — using real-time speech recognition, LLMs like OpenAI, and seamless integrations with external systems.

These agents can:

  • 📞 Answer voice calls like a smart receptionist
  • 💬 Chat via SMS
  • 🔗 Integrate with any backend via fetch@v2
  • 🧠 Understand natural language, summarize intent, and automate follow-up

🧠 What is an AI Agent?

A Multi-Channel AI Agent is a flow powered by:

  • voicebot@v1: a conversational agent for phone calls
  • 🔌 external tools (calendars, CRMs, databases)
  • 📡 real-time follow-up via SMS, email, or HTTP
  • 🧾 optional post-call summarization with OpenAI-compatible models

You define the assistant's behavior, tone, and tools. The rest is handled by the AI.


✅ Example Capabilities

With just YAML and a few HTTP APIs, your AI Agent can:

Use CaseExample Action
📆 Check availabilityQuery a calendar API via fetch@v2
🗓️ Book appointmentsPost a booking using structured JSON schema
🤝 Transfer to a humanUse dial@v1 with a fallback number
💬 Summarize the conversationUse openai@v2 on $bot.messages
📩 Notify your teamSend an email@v1 report
📲 Follow up with the callerSend a sms@v1 confirmation
📁 Enrich leads in your CRMPush metadata using fetch@v2 to your backend

🧪 Example Scenario: Real Estate AI Agent

This sample AI agent answers missed calls for a real estate agency. It can:

  • Qualify the caller's intent (buy/sell/inquire)
  • Capture property info (location, type, price)
  • Offer calendar availability via cal.com
  • Book appointments directly
  • Transfer to a human when requested
  • Send SMS to the caller and email to the team
description: >
  An smart voicemail capable of qualifying a lead
  and scheduling an appointment (cal.com).

variables:
  $cal: # cal.com
    auth: <cal.com-auth-token>
    username: <cal.com-username>
    eventTypeSlug: <cal.com-event-type-slug>
    eventTypeId: <cal.com-event-type-id>
    timeZone: Europe/Paris

  $openai:
    key: <your-openai-key>
    model: gpt-4o

  # voicebot prompt
  $messages:
    - role: developer
      content: >
        You are the intelligent answering machine for the TOP REAL ESTATE agency,
        located at 128 rue La Boétie, 75008, Paris.

        The agency's hours are:
          - Monday to Friday from 9am to 5pm
          - Saturday from 10am to 4pm
          - excluding public holidays

        The agency manager is Jean-Michel Mal-Logé. The staff are:
        - Sarah Pelle
        - Sarah Croche

        Your goal is to qualify the call. For example, the caller might be calling to:
        - Sell a property
        - Buy a property
        - Follow up on an ongoing file
        - Schedule an appointment
        - Inquire about the agency's hours

        If they call regarding a property or to schedule an appointment, ask questions to
        qualify their request and the property: location, area, desired price, etc.
        You must qualify their request as precisely as possible.

        If you are asked to call back, say that you will forward the information to the agency.
        If an appointment is requested, you can call the function "get_calendar_slot" to check
        availability. When listing available times to the caller, provide a maximum of 3 options.

        The booking is arranged over the phone with you. If the appointment is confirmed,
        tell them that they will receive a confirmation SMS.

        - Make sure to be humorous and witty!
        - Ensure that all your responses are short and simple. Use colloquial language.
        - This is a spoken conversation. Do not reply in markdown.
        - Do not use emojis (due to voice synthesis).

        You have the following functions (tools) available:

        - dial: to transfer the call to a human. phone_number = '+15559820800'. You can use it at the caller's request.
        - hangup: when the conversation is over, you can hang up.
        - get_calendar_slot: checks availability in the calendar.
    - role: developer
      content: >
        Today is ${{ now.toISOString() }}.
        Caller number: ${{ call.fromNumber }}.
    # all messages with role "assistant" will be spoken by the bot at the start of the call
    - role: assistant
      content: >
        Hello, I am Maria, the virtual assistant of TOP REAL ESTATE agency.
        How can I help you?

  # to extract information at the end of the call
  $ai: >
    Can you reply with a JSON object containing the following properties:
    {
      "short": "summary of the conversation in a few words",
      "long": "summary of the conversation in a few sentences",
      "booked": bool (true/false if an appointment is booked),
      "booked_date": "ISO 8601 if an appointment is booked",
      "sms": "(compose an SMS to send to the caller)"
    }

  $emails:
    - [email protected]

defaults:
  language: en-US

branches:
  inbound-call:
    actions:
      - action: voicebot@v1
        params:
          pleaseWait:
            # if the bot takes more than 2s (configurable) to respond,
            # the following phrases will be spoken
            enabled: true
            delay: 2
            sentences:
              - please hold on
              - one moment please
              - okay, one moment please
          initialMessages: ${{ $messages }}
          engine:
            name: OpenAI
            model: ${{ $openai.model }}
            auth: ${{ $openai.key }}
            params:
              maxCompletionTokens: 0 # default
              temperature: 0.2 # default
            # url: https://openai.compatible.endpoint

          # the tools defined here are HTTP endpoints that will be called directly by the bot
          tools:
            - name: get_calendar_slot
              url: https://api.cal.com/v2/slots?username=${{ $cal.username }}&eventTypeSlug=${{ $cal.eventTypeSlug }}&timeZone=${{ $cal.timeZone }}
              description: Returns a list of available slots
              method: GET
              headers:
                Authorization: Bearer ${{ $cal.auth }}
                cal-api-version: 2024-09-04
              # the parameters are automatically filled by the bot
              params:
                - name: start
                  type: string
                  description: ISO8601 date-time (start).
                - name: end
                  type: string
                  description: ISO8601 date-time (end).

            - name: book_slot
              url: https://api.cal.com/v2/bookings
              description: Books a slot for an appointment
              headers:
                Authorization: Bearer ${{ $cal.auth }}
                cal-api-version: 2024-08-13
              method: POST
              params:
                # each parameter must be defined with a JSON SCHEMA
                # https://platform.openai.com/docs/guides/function-calling?api-mode=chat#defining-functions
                # https://json-schema.org/
                - name: attendee
                  type: object
                  description: caller information
                  additionalProperties: false
                  required:
                    - phoneNumber
                    - name
                    - timeZone
                    - email
                  properties:
                    phoneNumber:
                      type: string
                      description: Phone number in E.164
                    name:
                      type: string
                      description: Caller name. Use phone number if empty.
                    timeZone:
                      type: string
                      description: IANA timezone, use ${{ $cal.timeZone }}
                    email:
                      type: string
                      description: use noreply+(phoneNumber)@example.com
                - name: start
                  type: string
                  description: Slot ISO8601 date-time start
                - name: eventTypeId
                  type: number
                  description: Use ${{ $cal.eventTypeId }}
        result: $bot

      # if the bot wishes to call the builtin tool "dial",
      # the number to call is in $bot.payload
      - if: $bot?.exitTool === 'dial' && $bot?.payload.match(/^\+/)
        then:
          - action: dial@v1
            params:
              targets:
                - number: ${{ $bot?.payload }}
                  timeout: 30

      # if the bot wishes to hang up
      - if: $bot?.exitTool === 'hangup'
        then:
          - action: say@v2
            params:
              what: Goodbye, see you soon
          - action: hangup@v1

  hangup:
    actions:
      # the conversation history (in OpenAI format)
      # is in $bot.messages
      - if: "!$bot?.messages?.length"
        goto:
          branch: exit

      # we add a message to extract information
      # using $ai
      - action: javascript@v1
        params:
          code: |
            const $messages = $bot.messages
            $messages.push({
              role: 'developer',
              content: $ai
            })
            return {
              $messages
            }

      # we call OpenAI in one-shot by passing the entire history
      # plus the $ai prompt added above
      - action: openai@v2
        params:
          secret: ${{ $openai.key }}
          endpoint: chat/completions
          timeout: 60
          properties:
            model: ${{ $openai.model }}
            messages: ${{ $messages }}
            response_format:
              type: json_object
        result: $openai

      # we parse the JSON response and create 3 new variables
      - action: javascript@v1
        params:
          code: |
            const res = JSON.parse($openai.choices[0].message.content)
            return {
              $sms: res.sms,
              $short: res.short,
              $long: res.long
            }

      # we send a summary via email
      - if: $short && $long
        then:
          - action: email@v1
            params:
              subject: Missed call from ${{ call.fromNumber }} - ${{ $short }}
              text: |
                Missed call from ${{ call.fromNumber }}

                ${{ $long }}

                Regards.
              to: ${{ $emails }}

      # if the caller is on a mobile phone, also send them a summary
      - if: $sms && call.fromType.match(/MOBILE/)
        then:
          - action: sms@v1
            params:
              to:
                - ${{ call.fromNumber }}
              body: ${{ $sms }}

🔌 External Tools via voicebot@v1

Your AI Agent can call external services using HTTP. The AI will fill the params automatically.

tools:
  - name: get_calendar_slot
    method: GET
    url: https://api.example.com/calendar/slots
    headers:
      Authorization: Bearer ${{ $calendar.auth }}
    params:
      - name: start
        type: string
        description: ISO8601 date
      - name: end
        type: string
        description: ISO8601 date

🧠

You define which tools are available, and the AI learns how to use them.


🧾 Post-call Summarization

After the call ends, you can process the full $bot.messages history with openai@v2 to extract:

  • Short + long summaries
  • Appointment details
  • Lead qualification info
  • A custom SMS to send back to the caller

You can send this data to your team or backend in real time via:

  • email@v1
  • sms@v1
  • fetch@v2

💼 Industry Use Cases

IndustryWhat Your Agent Can Do
🏠 Real EstateQualify buyers/sellers, book showings, push to CRM
🏥 HealthcareIntake patient intent, suggest time slots, follow up by SMS
⚖️ LegalDetect case type, triage leads, forward urgent calls
📞 SupportHandle missed calls, collect issue summaries, route tickets
📆 ServicesOffer appointment times, take bookings, auto-confirm

📣 What Makes It Multi-Channel?

  • ✅ Voice over phone (inbound or API-triggered)
  • ✅ Outbound calls with TTS + AI summarization
  • ✅ SMS notifications to caller or staff
  • ✅ Emails for internal reporting
  • ✅ Backend API integrations


🛠️ Requirements

To start using Multi-Channel AI Agents, you’ll need:

  • A Callr account with access to Callr Actions
  • An OpenAI-compatible API key (for voicebot@v1)
  • Optionally: access to your external systems via API