Improve Your Customer Workflow With AI: How To Build a Zendesk Chatbot using ReAct

In this step-by-step guide you’ll learn how to set up a custom AI-powered Zendesk chatbot to improve your customer service and sales CRM.

Zendesk's support platform is a popular solution that enables you to offer an excellent customer service experience. You can further enhance your customer support experience by deploying AI-powered custom chatbots that can answer complex queries and help users through workflows. This is a game changer, it can empower your customer and sales representatives or even directly interact with customer requests. We prepared this article to help you learn how to integrate modern artificial intelligence directly into Zendesk, and what Zendesk-integrated chatbots can do for your business.

Why Use Custom Chatbots With Zendesk

Zendesk’s Ai chatbots tend to be simplistic, like copy+pasting every question to ChatGPT. We don’t want that. The key problem with Zendesk's AI system is that it's a managed service that allows for little customization specific to your use case. It doesn’t naturally allow for customization of the prompting framework, data management system, and embedded vector generation pipeline. What you get is what you get. This is pretty common with managed service chatbots as they want to make it as easy as possible to get rocking and rolling. This lack of customization means it's nearly impossible to move the system to a production system that you fully control and adjust for specific use cases.

‍

If you need something more sophisticated, like multi-agent systems with advanced prompting that interact with other tools, you will need to implement a custom framework. Below we’ll take a look at exactly that - so let’s dive right in!

Our Custom Ai Chatbot Framework

In this section, we explain our field-tested framework for integrating a custom LLM based AI chatbot system into Zendesk. Our system supports the following capabilities out of the box:

• Use reasoning and dynamic data fetching to answer complex business questions from your customers and internal stakeholders (e.g. sales executives and customer service agents).
• Use retrieval augmented generation (RAG) to answer questions based on a set of documents of your choosing - e.g. product information, knowledge bases and standard operating procedures (SOPs).
• Include real-time dynamic data like product prices and availability in the answers - (these data can be fetched by the AI system directly from your business systems).
• Easy integration to external tools outside of Zendesk to enhance the chatbot workflow. Most managed service chatbot software systems built inside a specific product have limited connections to tools not serviced by the specific product. This custom framework allows you to leverage the API of multiple channels for a chatbot that connects more of your business.

‍

The architecture that enables all these is shown below:

You might be thinking, why not use the OpenAI Assistants API? Turns out, as of January 2024, the Assistants API has some critical drawbacks that make it unsuitable for production.

• ❌ No support external API function calling. This is critical for document search and data retrieval pipelines that rely on your custom business logic.
• ❌ Higher latencies and much more expensive than regular GPT-4 API calls. This is a usability drawback for our customer-facing interactive chatbots.
• ❌ Retrieved content is less accurate than what we can achieve using state-of-the-art methods with modern RAG packages (e.g. LlamaIndex). You have no ability to optimize the lookup model.
• ❌ No support for fine-tuned models. What you get is what you get.

Below we show you how to set-up a custom architecture to overcome these limitations.

Tool Using LLMs with Problem-Solving Skills

To answer a complex question, an intelligent system must be able to (i) break it up into a series of smaller information gathering steps, (ii) interact with external systems to gather the relevant information from disparate sources, and finally (iii) combine the information into a coherent response that directly addresses the question.

In summary our LLM application must be able to:

• ✅ Reason about what information is required to answer a question.
• ✅ Find this information across multiple sources like knowledge bases, SOPs, frequently asked questions (FAQs), manuals, discussions, and datasheets.
• ✅ Seamlessly fetch dynamic information (e.g. latest pricing) from your business systems along with the customer questions or support tickets from Zendesk.
• ✅ Combine all that information into a coherent, accurate answer.

To achieve this, below we show you how to equip GPT-4 (or any LLM of your choice) with ReAct. ReAct, short for Reasoning+Action, is a framework used to give LLMs better reasoning skills along with an ability to perform custom actions like using external APIs.

The full list of available actions is domain-specific and supplied via few-shot examples to the LLM prompt. The LLM agent then picks the actions that are most helpful for answering the question, and executes those actions. The results of the actions are provided back to the LLM agent as observations. Equipped with those data, the agent can either continue its reasoning process, seeking more relevant information, or terminate and provide a response. Some examples of a ReAct system in action are shown below:

Using ReAct we can design an application for your customers to seamlessly interact with your business tools. It’s a win-win, it saves them time and hassle, and it saves the time of your most knowledgeable employees.

Below we take a look at how to build a ReAct-based system that can handle complex questions & more challenging tasks unsuited for a vanilla GPT. In the next section we explain how to integrate your ReAct system with Zendesk for a natural customer relationship management workflow.

ReAct: An Agent That Understands Your Business

Here is what we have learned about designing an effective ReAct Agent.

• It starts by executing a high-level product search or info search in the Zendesk app.
• It then uses reasoning to explore product features against the user's criteria.
• It eliminates any products that don't match some of the criteria.
• It verifies if the remaining products satisfy each of the other criteria.
• It hones in on the product that best satisfies all the criteria.

There are two critical components to this system. First, to find information relevant to a query in static documents — like FAQs, product descriptions, customer emails, or discussion forum posts — we implement a ReAct action named  "search(query)." It does the following operations:

• Chunking: Documents are split into chunks of text so that information can be retrieved in chunks. The splitting is done on meaningful semantic boundaries like headers, sections, posts, or comments.
• Document embeddings: An embedding generator produces an embedding for each chunk. Our framework defaults to a SentenceTransformer model tuned for asymmetric search since we find that it works well for most use cases. However, the framework can also be configured to use OpenAI's Embeddings API or any alternative embedding API.
• Vector database storage: The embeddings are stored in the configured vector database like Qdrant, Weaviate, Pinecone, or other.
• Semantic search: When a customer or internal query is received, it looks for chunks that are relevant to the query. It first converts the query to a query embedding using the embedding generator. It then searches the vector database for chunk embeddings that are nearest neighbors of the query embedding. Such chunks are likely to be the most relevant to the query and provided to the framework for further processing.
• Advanced systems can use a natural language query generation step to generate new queries that perform a better search based on the entire conversation.

This framework also supports adding new documents or updating existing ones. After such changes, the document embeddings are recalculated and updated in the vector database. Once set up, managing the knowledge database your system has access to is intuitive.

As mentioned, retrieving static information from databases is just one part of the final answer though. The real value and productivity gains of our custom chatbots lie in their ability to weave in real-time dynamic information fetched from other business systems like Zendesk or ERP systems. That is the topic of the next section.

Zendesk Integration: How LLMs Can Interact With APIs

Some information can be uploaded to your database and not modified (e.g. custom documents, SOPs). However, to build a production ready system, we need our LLM to be albe to obtain real-time ticketing and customer relationship management data from Zendesk (as well as pricing and other dynamically changing data streams). Notice how in the diagram below, the Chatbot has an ability to interact with multiple data sources (on the right).

Using ReAct, we can give an LLM the ability to use these retrieval APIs:

• Ticket information: You can access conversational history, call audio recordings, and similar information related to each ticket using the Tickets API endpoint.
• Customer details: To obtain details about your costumes, use the Users API endpoint.
• Sales CRM information: Use the Sell APIs to access details about contacts, leads, and deals.
• Product catalog: Use the Products API to maintain a product catalog within Zendesk itself instead of using a separate ERP system.

Implementing ReAct Tools With GPT-4

We can implement ReAct actions using GPT-4 function-calling. One advantage of the GPT-4 API is that it provides a structured way to specify actions compared to ReAct's original proposals. Note however, that this is also possible with any other LLM (like Llama 2 or Anthropic), but falls outside the scope of this article - feel free to reach out if you want to learn more.

OpenAI's API enables you to specify actions and their information in great detail as shown below:

GPT-4 uses that semantic information and metadata — the function description, parameter property descriptions, parameter units, and so on — to make informed decisions on when to invoke your actions based on your queries. GPT-4 also enables the structured invocation of actions and transmission of action responses back to the LLM.

LLM Answers to Queries

The framework combines all the retrieved document chunks and the relevant dynamic data into a coherent answer by appropriately prompting an LLM. All that remains now is to get the user query via Zendesk and show the answer to the customer or internal stakeholder via Zendesk's front end. That's explained in the next step.

Therefore, by designing a ReAct based LLM you can enable your AI system to have access to multiple data sources, including static documents (e.g. FAQs, product descriptions, customer emails) as well as dynamic APIs (e.g. real-time product pricing, discussion forum posts). But how do you put this into the user’s hands? In the next section we describe how to integrate your AI system with Zendesk.

LLM Integration With Zendesk

Zendesk is a managed customer relationship management SaaS platform that enables businesses to provide customer support. Its two main offerings are Zendesk for Service (Zendesk Support) and Zendesk for Sales (Zendesk Sell).

• Zendesk for Service integrates multiple chat channels (e.g. your website, social media, texting, phone calls, emails) all aggregated under the same ticket management system.
• Zendesk for Sales is a CRM (for tracking and managing your leads, contacts and deals) focused on helping you convert prospects into customers.

Below we’ll show you how to integrate with Zendesk’s multiple APIs to customize. For the purposes of this article, we’ll focus on how to hook into Zendesk to receive user queries and display custom chatbot answers.

Note: It's easy to miss the announcement that the Chat Conversations API has been in maintenance mode since 2022. So, for all your custom chatbot implementations, use only the Sunshine Conversations API.

Send Zendesk Data To Your Chatbot

A useful feature of the Sunshine Conversations API is that your chatbot app isn't limited to website chat but can receive messages from all configured support channels, including social media messengers and voice calls.

Use this approach if:

• Your chatbot is running as a standalone web application deployed at some URL.
• You want to show custom textual information generated by your chatbot.

The API uses simple webhooks (HTTP callbacks) that are posted to your chatbot web application URL along with relevant data as payloads, as shown below.

The available webhooks are listed below:

To receive customer messages, subscribe to the "conversation:message" webhook. To send a plain text reply generated by your chatbot, post it as a message back to the Zendesk Sunshine Conversations endpoint.

‍

The default Web Widget is also capable of displaying a predefined set of rich content layouts, like a set of images or buttons. If your chatbot returns rich content like product images and image galleries, use the Conversations API's structured content capability to render it. Set the payload content type and make sure to include any necessary information, like image URLs, in your chatbot responses.

Use Your Chatbot To Post To Zendesk

If you are currently using Zendesk, to respond to each customer question, you might need to manually go fetch data to provide an appropriate response. However, as described above, ReAct allows LLMs to use API tools (e.g. salesforce, custom AWS APIs, ERP system) to dynamically fetch the requested data. This allows you to use the Zendesk webhook (described above) to receive user questions directly in your chatbot, then simply teach the LLM to fetch the appropriate data and post it back to the ticket.

For such use cases, you have two options:

• Client-side rendering: If your app is very interactive and you want a snappy single-page app user experience, implement it as a Javascript client-side app and deploy it with Zendesk. Zendesk injects your module into your agents' dashboards. The logic of your user interface must be contained in that Javascript module. If your chatbot is simple, like just forwarding prompts to the GPT-4 API, you can implement its logic in the same module as well. However, if you're planning a more complex chatbot, deploy it as a standalone web application instead and make your module talk with it using the fetch() API.
• Server-side rendering: Alternatively, if you don't want a single-page app experience, you can implement server-side rendering. Since each app runs as a separate HTML iframe, it'll render your custom content within its own iframe.

Another option you can consider is the ticketing subsystem's side conversations API to conduct any internal conversations, for example with a subject matter expert chatbot. Your chatbot can be one of the collaborators with whom agents can initiate side conversations over email, Slack, or Microsoft Teams.

Using these techniques, we integrate our chatbot framework into the Zendesk ecosystem to provide seamless AI-powered expert assistance to your customers and internal users.

Let’s get started with a custom Zendesk Chatbot

In this article, we showed how to deploy custom AI-powered chatbots in the Zendesk ecosystem. Contact us to learn more and discuss custom chatbots tuned to your business needs.

References

• Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, Yuan Cao (2022). "ReAct: Synergizing Reasoning and Acting in Language Models." arXiv:2210.03629 [cs.CL]. https://arxiv.org/abs/2210.03629

‍