92% 4 Level Deep Product Categorization with MultiLingual Dataset for Wholesale Marketplace
How we were able to reach 97% on top level categories, and 92% on bottom level categories with a multilingual dataset for a customer of Pumice.ai.
In this step-by-step guide you’ll learn how to set up a custom AI-powered Zendesk chatbot to improve your customer service and sales CRM.
Zendesk's support platform is a popular solution that enables you to offer an excellent customer service experience. You can further enhance your customer support experience by deploying AI-powered custom chatbots that can answer complex queries and help users through workflows. This is a game changer, it can empower your customer and sales representatives or even directly interact with customer requests. We prepared this article to help you learn how to integrate modern artificial intelligence directly into Zendesk, and what Zendesk-integrated chatbots can do for your business.
Zendesk’s Ai chatbots tend to be simplistic, like copy+pasting every question to ChatGPT. We don’t want that. The key problem with Zendesk's AI system is that it's a managed service that allows for little customization specific to your use case. It doesn’t naturally allow for customization of the prompting framework, data management system, and embedded vector generation pipeline. What you get is what you get. This is pretty common with managed service chatbots as they want to make it as easy as possible to get rocking and rolling. This lack of customization means it's nearly impossible to move the system to a production system that you fully control and adjust for specific use cases.
If you need something more sophisticated, like multi-agent systems with advanced prompting that interact with other tools, you will need to implement a custom framework. Below we’ll take a look at exactly that - so let’s dive right in!
In this section, we explain our field-tested framework for integrating a custom LLM based AI chatbot system into Zendesk. Our system supports the following capabilities out of the box:
The architecture that enables all these is shown below:
You might be thinking, why not use the OpenAI Assistants API? Turns out, as of January 2024, the Assistants API has some critical drawbacks that make it unsuitable for production.
Below we show you how to set-up a custom architecture to overcome these limitations.
To answer a complex question, an intelligent system must be able to (i) break it up into a series of smaller information gathering steps, (ii) interact with external systems to gather the relevant information from disparate sources, and finally (iii) combine the information into a coherent response that directly addresses the question.
In summary our LLM application must be able to:
To achieve this, below we show you how to equip GPT-4 (or any LLM of your choice) with ReAct. ReAct, short for Reasoning+Action, is a framework used to give LLMs better reasoning skills along with an ability to perform custom actions like using external APIs.
The full list of available actions is domain-specific and supplied via few-shot examples to the LLM prompt. The LLM agent then picks the actions that are most helpful for answering the question, and executes those actions. The results of the actions are provided back to the LLM agent as observations. Equipped with those data, the agent can either continue its reasoning process, seeking more relevant information, or terminate and provide a response. Some examples of a ReAct system in action are shown below:
Using ReAct we can design an application for your customers to seamlessly interact with your business tools. It’s a win-win, it saves them time and hassle, and it saves the time of your most knowledgeable employees.
Below we take a look at how to build a ReAct-based system that can handle complex questions & more challenging tasks unsuited for a vanilla GPT. In the next section we explain how to integrate your ReAct system with Zendesk for a natural customer relationship management workflow.
Here is what we have learned about designing an effective ReAct Agent.
There are two critical components to this system. First, to find information relevant to a query in static documents — like FAQs, product descriptions, customer emails, or discussion forum posts — we implement a ReAct action named "search(query)." It does the following operations:
This framework also supports adding new documents or updating existing ones. After such changes, the document embeddings are recalculated and updated in the vector database. Once set up, managing the knowledge database your system has access to is intuitive.
As mentioned, retrieving static information from databases is just one part of the final answer though. The real value and productivity gains of our custom chatbots lie in their ability to weave in real-time dynamic information fetched from other business systems like Zendesk or ERP systems. That is the topic of the next section.
Some information can be uploaded to your database and not modified (e.g. custom documents, SOPs). However, to build a production ready system, we need our LLM to be albe to obtain real-time ticketing and customer relationship management data from Zendesk (as well as pricing and other dynamically changing data streams). Notice how in the diagram below, the Chatbot has an ability to interact with multiple data sources (on the right).
Using ReAct, we can give an LLM the ability to use these retrieval APIs:
We can implement ReAct actions using GPT-4 function-calling. One advantage of the GPT-4 API is that it provides a structured way to specify actions compared to ReAct's original proposals. Note however, that this is also possible with any other LLM (like Llama 2 or Anthropic), but falls outside the scope of this article - feel free to reach out if you want to learn more.
OpenAI's API enables you to specify actions and their information in great detail as shown below:
GPT-4 uses that semantic information and metadata — the function description, parameter property descriptions, parameter units, and so on — to make informed decisions on when to invoke your actions based on your queries. GPT-4 also enables the structured invocation of actions and transmission of action responses back to the LLM.
The framework combines all the retrieved document chunks and the relevant dynamic data into a coherent answer by appropriately prompting an LLM. All that remains now is to get the user query via Zendesk and show the answer to the customer or internal stakeholder via Zendesk's front end. That's explained in the next step.
Therefore, by designing a ReAct based LLM you can enable your AI system to have access to multiple data sources, including static documents (e.g. FAQs, product descriptions, customer emails) as well as dynamic APIs (e.g. real-time product pricing, discussion forum posts). But how do you put this into the user’s hands? In the next section we describe how to integrate your AI system with Zendesk.
Zendesk is a managed customer relationship management SaaS platform that enables businesses to provide customer support. Its two main offerings are Zendesk for Service (Zendesk Support) and Zendesk for Sales (Zendesk Sell).
Below we’ll show you how to integrate with Zendesk’s multiple APIs to customize. For the purposes of this article, we’ll focus on how to hook into Zendesk to receive user queries and display custom chatbot answers.
Note: It's easy to miss the announcement that the Chat Conversations API has been in maintenance mode since 2022. So, for all your custom chatbot implementations, use only the Sunshine Conversations API.
A useful feature of the Sunshine Conversations API is that your chatbot app isn't limited to website chat but can receive messages from all configured support channels, including social media messengers and voice calls.
Use this approach if:
The API uses simple webhooks (HTTP callbacks) that are posted to your chatbot web application URL along with relevant data as payloads, as shown below.
The available webhooks are listed below:
To receive customer messages, subscribe to the "conversation:message" webhook. To send a plain text reply generated by your chatbot, post it as a message back to the Zendesk Sunshine Conversations endpoint.
The default Web Widget is also capable of displaying a predefined set of rich content layouts, like a set of images or buttons. If your chatbot returns rich content like product images and image galleries, use the Conversations API's structured content capability to render it. Set the payload content type and make sure to include any necessary information, like image URLs, in your chatbot responses.
If you are currently using Zendesk, to respond to each customer question, you might need to manually go fetch data to provide an appropriate response. However, as described above, ReAct allows LLMs to use API tools (e.g. salesforce, custom AWS APIs, ERP system) to dynamically fetch the requested data. This allows you to use the Zendesk webhook (described above) to receive user questions directly in your chatbot, then simply teach the LLM to fetch the appropriate data and post it back to the ticket.
For such use cases, you have two options:
Another option you can consider is the ticketing subsystem's side conversations API to conduct any internal conversations, for example with a subject matter expert chatbot. Your chatbot can be one of the collaborators with whom agents can initiate side conversations over email, Slack, or Microsoft Teams.
Using these techniques, we integrate our chatbot framework into the Zendesk ecosystem to provide seamless AI-powered expert assistance to your customers and internal users.
In this article, we showed how to deploy custom AI-powered chatbots in the Zendesk ecosystem. Contact us to learn more and discuss custom chatbots tuned to your business needs.