Enterprise Chatbots: What We Know, and How We’ll Act

enterprise chatbots user experience

Enterprise chatbots are coming. There will be winners and losers among adopters. Some will be suckered into half-functioning products that harm their IT brands, but not us. We understand artificial intelligence, and what today’s capabilities are. We’ll get there, but timing is everything.

There are two important distinctions in how chatbots are approached…

Retrieval-based vs. Generative

Retrieval-based models deliver set answers to questions using a repository of outputs tied to specific inputs. The scope of answerable questions is limited, and you are limited to direct question and answer exchanges, but you have complete control over the accuracy of  information delivered.

Generative models synthesize answers to questions. They are smarter in that they can handle unseen questions with conversational flow, but they require a huge amount of data to train the machine learning algorithms that run them.

Open Domain vs. Closed Domain

An open domain conversation is like a conversation with a friend–it can go anywhere. It’s informal, and has a human-like flow because there is no defined objective.

Closed domain conversations are more driven towards an objective and are in the style of one-to-one question and answer exchanges. A question is asked and answered, with clarifying questions posed when necessary.enterprise chatbots generative vs. retrieval

What works now?

Generative open domain models are progressing, but the technology is still premature and the data requirement is too large. We will see this approach at companies with massive external customer support long before any internal use because they will be the first to satisfy the data requirement. 

Generative closed domain models are more achievable from a technological perspective, but given that speech recognition accuracy is only now approaching 90%, user experience will be too poor.  

The only current option for delivering a fully functioning enterprise chatbot with certain ROI is a closed domain retrieval-based model. It’s less sexy, but exciting because it’s something we can get started on while our competition is wasting time exploring technologies that don’t work.

If we start with the retrieval-based model we can deliver high value to the business now, and then implement a generative approach when its mature to take user experience another level higher.

Getting Started

To hit the ground running and deliver a tool that people will love, we need to have a comprehensive database of questions and answers. Luckily, we can prove that collecting Q&A will produce ROI before ever plugging into a chat interface. This will give us room in the budget to get started.

There are tools we can buy to automate the creation and delivery of Q&A, but we can get started manually.

3-Step Strategy for Enterprise Chatbot Data

1) Capture Question and Answer Exchanges

People are asking and answering questions every day. If we focus in on where these exchanges are taking place–like in email, chat, or the service desk–and find a way to capture them, we can develop a large Q&A database quickly.  

Business Value

If this Q&A is made accessible to employees, questions will only ever need to be answered once. Imagine all the questions that could be deflected.

2) Convert Existing Resources

Not all questions are simple how-to’s. Sometimes there are more specific questions where detailed documents like runbooks and standard operating procedures are the points of reference. These documents can also be broken down into Q&A, lots of it.

Business Value

If we integrate Q&A into a repository search, like SharePoint, we can completely revolutionize that tool by linking back to source documents. Think about last time you had to find an answer to a question in a repository. It probably took a while, and it isn’t even guaranteed that you found your answer. Allowing people to ask a question and get an answer with a document will introduce value never seen before.

3) Ensure Accuracy

An advantage of the retrieval-based model is that we can guarantee accuracy, so we need to make sure our data is always correct. Answers can be wrong or outdated. Manual maintenance isn’t going to work. We need to figure out how to crowdsource accuracy. If we allow people to engage around answers with comments and flags, they will take care of the work for us.  

Business Value

Repositories contain a lot of outdated material. If we have a FAQ layer that’s easy to control for accuracy, we can reduce the cost of employees acting on bad information. We can direct people to the right information without having to sift through garbage.

Location, Location, Location

Other than improving access to existing documents, Q&A will benefit us by simplifying workflows. When people need information, they are required to move between systems to find what they need. If we store Q&A in the applications they are relevant to, we can unsilo knowledge to improve user experience. For example, if we imbed SAP Q&A into a search bar inside SAP, people will not have to leave where they are working to get answers. Let’s always think about where we can store Q&A to maximize the value of our existing tools.


Enterprise chatbots need to be introduced through continuous innovation. The technology needs to be a logical step in adding value to the business without assuming risk. For this reason we see that the generative approach is premature, but the retrieval method is ready. What really sells this is that, if we do it right, we can create tremendous business ROI while we collect our retrieval data. We’ll deliver a series of small wins on our journey to chatbots.  

Posted in

Ready to find out more?

Drop us a line today for a free quote!

Chris Hill

Leave a Comment