Blog, Mobile, Product Management

My first week with Wit.ai Bot Engine

wit ai bot engine robot

More than one year has passed since Facebook announced the purchase of the startup Wit.ai showing the world its interest in understanding the human language. In these 15 months many improvements have been introduced to the original voice-activated interface.

I have been playing for one week with the new product developed by Wit.ai, a very early beta of the Bot Engine, and I thought I’d share my impressions about it.

Let’s start from the long term goal.

This year at the F8, Facebook surprised the audience showing off the integration of machine driven conversations inside the Messenger platform. Yes, the same Messenger platform that is used by 900M users for personal messaging, now extended to talk with brands and service providers.

No human being is expected to answer the user requests, but a smart bot capable of understanding human language and providing a meaningful feedback.

Quick, scalable, useful.

Spring chatbot on Facebook Messenger from Industry Dive on Vimeo.

The video above shows how Spring (one of the fastest growing fashion startups in the US) showcased a simple bot in the newly available Messenger Platform, allowing customers to buy sneakers directly from the messaging app.

The results shown in this prototype are still embryonal. The only possible answers are already predefined and the user can only choose between a couple of options.

But still it sounds like a smart move from Facebook. Specially because not long ago they launched the Messenger assistant or Facebook M.

Now everything starts to make sense. An entire platform for third-party conversations and a human friendly bot to run it. Sleek.

Thanks to these two technologies, the final user experience can finally get closer to a proper human-human conversation, but with a substantial set of differences:

  • Facebook users can now complete quick actions directly from the Messenger App, where they spend already most of their time, using human language to use the service
  • The scope of conversations will be limited to the action itself. No chit chat about the weather or  football results (well, excluding weather and sport services)
  • Apps will need time to reach a high language proficiency. The first interactions will be limited to buy and deliver, only with the time the experience will reach a new deepness – should I buy the red dress or the blue one?

Here lies the real innovation: the natural language used for the actions. Human language.

I was looking forward to trying first hand the integration between the Messenger Platform (conversational action-reaction) and test the human language understanding offered by Wit.ai. So I just created an account in Wit and started playing, giving me one week to understand the state of the Bot Engine and its limitations.

I wanted to experiment with bots and check if they are still far away from a real human to human interaction. I also tested myself to create a simple bot and to teach it some basic interactions.

 

My first week with Wit.ai Bot Engine

wit ai Bot Engine

Disclaimer. I am a Product Manager, not a developer, and I am not really interested in creating a fully operational application running with the Bot Engine. What I wanted to achieve is having a bot based on human language comprehension that replies to set of predefined requests with some meaningful answers.

Very interestingly, Wit.ai allows me to create the bot, talk to him and teach him some basics stories, all without writing any line of code. The web interface is powerful enough to feed the bot engine and even the learning curve for the teaching phase is pretty short. This is a big plus. Easy to use interface and learning curve: A.

A short digression on the teaching method

Generally speaking, there are two ways of teaching an AI how to think.

The “correct” teaching strategy of machine learning is feeding the AI with millions of conversation examples and wait for it to find patterns and learn from of them.

The problem is that when I say millions, I may actually say billions. Achieving a simple level of understanding requires A LOT of examples. It is more or less as learning how to speak German just by overhearing conversations. Slowly you start acquiring basic fundaments, but before being able to construct a sentence it will require A LOT of time.

The other method is teaching the bot how to answer a predefined set of sentences (aka rules).

For instance I can teach it to answer what’s the time in {location}?. It is a very fast way to achieve some results. And for some “not-so-smart AI” it is actually enough. But language is so complex that it would never be possible to hardcode all the possible variations. So the question what’s the time now in {location}? won’t produce any answer.

Wit.ai bets on a third approach, which is a combination of the previous two.

It is based on Stories, or simple conversations that follow the same pattern. Similar to the basic conversations between students of a language course that repeat the same conversation all over again with small differences.

¿Qué tal el día en Granada?
Está soleado. ¿Y cómo está el día en Madrid?
Un poco nublado.

Questions and answers can be expanded with entities and contexts that are the core part of the bot experience. Innovative machine teaching approach: A.

But all that glitters is not gold.

The premises are definitely positive, but the final product is still in a very early beta phase. I have been creating dozens of stories, from basic requests-responses to more complex ones.

Each time the very first couple of stories are validated positively but when introducing some complexity the entire structure starts to collapse. Basic keywords are not recognized anymore and the overall learning phase gets stuck. Reliability of the story creation algorithm: C.

example of usage Bot Engine wit.ai

An important feature for me is time.

As mentioned before, the learning curve is pretty short. It is uncomplicated to start playing with the Bot and start teaching it something.

But to achieve a nice level of self understanding, things are a little different.

The first problem is the unreliability of the algorithm. Even the simplest story can take forever if the matching doesn’t work. I have seen myself deleting and rewriting dozens of stories, getting to the point of copy/pasting the same one and still getting some errors.

The second time consuming operation is expanding the reach of the Bot Engine. And here lies the real problem: I don’t know what a possible user will ask my Bot, which information they will provide and which not, even for the easiest of actions. Hence I’d need to create enough stories to cover all chances. Time consuming learning phase: C.

I tend to focus on the frustrating experiences sometimes and don’t get surprised by magic. So don’t get me wrong, this tool is pure wizardry. After a short while I forget that I am talking with a bot, a bot that I am teaching how to behave.

The language comprehension is advanced enough to understand the same meaning from sentences like I’d like to buy a new sofa or my sofa is too old and I need to buy a new one or even a way less polite Bot, stop talking and help me buy a stupid sofa. It just works. Almost magic language understanding: A.

Whilst the Bot Engine excels in language understanding it also lacks on proper property of language.

Probably it is just because there is no real use case for it. But after some time it gets very annoying to receive the same answers from the virtual assistant.

There is no option to define similar sentences for the predefined answers. So if the conversation leads to some sort of greeting, the bot will always say of course I can help you book your flight, where are you flying to? The first time, the second time, the hundredth time. Boring not-very-human answer mechanism: B-.

Creepy enough, after some hours chatting with the Bot, I am starting to humanizing him. I am calling it “Bot” and pretending it is a he. I am starting to get scared. Bonus: Creepy humanization of a machine.

Now we get to the final point.

Actually I don’t know how I have been able to keep this quiet until the end: the Wit.ai Engine Bot works with voice commands! Anything the Bot learned during never-ending repeated conversations can also be triggered by voice. No additional hooks or third party integrations. I was impressed. Voice controlled Bot out of the box: A++.

My final impression is more than positive. There is still a lot of work before getting out of the beta, but now it is the right time to start planning some near-future further implementation.

 

What about you Bot, did you enjoy my review?

hi to you. how can I assist you?

Seriously? 😓

nice to hear that. What about a white one?

 😓

Previous Post Next Post

You Might Also Like

  • Caleb Smith

    Great writeup! I have been experimenting with Wit.ai over the past week and trying to integrate the system with my server and Facebook Messenger – success so far! The learning curve has definitely been in two different areas: conversing and intent through the API.

    Even though the stories and conversations will be spot on – the confidence level will almost always be less 0.1, which is way too low. Naturally – I want to keep confidence high in my results so I can return a message that is similar to an error message, if the robot doesn’t understand what was being asked in a conversation. I assume most of this will change once I start processing more expressions.

    Overall – great system and great API! I wish I could see some examples of how many stories others are using for their bots. I am up to 15 right now and I am already becoming slightly overwhelmed with making sure I tag expressions appropriately.

  • Youssef

    I completely agree, i think it got quite better and more intuitive during the past few months and i hope it’ll continue to do so, here is a simple example i was able to make in a few days https://github.com/youssef06/show-around