Who’s The Best Free Chatbot?

Author(s): Dor Meir

Originally published on Towards AI.

ChatGPT vs. Bing Chat vs. Bard

Disclaimer (originated in here): This is not a sponsored article. I don’t have any affiliation with OpenAI, Microsoft or Google. The article shows an unbiased overview of chatbots, intending to make them accessible to the broader masses.

To jump straight to the end and find out who’s the best chatbot, click here.

1. Why Should I Know?
2. The Criteria
3. General Intelligence
4. Connectivity to the outside world
5. Memory
6. Additional Capabilities
7. Reliability
8. Conclusion

Why Should I Know?

I’ve been using OpenAI’s ChatGPT for about 6 months now, Microsoft’s Bing Chat for almost 3 months, and Google’s Bard for a month. I used ChatGPT and Bing for personal assistance in Data Science work, for answering theoretical questions, for writing articles and professional posts (see 1, 2, 3), and even for some career advice. I gave Bard a try just to understand if it is indeed another great Google product, and of course to see how it compares with ChatGPT and Bing Chat.

The Criteria

First, we are bounded by a very strict condition:

I only reviewed completely free-for-use chatbots.

Let’s quickly explain our criteria, ordered by level of importance:

• General Intelligence — The most important quality. How smart is the chatbot? Does it understand short, vague, and misspelled messages? Does it answer with the context of older messages in mind?
• Connectivity to the outside world — Another aspect of intelligence that can give huge value to the user. Internet connectivity is especially useful for summarizing or filtering the content of a specific website. It’s also highly useful if we need the chatbot’s answers to be backed by online sources, and it’s also just fun to ask it about the news.
• Memory — Does the bot allow you to go back to previous Chats? Are the chat sessions limited by a predefined number of messages? That’s an important trait: while clean slate chats are useful for quick queries, a chat that you have “a long history” with can be more customized for your particular needs and therefore used for long projects.
• Additional Capabilities — Not your everyday needs from a chatbot, but still very useful to have. If you’re tired of having to write everything down, it’s rather useful to have the bot suggest follow-up questions or contain a mouse-click composition tool. And a text-to-image engine that’s embedded inside the chat can be a major timesaver.
• Reliability— How frequent is it to encounter a service downtime? How easy is it to access? Do you get the full chat capabilities no matter what browser you’re using? And will it do everything you ask for?

Now, here’s the comparison for each criterion:

General Intelligence

The comparison between ChatGPT and Bing Chat is rather straightforward: ChatGPT free bot is GPT3.5 based, while Bing Chat uses GPT4. But does it matter? You probably wouldn’t notice the difference if you just asked simple questions. Still, heavy users of the platform that dive into deep conversations will probably see the difference. Here’s an example of how the two GPT versions tested in more complex queries like riddles:

However, there’s a catch to this comparison, as Bing Chat has three different conversation styles: More Precise (green), More Balanced (blue), and More Creative (purple):

I only use Creative mode, and that’s because Creative mode is not only more imaginative — it’s also way more intelligent than the others.

Let me show you why.

wahts basisyian betwrok?

Precise mode

Precise mode got the misspelling correct. Its explanation of the Bayesian network is good, but rather short and uninformative. I can’t understand from this what a DAG is, and how can Bayesian network “represents the probabilistic relationship between diseases and symptoms”.

Balanced mode

This explanation is also good, and it might be more useful to know of other names used for Bayesian networks. Even so, I still don’t understand how the network works.

Creative mode

Creative mode didn’t get what I wanted on the first try — maybe since its creativity allows for a bigger space of possible words to replace the current misspellings. The follow-up questions suggested already had the correct spelling, and when I used it, I got not only an explanation containing bold formatting for important words but also a clear and explicit explanation of the graph:

“Each node in the graph corresponds to a random variable, and each edge represents the conditional probability for the corresponding random variables”.

For the sake of the exercise, let’s also try ChatGPT:

That’s a decent explanation! Perhaps a little TMI, but it makes sense OpenAI will make ChatGPT spill out long outputs, as the bot has no way to refer to sources like Bing Chat.

How’s Bard's intelligence, do you ask?

Well…

That’s right, it got it all wrong thinking I was asking about “a group of people who play the bass guitar”.

Is Bard less intelligent?

In contrary to ChatGPT and Bing Chat, Bard is based on PaLM 2. Which one is considered more intelligent? Opinions differ:

Surprise! Bard has answered PaLM is more intelligent… Bing Chat was more balanced, though.

Nevertheless, my experience with Bing Chat reasoning, as Bing suggests, is simply better. I once gave both bots an interesting challenge: I chatted with them for a specific number of messages on the same exact topic and then asked them which country they thought I was from. It took Bing Chat only an extra question to get this right, while for Bard, it was about 20…

Winner: Bing Chat (Creative mode)

Connectivity to the outside world

The obvious problem with the intelligence of chatbots, or more precisely, their knowledge, is that training the models costs millions of dollars and cannot be done on a regular basis. Thus, their models are trained on data imported from the internet up to a certain cutoff date and later used for months and years to come.

However, there’s a rather simple (but dangerous) solution for this — connect the chatbot to the internet.

Attention + Internet is all you need.

Moreover, an internet connection not only expands the knowledge of the chatbots beyond their trained data but also allows us to do amazing things like review your LinkedIn page or summarize a paper:

ChatGPT is the obvious loser of this criterion. But who’s the winner?

The way Bing Chat incorporates citations inside the text like in academic papers, is remarkable. You don’t ever have to fully trust what the machine says, you can just follow the link and look it up yourself. Of course, you can also just use these links for deep diving into the subject or as sources to an argument you’re trying to base.

Winner: Bing Chat

Memory

Another aspect of intelligence is the capability to remember the past. In this case, remember prior queries of the user since, most of the time, it’ll take more than a few messages to get it to understand your point. You can converse with the smartest chatbot ever trained, yet if it refreshes every time you exit the browser or is limited by a number of messages per session, the mighty intelligence will be rather flawed.

Furthermore, sometimes it’s necessary to engineer the prompt the way you want the queries to look. This might happen if you teach your chatbot, through a series of long messages, to write something in a specific style or to master a new skill you’re interested in automating.

Who has the best memory?

Bard seems to only remember user queries; without the actual answers, it gave. I also couldn’t find any clear distinction between sessions; each query seems to stand on its own:

Bing Chat, on the other hand, can not only show you past queries but also the answers. You can even get back to the old chat and continue the conversation. The only problem is, and that’s a major problem, that the chat is limited to 30 user queries only (and 5 if you’re using Chrome).

The ultimate winner here is undoubtedly ChatGPT:

ChatGPT’s memory is outstandingly good. I can get back to conversations I had with him as early as in January and just continue the conversation without any limit or boundary. Impressive.

Winner: ChatGPT

Additional Capabilities

In this criterion, I aggregated all those extra neat qualities that are not the basic ones but surely make the chatbots more elaborate and useful.

Those are:

Suggested follow-up questions

If you paid attention to the screenshots above, you might have noticed after each Bing Chat reply, there are three suggested follow-up question balloons. Many times, it feels like those questions have already understood the intent of my original query, even when the bot has yet to understand it. Here’s an example:

The follow-up questions after that were also surprisingly to-the-point:

So does the next one:

Eventually you find yourself not having to write at all — and anything you wanted to learn is just almost automatically prompted for you. Sometimes those questions even inspire you with a new direction of thought you have yet to consider.

Built-in additional tools

Bing Chat also has another tool embedded inside, that is, quite annoyingly, accessible only in the Edge browser. The tool is called Compose and it allows us to create content in a more organized, mouse-clicky way. Compose lets you set the tone, format, and length of the composition, try different drafts, and use the follow-up balloons for edits:

Let’s take the first idea Bing’s Compose has suggested, “How to use Bayesian network for causal inference”, and turn it into a long informational blog post:

The blog post turned out quite good but is missing sources. Luckily, I can add them with yet another mouse click:

I actually used Bing’s Compose to write the entire content of an article that rationally compares Lebron James to Michael Jordan.

Image Creation

It seemed like the Dall-E text-to-image generator was brought into the world ages ago. Dall-E and his successors created a massive buzz around Generative AI and AI in general but were also always quite separated from the new LLMs that followed. That is, until now:

Notice the little trick of asking the chatbot to create the text-to-image prompt — it always improves the quality of the images generated.

Another amazing thing is that the number of images created seems unlimited! Or, more precisely, unlimited per chat session that is limited to 30 messages.

Here’s the final result:

ChatGPT and Bard simply don’t possess these additional capabilities.

Winner: Bing Chat

Reliability

Almost anyone using ChatGPT has probably encountered issues using the service due to high demand. In fact, more than a few colleagues of mine have purchased the Plus subscription so that they can always guarantee service availability — but sometimes even the Plus subscription is unavailable!

With Bard and Bing Chat, however, I never came across availability issues.

Nonetheless, a service can be denied for more reasons than just overload. Bing Chat alignment is so severe that whenever the conversations get a little too deep or dark, you are immediately get thrown out of the chat:

Another problem with Bing Chat’s service concerns the browser. Bing Chat forces you to use the Edge browser, and if you bypass it using an extension, you don’t get the full chat capabilities (history, Compose, 30 messages long sessions).

However, Bing also has a great plus in its accessibility as it’s the only one with a designated app:

Bard doesn’t have an app but works perfectly fine from Android and has the major plus of using a Google account. Lastly, while Google and Microsoft saved accounts to allow you to quickly enter the chat and start asking questions, I always seem to have to make more than just a mouse click to log in to my OpenAI account:

Winner: Bard

Conclusion

You’ve made it this far! What, then, or who, is the best chatbot? Let’s tally up the scores from each criterion:

General Intelligence — Bing Chat

Connectivity to the outside world — Bing Chat

Memory — ChatGPT

Additional Capabilities — Bing Chat

Reliability — Bard

Giving equal weight to each criterion, Bing Chat is the winner with 3–1–1!

I’m not afraid to say I’m a big fan of Bing Chat. Yes, it has a very short memory (30 messages max per session) and sometimes throws me out of the chat for going in too deep (see the dramatic ending here). The reason I still love it is that it is remarkably bright! Not only does it usually understand quickly what I intend, but the follow-up suggestions are also radically precise, the bold formats and internet citations are particularly useful, and the capability to create an infinite number of images, and use the follow-up suggestions to shape them, is remarkable. Well done, Microsoft, you nailed it with a very tight product…

However, I also use ChatGPT on an almost daily basis. The chat’s long memory is immensely helpful when conducting a long project. Yes, it doesn’t have internet connectivity but doesn’t forget it is trained on a vast amount of knowledge, way more than ever needed. The long memory and very long output remind me, in a way, of the serious research-oriented vibe coming out of OpenAI.

Sometimes I also use the two simultaneously, ChatGPT for longer and deeper conversations and Bing Chat for quick, smart, and cited responses that can validate ChatGPT.

Alas, it looks like Bard still has a way to go U+1F614. Nevertheless, as with most Google products, I suspect it will sooner or later deliver much better results and perhaps even surpass the others.

Feel free to share your feedback and contact me on LinkedIn.

Thank you for reading, and good luck! U+1F340

Join thousands of data leaders on the AI newsletter. Join over 80,000 subscribers and keep up to date with the latest developments in AI. From research to projects and ideas. If you are building an AI startup, an AI-related product, or a service, we invite you to consider becoming a sponsor.

Published via Towards AI