🟢 Chatbots

What are Chatbots?

Chatbots are computer programs that simulate interactive human conversations. In the realm of artificial intelligence, there are various types of LLMs that serve different purposes. One of the most commonly used models in this course is ChatGPT, a chatbot that remembers previous messages so that you can have a conversation with it. However, chatbots are not the only type of LLM. Another example is GPT-3, an earlier AI developed by OpenAI, which, unlike chatbots, has no memory. Both GPT-3 and ChatGPT can perform basic tasks such as answering questions and generating summaries.

How Chatbots Work

In order to have a conversation, chatbots must remember the entire conversation history. What this means for AIs like ChatGPT is that every time you send them a new message, they read all previous messages that you both sent at the same time, since they don't have a true memory. This 'memory' of conversation is the only important factor that distinguishes them from non-chatbots.

Non-chatbots

GPT-3 is a complete type of model, rather than chat. Here is an example that shows you how it differs from ChatGPT:

Copy

Prompt

---

What is 2+

Copy

GPT-3 output

---

2

2+2 = 4

Copy

ChatGPT output

---

It seems like you didn't complete your question. If you meant to ask "What is 2 + 2?" then the answer is 4. If you have a different question or need further assistance, feel free to ask!

We can see that GPT-3 has completed our input with what it felt were the most probable next characters. On the other hand, ChatGPT has responded to us as if we suddenly stopped speaking in a conversation. The conversationality of chatbots makes using them feel more natural to use, so most people prefer them over other AIs. However, the most significant reason that chatbots are better is that companies like OpenAI and Anthropic have built chatbots that are very smart and can better respond to your prompts.

Tokens

Gen AI like ChatGPT don't read words in the same way that we do. While we would read the sentence "I don't like eggs" word by word, they might break it into their own version of words, and read it like this: I, don, 't like egg s. These "words" are called tokens, and they are used in nearly every modern Gen AI. Each token is then converted into a list of numbers, so the AI can process it. You don't need to know why Gen AIs use tokens, but they are important to understand when considering pricing and context length.

Context Length

Context length refers to the amount of text that a language model can consider when generating a response. For both chatbots and non-chatbots, there is a maximum limit to the context length they can handle. If the conversation or text exceeds this limit, the model will not be able to remember the entire conversation when generating a response. This is why it is sometimes necessary to restate important information or re-prime the chatbot.

750 words is approximately 1,000 tokens. ChatGPT can remember 4K tokens, while more advanced versions of ChatGPT can remember up to 16K tokens. GPT-4 can process up to 32K tokens, and Anthropic's AI, Claude, can handle up to 100K tokens. Deciding which model to use is sometimes a trade-off between pricing and the need for longer context lengths. ChatGPT will be sufficient for this course.

Conclusion

Chatbots like ChatGPT are the primary channel through which we interact with AI models today. Understanding how they work and how to prompt is essential for leveraging the full potential of LLM capabilities.

FAQ

What is an example of a chatbot?

A popular example of a chatbot is ChatGPT, and we will commonly use it in this course.

What makes an LLM a chatbot?

A chatbot has memory, so it remembers previous messages. That's why a conversation with ChatGPT can resemble a human interaction. Other LLMs like GPT-3 can also perform basic tasks like ChatGPT, but they have no memory and therefore are not chatbots.

Why are tokens and context length important?

Understanding how chatbots read words as tokens can help developers make decisions based on pricing and context length, which is the amount of text a chatbot can remember when generating responses.