AI Demystified: Part 2 — Context is Everything, controlling LLMs

LLMs, Chat GPT. Everything is out of control!

There is a common misconception that GPT models / LLMS cannot be controlled, and this is stopping us from diving down the path of embedding them into our businesses. I’ll provide a counter to that, everyone IS using them anyway, and we can control what they output enough that we can use them safely and effectively in business.

Bing chat adding context to conversation

Bing Chat building context into its conversation

Common chat tools like Bing chat as shown above, use context to direct conversation and ensure correct responses.

In the previous article, we learned about the basics of large language models (LLMs), today we are diving into the weeds, how can we “control LLMs” by providing context. I’m going to dive straight in, so read the previous article if you need more context 😉!

Context: What is it and why does it matter?

Context is the information that helps an LLM understand the meaning, purpose and tone of a given prompt. As part of most “prompts” to LLMs context is baked into the response in the background, be it via code within the prompt call, or via pretraining. Context can be provided in various ways, such as:

1. Using keywords or phrases that indicate the topic, domain or genre of the text to be generated.

2. Using examples or templates that show the desired format, structure or style of the text to be generated.- Using metadata or tags that specify the attributes, constraints or preferences of the text to be generated.

Providing context to an LLM can help shape the way prompts are understood and the output is generated. For example, if we want to create an LLM that is built to generate product reviews for laptops, the context we’d provide it before every prompt could be.

Keywords: laptop, review, pros and cons,

Example: “I bought this laptop a month ago and I am very happy with it. It has a sleek design, a fast processor and a long battery life. The only downside is that it is a bit heavy and the speakers are not very loud. I would give it a 4 out of 5 stars.”

Metadata: product name, price, features, target audience

If this context is behind every prompt (which you can hide), the model will try to format a response like a laptop review

Personality: How do you define a business persona using context?

One of the applications of LLMs in business is to create a persona or a voice for a brand, product or service. A persona is a representation of the characteristics, values and goals of a business entity that communicates with its customers or users. A persona can help create a distinctive and engaging identity for a business entity and establish trust and rapport with its customers or users.

To define a persona using context, we need to consider the following aspects:

1. Who is the persona? What is its name, role, background and personality?

2. Who is the audience? What are their demographics, needs, preferences and expectations?

3. What is the goal? What is the purpose, message and tone of the communication?

For example, if we want to create a persona for a chatbot that provides customer service for an online bookstore, we can provide context such as:

1. Who is the persona? “Hi, I’m Booky, your friendly online bookstore assistant. I love reading books and helping people find their next favorite read.”-

2. Who is the audience? “You are a book lover who likes to browse and buy books online. You are looking for recommendations, reviews or information about books or authors.”

3. What is the goal? “I’m here to help you with your queries and requests. I can suggest books based on your preferences, show you ratings and reviews from other customers, or answer any questions you have about our products or services.”

By providing context, we can help the LLM generate texts that reflect the persona’s identity, address the audience’s needs and achieve the goal of the communication.

Baking in complex information: The RAG method

One of the challenges of using LLMs in business is to ensure that they generate accurate and reliable information that is relevant to the domain and task at hand. One way to address this challenge is to use the RAG methodology, which stands for Retrieve, Answer and Generate.

The RAG method allows you to embed details into context that far exceed the number of tokens available

The RAG method allows you to embed details into context that far exceed the number of tokens available

The RAG methodology consists of three steps:

1. Retrieve: The first step is to retrieve relevant information from external sources such as databases, knowledge bases or web pages. This information can be used to answer queries or provide facts or evidence for assertions.

2. Answer: The second step is to answer queries or provide facts or evidence using the retrieved information. This can be done by using natural language understanding techniques such as question answering or information extraction.

3. Generate: The third step is to generate natural language texts using the answers or facts or evidence as input. This can be done by using natural language generation techniques such as text summarization or paraphrasing.

Guiderails and de-risking: Providing context to avoid hallucination.

One of the risks of using LLMs in business is that they may generate texts that are inaccurate, unreliable or irrelevant to the domain and task at hand. This may happen due to hallucination, which is the phenomenon of generating texts that are not supported by the input or the external sources. Hallucination may result from various factors, such as:

• Lack of context: The LLM may not have enough information to understand the meaning, purpose and tone of the input or the output.

• Lack of knowledge: The LLM may not have enough knowledge about the domain or the task at hand to generate relevant and accurate information.

• Lack of constraints: The LLM may not have enough constraints or preferences to generate consistent and coherent texts.

To avoid hallucination, we need to provide context to the LLM that can act as guiderails or de-risking mechanisms. Guiderails are the information that helps the LLM stay on track and avoid generating texts that are inaccurate, unreliable or irrelevant. De-risking mechanisms are the information that helps the LLM detect and correct errors or inconsistencies in the generated texts.

Some examples of guiderails and de-risking mechanisms are:

• Providing keywords or phrases that indicate the topic, domain or genre of the text to be generated.

• Providing examples or templates that show the desired format, structure or style of the text to be generated.

• Providing metadata or tags that specify the attributes, constraints or preferences of the text to be generated.

• Providing feedback or ratings that indicate the quality or satisfaction of the generated texts.

• Providing verification or validation methods that check the accuracy or reliability of the generated texts.
  

By providing context to avoid hallucination, we can help the LLM generate texts that are trustworthy and useful for business purposes.

Lets build some trust

I’ve dived into a lot of detail here, but if you are a user of chatGPT, or Bing Chat or Bard, the best way to understand context is to keep an eye out for these tools trying to force context into your prompts to direct them.

Bard tries to bake in context as you start a chat.

This can come in the form as shown above where Bard sets the context for a conversation. Even more simply when tools ask if you want an answer as a report, a post, a paragraph or bulletpoints.

By providing our own context to LLMs we can adjust them for how we want them used in our business. We can bake in personality, knowledge and de-risk models by providing guiderails. If you want to learn more get in touch!