Using variables in Prompts

Prompt Engineering is a crucial skill for shaping precise and contextually accurate AI responses. By crafting effective prompts, you can ensure optimal performance and end user satisfaction from your AI assistants. One powerful technique in prompt engineering is the use of variables.

Incorporating variables within prompts allows for dynamic content substitution at runtime, enabling the system to respond contextually to user inputs and enhancing the overall conversational experience.

It is very common to add context, user, metadata, and other information that should be added to the prompt in specific places and will vary. Also, it should be composable from the caller. From a Prompt Perspective, it is to inject data with the desired values at certain locations.

Variable Syntax

Use the {variableName} pattern as a placeholder for variable substitution. Variable names can include alphanumeric characters, underscores, and hyphens.

Defining and Using Variables

1. Define Variables: You need to define the variables you want to use in your prompt. This involves specifying the variable name and its corresponding value.
2. Placeholders in Prompts: Insert the variable placeholders ({variableName}) within your prompt where you want the values to be substituted.
3. Runtime Substitution: When you execute the prompt, the defined variables will be substituted into their corresponding placeholders.

Reserved Variable Names

The following variable names are reserved:

• {inputText}: This is the raw input text provided by the user. It's the unprocessed text before any potential modifications or analysis.
• {context}: This is the context provided to the AI assistant. It can be any relevant information, such as text, code, or data.
  Within RAG Assistants, this variable represents the chunks obtained from the vectorstore closest to the end user's question. The context is automatically populated based on the most relevant information retrieved from the vectorstore.
• {chat_history}: This represents the history of the conversation between the user and the AI assistant. It includes all previous messages exchanged.
• {question}: This is the question the user asks the AI assistant.

Variable Substitution with the Assistants API

From the API caller, you will not only need to define the prompt strategy on where to place the definition of the variables, but also to fill in the associated data at runtime.

The Chat API supports the use of an optional variables collection definition using the key(string)/value(string) pattern for each element. This allows for dynamic substitution of variable definitions within a Prompt.

The supported endpoints are:

• /chat
•  /chat/completions

Sample

Imagine you want to create an AI assistant with variables to specify the type of assistant, a disclaimer, the context for the question, and a hint for the response style. The format would be as follows:

{
  ...
  "prompt": You are a {type}. Use the following pieces of context to answer the question at the end.
            {disclaimer}
            {hardDisclaimer}

            {context}

            Question: {question}
            {responseHint}
}

Where the variables that have been defined are the following:

• {type}: The type of AI assistant (e.g., "helpful", "creative", "technical").
• {disclaimer}: A disclaimer about the assistant's limitations.
• {hardDisclaimer}: A stricter disclaimer about the assistant's limitations.
• {responseHint}: A hint for the assistant's response style (e.g., "Provide a concise answer", "Explain in detail").

Once you have created an assistant with variables, you need to pass the actual values for those variables when you use the assistant:

{
  ...
  "variables": [
      {"key": "type","value": "helpful AI assistant"},
      {"key": "disclaimer","value": "If you don't know the answer, just say you don't know. DO NOT try to make up an answer."},
      {"key": "hardDisclaimer","value": "If the question is not related to the context, politely respond that you are tuned to only answer questions that are related to the context."},
      {"key": "responseHint","value": "Helpful answer in Markdown:"}
  ]
}

In the resulting prompt after the variable substitution process, the {context} and {question} variables are left unchanged:

You are a helpful AI assistant. Use the following pieces of context to answer the question at the end.
If you don't know the answer, just say you don't know. DO NOT try to make up an answer.
If the question is not related to the context, politely respond that you are tuned to only answer questions that are related to the context.

{context}

Question: {question}
Helpful answer in Markdown:

The Assistant execution process will continue as usual.

JSON Value Notation

When sending JSON values in the value element, it's important to use the correct notation, which involves using an extra set of delimiters:

• JSON Objects: Use curly brackets ({{}}).
• JSON Arrays: Use square brackets ([]).

Note: RAG Assistants API requires double curly brackets ({{}}) for JSON values.

Sample

Below are samples of valid values:

String case

{"key": "type", "value": "AI assistant"}

Array case

# Assistant sample
{"key": "properties", "value": "[{\"name\": \"something\", \"value\": \"something else\"}]"}
# RAG Assistants sample
{"key": "properties", "value": "[{{\"name\": \"something\", \"value\": \"something else\"}}]"} 

Object case

# Assistant samplecase
{"key": "item", "value": "{\"name\": \"something\"}"}
# RAG Assistants sample
{"key": "item", "value": "{{\"name\": \"something\"}}"} 

Considerations

• Parameter substitution applies to all prompts, including those used with RAG Assistants (depending on the retriever).
• Parameter substitution is Case Insensitive.
• No server-side validation is performed if variables are not substituted.
• Use the Console Requests section to review the resulting prompt.