Conversation Memory

Overview

Leveraging LLMs to perform specific functions within your Assistant can be more powerful if they understand the full context of the conversation leading up to that point in the conversation. To simplify adding that context into a Prompt, you have the option to leverage Memory on each Set AI and Response AI Step. 

If you're using the Memory option on any of your AI Steps, we will automatically include the previous 8 Turns (user inputs and system responses) in the conversation into the prompt to provide context on:

• What the user has been asking
• Specific entities or topics they have mentioned
• The responses your Assistant has provided (static or LLM-generated)
• The tone of the conversation and persona of your Assistant

This enables your Assistant to dynamically reference content from previous interactions in the conversation. For ex.

Assistant: Welcome to Voiceflow Pizza, can I take your order?

User: I'd like a large, 3 topping pizza.

Assistant: Sure thing, which toppings would you like?

User: Pepperoni, olives and mushrooms. And make it a small.

Assistant: Got it, so thats a small pizza, with pepperoni, olives and mushrooms. Anything else?

This allows for much more dynamic, human interactions, when memory can be preserved across the conversation.

This is still an experimental feature, and for this reason and is not recommended to be used for serious production use cases.

Enabling Memory on a Response AI or Set AI Step

Once you have placed your step, you can configure it in the Editor. In the editor, you will have three different options to prompt your Assistant:

• Use Prompt only - When the Step is hit during a user's session, the Prompt you provide will be the only data passed to the LLM to generate a response. This option is useful when you want an LLM to perform a very specific function, or generate a very specific response, and including Memory could mess that up. 
• Use Memory and Prompt - When hit during a user session, the Prompt you provide will be augmented with the previous 8 turns in the conversation, and the LLM will generate a response from both pieces of data. This option is best if you want to enable the most dynamic possible output from the LLM, because this will provide the most context possible for it to inform it's response. 
• Use Memory only (only available on Response AI) - This will pass only the previous 8 turns of the conversation to the LLM and allow it to response without any guidance from you. This will enable a purely-conversational interaction between the user and your Assistant, without you providing it a specific goal or task.

Testing your Generated Responses

You can test your prompt using the Preview button in the Editor, but this will not include any Memory. To test the function of your Set AI or Response AI Steps using a Memory option, you will need to run a conversation in the Test Tool.