Skip to main content

Working with Chatflows

Chatflows give you more control than Assistants. Instead of filling out a form, you visually connect building blocks to create exactly the workflow you want. Think of it as going from using a microwave (Assistant) to cooking on a stove (Chatflow) — more control, more possibilities.

When to Use Chatflows vs Assistants​

FeatureAssistantChatflow
Ease of use⭐⭐⭐⭐⭐⭐⭐⭐⭐
Flexibility⭐⭐⭐⭐⭐⭐⭐
Custom RAG setupBasicAdvanced
Multiple vector storesNoYes
Custom chainsNoYes
Best forQuick prototypesProduction chatbots

Step 1: Create a New Chatflow​

  1. In the left sidebar, click "Chatflows"
  2. Click the "+ Add New" button
  3. You'll see a blank canvas — this is your workspace!

Step 2: Understanding Nodes​

On the left side of the canvas, you'll see a panel with categories of nodes. Let's understand the main ones:

Chat Models 🧠​

These are the AI brains. You'll almost always need one of these:

  • ChatOpenAI: OpenAI's models (GPT-4o, GPT-4o-mini)
  • ChatAnthropic: Anthropic's Claude models
  • ChatGoogleGenerativeAI: Google's Gemini models
  • ChatOllama: Run open-source AI models

Chains 🔗​

Chains connect your AI model to other components:

  • Conversation Chain: The simplest — just chat back and forth
  • Conversational Retrieval QA Chain: Chat + search through documents

Memory 💭​

Memory lets your chatbot remember previous messages:

  • Buffer Memory: Remembers everything (uses more tokens)
  • Buffer Window Memory: Remembers the last N messages
  • Conversation Summary Memory: Summarizes old messages to save space

Vector Stores 📦​

Where your documents are stored for searching:

  • In-Memory Vector Store: Quick and easy, but data disappears when you restart
  • Pinecone: Cloud-based, production-ready
  • Qdrant: High-performance vector database

Document Loaders 📄​

How you get documents into the system:

  • PDF File: Upload PDF documents
  • Web Scraper: Pull content from websites
  • Text File: Upload plain text files

Text Splitters ✂️​

Break large documents into smaller, searchable pieces:

  • Recursive Character Text Splitter: The most commonly used — smart about where it splits

Embeddings 🔢​

Convert text into numbers so the AI can search through it:

  • OpenAI Embeddings: Most popular choice
  • Google Embeddings: Good alternative

Step 3: Build a Simple Chatbot​

Let's build a basic chatbot step by step:

3a. Add a Chat Model​

  1. In the left panel, expand "Chat Models"
  2. Drag "ChatOpenAI" onto the canvas
  3. Click on the node to configure it:
    • Credential: Select your OpenAI API key
    • Model Name: Choose gpt-4o-mini
    • Temperature: Leave at 0.9 (higher = more creative, lower = more focused)

3b. Add Memory​

  1. Expand "Memory" in the left panel
  2. Drag "Buffer Memory" onto the canvas
  3. No configuration needed — it works out of the box

3c. Add a Conversation Chain​

  1. Expand "Chains" in the left panel
  2. Drag "Conversation Chain" onto the canvas
  3. Now connect the pieces:
    • Draw a line from ChatOpenAI → Conversation Chain (connect to the "Chat Model" input)
    • Draw a line from Buffer Memory → Conversation Chain (connect to the "Memory" input)

3d. Test It!​

  1. Click "Save" in the top right
  2. Click the chat bubble icon in the bottom right
  3. Type a message and see your chatbot respond!

Your canvas should look something like this:

┌──────────────┐     ┌─────────────────────┐
│  ChatOpenAI  │────▶│  Conversation Chain  │
└──────────────┘     └─────────────────────┘
                              ▲
┌──────────────┐              │
│ Buffer Memory│──────────────┘
└──────────────┘

Step 4: Add Document Search (RAG)​

Let's upgrade your chatbot so it can answer questions about a specific document:

4a. Replace the Chain​

  1. Delete the Conversation Chain node (click it, press Delete)
  2. Drag a "Conversational Retrieval QA Chain" onto the canvas

4b. Add Document Loading​

  1. Drag a "PDF File" loader onto the canvas
  2. Click on it and upload a PDF document

4c. Add Text Splitting​

  1. Drag a "Recursive Character Text Splitter" onto the canvas
  2. Connect it to the PDF File loader
  3. Configure it:
    • Chunk Size: 1500 (how big each piece of text is)
    • Chunk Overlap: 200 (how much pieces overlap — helps maintain context)

4d. Add Embeddings​

  1. Drag "OpenAI Embeddings" onto the canvas
  2. Select your OpenAI credential

4e. Add a Vector Store​

  1. Drag "In-Memory Vector Store" onto the canvas
  2. Connect everything:
    • Text Splitter → PDF File (Document input)
    • PDF File → In-Memory Vector Store (Document input)
    • OpenAI Embeddings → In-Memory Vector Store (Embeddings input)

4f. Connect It All​

  • ChatOpenAI → Conversational Retrieval QA Chain (Chat Model input)
  • In-Memory Vector Store → Conversational Retrieval QA Chain (Vector Store input)
  • Buffer Memory → Conversational Retrieval QA Chain (Memory input)

4g. Test It!​

Save and test by asking questions about your uploaded document. The AI will search through the document and give you answers based on what it finds!

Understanding How RAG Works​

Here's what happens when you ask a question:

You: "What is the return policy?"
         │
         ▼
┌─────────────────────┐
│ 1. Your question is  │
│    converted to      │
│    numbers (embedded)│
└─────────┬───────────┘
          │
          ▼
┌─────────────────────┐
│ 2. Those numbers are │
│    compared to your  │
│    document chunks   │
└─────────┬───────────┘
          │
          ▼
┌─────────────────────┐
│ 3. The most relevant │
│    chunks are found  │
└─────────┬───────────┘
          │
          ▼
┌─────────────────────┐
│ 4. The AI reads those│
│    chunks + your     │
│    question and      │
│    writes an answer  │
└─────────────────────┘

Tips for Better Chatflows​

  1. Start simple: Begin with a basic chain, then add complexity
  2. Chunk size matters: Smaller chunks = more precise answers but might miss context. Larger chunks = more context but less precise
  3. Use the right memory: Buffer Window Memory (last 5-10 messages) is usually the best balance
  4. Test with real questions: Use the kinds of questions your actual users would ask
  5. Check source documents: Enable "Return Source Documents" to see which chunks the AI used — this helps you debug

What's Next?​

Chatflows are great for single-agent systems, but what if you need multiple AI agents working together, conditional logic, or human approval steps? That's where Agentflow comes in!

👉 Mastering Agentflow