AI Security

AI Chatbot Not Working? Common Issues and How to Fix Them

DeviDevs Team
••7 min read
#chatbot#ai#troubleshooting#openai#llm

Your AI chatbot isn't working and you don't know why. This guide covers the most common chatbot problems and their solutions.

Problem: Chatbot Returns Blank/Empty Responses

Symptom:

response = get_chatbot_response("Hello")
print(response)  # None or ""

Solution 1 - Check API response structure:

# ❌ Wrong - accessing response incorrectly
response = client.chat.completions.create(...)
answer = response.content  # Wrong!
 
# ✅ Correct - proper access path
response = client.chat.completions.create(...)
answer = response.choices[0].message.content
 
# Debug: print full response
print(response)  # See actual structure

Solution 2 - Handle empty choices:

def get_safe_response(client, messages):
    response = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=messages
    )
 
    # Check if we got a valid response
    if not response.choices:
        return "No response generated"
 
    content = response.choices[0].message.content
    if not content:
        return "Empty response"
 
    return content

Solution 3 - Check finish reason:

response = client.chat.completions.create(...)
finish_reason = response.choices[0].finish_reason
 
# finish_reason can be:
# "stop" - Normal completion ✅
# "length" - Hit max_tokens limit - increase it
# "content_filter" - Response was filtered
# "null" - Still generating (shouldn't happen in non-streaming)
 
if finish_reason == "length":
    print("Response was cut off - increase max_tokens")
elif finish_reason == "content_filter":
    print("Response was filtered by safety system")

Problem: Chatbot Responds Too Slowly

Symptom: Takes 10+ seconds to respond to simple questions.

Solution 1 - Use faster model:

# Slow (but more capable)
model = "gpt-4"
 
# Fast and cheap
model = "gpt-4o-mini"  # 3-5x faster than gpt-4

Solution 2 - Use streaming:

def stream_response(messages):
    stream = client.chat.completions.create(
        model="gpt-4o-mini",
        messages=messages,
        stream=True
    )
 
    for chunk in stream:
        if chunk.choices[0].delta.content:
            yield chunk.choices[0].delta.content
 
# User sees response appear word by word - feels faster!
for word in stream_response(messages):
    print(word, end="", flush=True)

Solution 3 - Reduce prompt length:

# ❌ Too much context
messages = [
    {"role": "system", "content": long_system_prompt},  # 2000 tokens
    *conversation_history,  # 5000 tokens
    {"role": "user", "content": question}
]
 
# ✅ Trim context
def trim_messages(messages, max_tokens=3000):
    # Keep system message and last few exchanges
    system = messages[0]
    recent = messages[-6:]  # Last 3 exchanges
    return [system] + recent

Solution 4 - Add timeout:

import openai
import httpx
 
client = openai.OpenAI(
    timeout=httpx.Timeout(30.0)  # 30 second timeout
)
 
try:
    response = client.chat.completions.create(...)
except openai.APITimeoutError:
    return "Sorry, I'm taking too long. Please try again."

Problem: Chatbot Gives Wrong/Irrelevant Answers

Symptom: Bot answers questions incorrectly or goes off-topic.

Solution 1 - Improve system prompt:

# ❌ Vague system prompt
system = "You are a helpful assistant."
 
# ✅ Specific, detailed system prompt
system = """You are a customer support agent for TechStore.
 
Your role:
- Answer questions about our products and policies
- Help with order tracking and returns
- Escalate complex issues to human agents
 
Important rules:
- Only answer questions about TechStore
- If you don't know something, say "I'll connect you with a human agent"
- Never make up product information
- Be friendly but professional
 
Our return policy: 30 days, original packaging required.
Our support hours: 9 AM - 6 PM EST."""

Solution 2 - Lower temperature:

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=messages,
    temperature=0.3  # Lower = more focused, deterministic
    # temperature=1.0 = more creative, variable
)

Solution 3 - Add few-shot examples:

messages = [
    {"role": "system", "content": "You help with product recommendations."},
 
    # Example 1
    {"role": "user", "content": "I need a laptop for gaming"},
    {"role": "assistant", "content": "For gaming, I recommend the XG Pro 15. It has an RTX 4070 GPU, 32GB RAM, and a 165Hz display. Price: $1,299. Would you like more options?"},
 
    # Example 2
    {"role": "user", "content": "What's good for office work?"},
    {"role": "assistant", "content": "For office work, the SlimBook Air is perfect. It's lightweight, has 12-hour battery, and great keyboard. Price: $799. Need more details?"},
 
    # Actual user question
    {"role": "user", "content": actual_question}
]

Problem: Chatbot Forgets Previous Messages

Symptom:

User: My name is John
Bot: Nice to meet you, John!
User: What's my name?
Bot: I don't know your name.

Solution 1 - Maintain conversation history:

class Chatbot:
    def __init__(self):
        self.messages = [
            {"role": "system", "content": "You are a helpful assistant."}
        ]
 
    def chat(self, user_message):
        # Add user message to history
        self.messages.append({"role": "user", "content": user_message})
 
        response = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=self.messages  # Send full history!
        )
 
        bot_message = response.choices[0].message.content
 
        # Add bot response to history
        self.messages.append({"role": "assistant", "content": bot_message})
 
        return bot_message
 
# Usage
bot = Chatbot()
print(bot.chat("My name is John"))  # Nice to meet you, John!
print(bot.chat("What's my name?"))  # Your name is John!

Solution 2 - Store in database for persistence:

import sqlite3
 
def save_message(session_id, role, content):
    conn = sqlite3.connect('chat_history.db')
    conn.execute('''
        CREATE TABLE IF NOT EXISTS messages
        (session_id TEXT, role TEXT, content TEXT, timestamp DATETIME DEFAULT CURRENT_TIMESTAMP)
    ''')
    conn.execute(
        'INSERT INTO messages (session_id, role, content) VALUES (?, ?, ?)',
        (session_id, role, content)
    )
    conn.commit()
    conn.close()
 
def get_history(session_id):
    conn = sqlite3.connect('chat_history.db')
    cursor = conn.execute(
        'SELECT role, content FROM messages WHERE session_id = ? ORDER BY timestamp',
        (session_id,)
    )
    messages = [{"role": row[0], "content": row[1]} for row in cursor]
    conn.close()
    return messages

Problem: Chatbot Works Locally But Fails in Production

Symptom: Works on your computer, crashes when deployed.

Solution 1 - Check environment variables:

import os
 
api_key = os.environ.get("OPENAI_API_KEY")
 
if not api_key:
    raise ValueError("OPENAI_API_KEY not set! Check your environment variables.")
 
# Don't do this:
# api_key = "sk-xxxx"  # Hardcoded key - security risk!

Solution 2 - Add proper error handling:

import openai
import logging
 
logging.basicConfig(level=logging.INFO)
logger = logging.getLogger(__name__)
 
def production_chat(messages):
    try:
        response = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=messages,
            timeout=30
        )
        return response.choices[0].message.content
 
    except openai.AuthenticationError:
        logger.error("Invalid API key")
        return "Configuration error. Please contact support."
 
    except openai.RateLimitError:
        logger.warning("Rate limit hit")
        return "I'm getting too many requests. Please wait a moment."
 
    except openai.APIConnectionError:
        logger.error("Connection failed")
        return "Can't reach AI service. Please try again."
 
    except Exception as e:
        logger.exception("Unexpected error")
        return "Something went wrong. Please try again."

Solution 3 - Add health checks:

from flask import Flask, jsonify
 
app = Flask(__name__)
 
@app.route('/health')
def health_check():
    try:
        # Test API connection
        client.models.list()
        return jsonify({"status": "healthy"})
    except Exception as e:
        return jsonify({"status": "unhealthy", "error": str(e)}), 500

Problem: Chatbot Costs Too Much

Symptom: Unexpected high API bills.

Solution 1 - Use cheaper models:

# Expensive
model = "gpt-4"  # ~$30 per 1M tokens
 
# Much cheaper
model = "gpt-4o-mini"  # ~$0.15 per 1M input tokens

Solution 2 - Limit conversation length:

def trim_conversation(messages, max_messages=10):
    system = messages[0]  # Keep system prompt
    recent = messages[-max_messages:]
    return [system] + recent

Solution 3 - Set max_tokens:

response = client.chat.completions.create(
    model="gpt-4o-mini",
    messages=messages,
    max_tokens=150  # Limit response length
)

Solution 4 - Cache common responses:

import hashlib
 
cache = {}
 
def cached_chat(messages):
    # Create cache key from messages
    key = hashlib.md5(str(messages).encode()).hexdigest()
 
    if key in cache:
        return cache[key]
 
    response = client.chat.completions.create(...)
    answer = response.choices[0].message.content
 
    cache[key] = answer
    return answer

Quick Diagnostic Checklist

def diagnose_chatbot():
    issues = []
 
    # 1. Check API key
    if not os.environ.get("OPENAI_API_KEY"):
        issues.append("❌ OPENAI_API_KEY not set")
    else:
        issues.append("✅ API key found")
 
    # 2. Test API connection
    try:
        client.models.list()
        issues.append("✅ API connection working")
    except Exception as e:
        issues.append(f"❌ API connection failed: {e}")
 
    # 3. Test simple chat
    try:
        response = client.chat.completions.create(
            model="gpt-4o-mini",
            messages=[{"role": "user", "content": "Hi"}],
            max_tokens=10
        )
        if response.choices[0].message.content:
            issues.append("✅ Chat completion working")
        else:
            issues.append("❌ Empty response from API")
    except Exception as e:
        issues.append(f"❌ Chat failed: {e}")
 
    return "\n".join(issues)
 
print(diagnose_chatbot())

Need Help With Your Chatbot?

Building production chatbots requires expertise in error handling, scaling, and security. Our team offers:

  • Chatbot architecture design
  • Performance optimization
  • Security audits
  • Custom AI solutions

Get chatbot help

Weekly AI Security & Automation Digest

Get the latest on AI Security, workflow automation, secure integrations, and custom platform development delivered weekly.

No spam. Unsubscribe anytime.

Related Articles

AI Security

OpenAI API Errors: Complete Troubleshooting Guide

Fix OpenAI API errors including rate limits, context length exceeded, invalid API key, timeout errors, and model not found with working code solutions.

•6 min read
AI Security

LangChain Errors: Complete Troubleshooting Guide

Fix LangChain errors including chain invoke failures, memory issues, vector store errors, agent loops, and retrieval problems with practical Python solutions.

•6 min read
AI Security

RAG Not Working? Common Issues and How to Fix Them

Fix RAG (Retrieval Augmented Generation) issues including poor retrieval, wrong answers, slow performance, and embedding problems with practical solutions.

•7 min read
← Back to Blog