OpenAI Integration: Build a RAG Chatbot That Answers Questions From Your PDFs
•Cocoding Team
OpenAI Integration: Build a RAG Chatbot That Answers Questions From Your PDFs
Large Language Models are powerful, but they have a critical flaw: hallucinations. They confidently make up information that sounds plausible but is completely wrong. What if you could build an AI chatbot that ONLY answers from your documents, with zero made-up facts? With Cocoding AI's OpenAI integration, you can build a Retrieval-Augmented Generation (RAG) system that's grounded in truth.
The Hallucination Problem
Standard chatbots have serious limitations:
- Invented facts - LLMs confidently state false information
- Outdated knowledge - Training data has cutoff dates
- No source citations - Can't verify where answers come from
- Generic responses - No access to your specific documents
- Compliance risks - Wrong information can have legal consequences
RAG solves these problems by grounding AI responses in your actual documents.
What is RAG (Retrieval-Augmented Generation)?
RAG is a technique that combines:
- Retrieval: Find relevant information from your documents
- Augmentation: Add that information to the AI's context
- Generation: AI generates answers using ONLY retrieved content
Traditional Chatbot:
User Question → LLM → Answer (possibly hallucinated)
RAG Chatbot:
User Question → Search Documents → Find Relevant Chunks
→ Send Chunks + Question to LLM → Grounded Answer
Result: Answers that are factual, verifiable, and cite sources.
What You Can Build
Complete RAG Chatbot Application
Cocoding AI generates a full-featured system:
Application Features:
├── Document Management
│ ├── PDF drag-and-drop upload
│ ├── Document library view
│ ├── Delete documents
│ └── Processing status
├── AI Chat Interface
│ ├── Natural language questions
│ ├── Source citations
│ ├── Conversation history
│ └── Follow-up questions
├── Vector Search Engine
│ ├── OpenAI embeddings
│ ├── Semantic similarity search
│ ├── Configurable relevance
│ └── PostgreSQL pgvector
└── User Authentication
├── Account creation
├── Secure login
└── Personal document storage
Technical Architecture
Tech Stack:
├── Frontend (React)
│ ├── Document upload UI
│ ├── Chat interface
│ └── Document management
├── Backend (NestJS)
│ ├── PDF text extraction
│ ├── OpenAI embeddings API
│ ├── Groq LLM integration
│ └── Vector similarity search
└── Database (PostgreSQL + pgvector)
├── User accounts
├── Document metadata
└── Vector embeddings
How the RAG System Works
The 8-Step Process
Document Ingestion:
┌─────────────────────────────────────────────────────────────────┐
│ 1. Upload PDF │
│ ↓ │
│ 2. Extract text from PDF │
│ ↓ │
│ 3. Split into ~1000 character chunks │
│ ↓ │
│ 4. Generate OpenAI embeddings for each chunk │
│ ↓ │
│ 5. Store embeddings in PostgreSQL with pgvector │
└─────────────────────────────────────────────────────────────────┘
Query Processing:
┌─────────────────────────────────────────────────────────────────┐
│ 6. User asks question → Generate embedding for query │
│ ↓ │
│ 7. Semantic search → Find top 5 most similar chunks │
│ ↓ │
│ 8. Send chunks + question to Groq LLM → Return grounded answer │
└─────────────────────────────────────────────────────────────────┘
Vector Embeddings Explained
Embeddings convert text into numerical representations:
Text: "The quarterly revenue increased by 15%"
↓ OpenAI Embedding API
Embedding: [0.023, -0.156, 0.892, ..., 0.445] (1536 dimensions)
Similar concepts have similar embeddings, enabling semantic search—finding relevant content even when exact words don't match.
Why Semantic Search Beats Keyword Search
| Query | Keyword Search | Semantic Search |
|---|---|---|
| "How much did we make?" | No results | Finds "revenue", "profit", "earnings" |
| "Team size" | No results | Finds "employees", "headcount", "staff" |
| "Product launch date" | Partial match | Finds "release", "go-live", "deployment" |
Step-by-Step Setup Guide
Step 1: Get Your OpenAI API Key
- Visit platform.openai.com
- Sign up or log in
- Go to API Keys
- Click "Create new secret key"
- Copy your key:
sk-proj-xxxxx
Cost: ~$0.02 per 1 million tokens (extremely affordable)
- 100-page PDF ≈ $0.001 to embed
Step 2: Get Your Groq API Key (Free)
- Visit console.groq.com
- Create an account
- Go to API Keys
- Generate a key:
gsk_xxxxx
Cost: Completely FREE with generous limits
Step 3: Generate Your Application
Describe your needs to Cocoding AI:
Build a RAG chatbot application:
- React frontend with document upload and chat interface
- NestJS backend with PostgreSQL and pgvector
- PDF text extraction and chunking
- OpenAI embeddings for vector search
- Groq LLM (Llama 3.3) for chat responses
- User authentication with JWT
- Source citations in responses
- Document management (upload, view, delete)
Step 4: Configure Environment
Add your API keys:
# backend/.env
OPENAI_API_KEY=sk-proj-xxxxxxxxxxxxxxxxxxxxx
GROQ_API_KEY=gsk_xxxxxxxxxxxxxxxxxxxxx
DATABASE_URL=postgresql://user:password@localhost:5432/ragbot
JWT_SECRET=your-secure-jwt-secret
Step 5: Set Up pgvector
Enable vector extensions in PostgreSQL:
-- Enable pgvector extension
CREATE EXTENSION IF NOT EXISTS vector;
-- Create embeddings table
CREATE TABLE document_chunks (
id SERIAL PRIMARY KEY,
document_id INTEGER REFERENCES documents(id),
content TEXT NOT NULL,
embedding vector(1536),
chunk_index INTEGER
);
-- Create index for fast similarity search
CREATE INDEX ON document_chunks
USING ivfflat (embedding vector_cosine_ops)
WITH (lists = 100);
Step 6: Test Your System
- Upload a PDF document
- Wait for processing (embeddings generation)
- Ask a question about the document
- Verify the answer includes source citations
Building the User Experience
Document Upload Interface
┌──────────────────────────────────────────────────────┐
│ Upload Documents │
├──────────────────────────────────────────────────────┤
│ │
│ ┌──────────────────────────────────────────────┐ │
│ │ │ │
│ │ 📄 Drag and drop PDFs here │ │
│ │ │ │
│ │ or click to browse │ │
│ │ │ │
│ └──────────────────────────────────────────────┘ │
│ │
│ Uploaded Documents: │
│ ┌──────────────────────────────────────────────┐ │
│ │ 📄 Company_Handbook.pdf ✅ Ready [🗑️] │ │
│ │ 📄 Q4_Financial_Report.pdf ✅ Ready [🗑️] │ │
│ │ 📄 Product_Manual.pdf ⏳ Processing │ │
│ └──────────────────────────────────────────────┘ │
│ │
└──────────────────────────────────────────────────────┘
Chat Interface
┌──────────────────────────────────────────────────────┐
│ Document Q&A │
├──────────────────────────────────────────────────────┤
│ │
│ 👤 What is our vacation policy? │
│ │
│ 🤖 According to the Company Handbook, employees │
│ are entitled to: │
│ │
│ • 15 days of paid vacation per year │
│ • 5 additional days after 3 years of service │
│ • Unused days can roll over (max 5 days) │
│ │
│ 📚 Sources: │
│ - Company_Handbook.pdf (page 23) │
│ - Company_Handbook.pdf (page 24) │
│ │
│ ───────────────────────────────────────────────── │
│ │
│ 👤 How do I request time off? │
│ │
│ 🤖 To request time off, follow these steps: │
│ 1. Submit request through HR portal │
│ 2. Request must be submitted 2 weeks in advance │
│ 3. Manager approval required for 3+ days │
│ │
│ 📚 Sources: │
│ - Company_Handbook.pdf (page 25) │
│ │
├──────────────────────────────────────────────────────┤
│ ┌──────────────────────────────────────────────┐ │
│ │ Ask a question about your documents... │ │
│ └──────────────────────────────────────────────┘ │
│ [ Send ] │
└──────────────────────────────────────────────────────┘
Configuration Options
Embedding Models
| Model | Dimensions | Cost | Best For |
|---|---|---|---|
| text-embedding-3-small | 1536 | $0.02/1M tokens | Cost-effective |
| text-embedding-3-large | 3072 | $0.13/1M tokens | Maximum accuracy |
| text-embedding-ada-002 | 1536 | $0.10/1M tokens | Legacy support |
LLM Options (via Groq)
| Model | Speed | Quality | Best For |
|---|---|---|---|
| llama-3.3-70b | Fast | Excellent | General use |
| llama-3.1-405b | Medium | Best | Complex reasoning |
| mixtral-8x7b | Very fast | Good | Quick responses |
| gemma2-9b | Fastest | Good | High volume |
Chunking Parameters
| Parameter | Default | Range | Impact |
|---|---|---|---|
| Chunk size | 1000 chars | 500-2000 | Larger = more context per chunk |
| Chunk overlap | 200 chars | 0-500 | Higher = better continuity |
| Top K results | 5 | 3-10 | More = broader context |
Use Cases
Knowledge Base / Help Desk
Documents: Product manuals, FAQs, support articles Use: Customer self-service, support agent assistance
Example:
- "How do I reset my password?"
- "What's the return policy?"
- "How do I set up two-factor authentication?"
Legal Document Analysis
Documents: Contracts, policies, regulations Use: Quick clause lookup, compliance checking
Example:
- "What are the termination conditions in this contract?"
- "Does this policy comply with GDPR?"
- "What are our liability limits?"
Research Assistant
Documents: Academic papers, reports, studies Use: Literature review, fact-checking
Example:
- "What methodology did the Smith study use?"
- "What were the key findings on customer retention?"
- "How does this compare to previous research?"
Employee Onboarding
Documents: Handbooks, procedures, training materials Use: New hire questions, policy lookup
Example:
- "What benefits am I eligible for?"
- "How do I submit expense reports?"
- "What's the dress code?"
Technical Documentation
Documents: API docs, architecture guides, runbooks Use: Developer assistance, troubleshooting
Example:
- "How do I authenticate with the API?"
- "What's the database schema for users?"
- "How do I deploy to production?"
Cost Analysis
Extremely Affordable
| Document Size | Embedding Cost | Chat Cost |
|---|---|---|
| 10-page PDF | $0.0001 | FREE (Groq) |
| 100-page PDF | $0.001 | FREE (Groq) |
| 500-page PDF | $0.005 | FREE (Groq) |
Monthly Cost Examples
| Usage Level | Documents | Questions/Month | Total Cost |
|---|---|---|---|
| Light | 50 pages | 500 | < $0.50 |
| Moderate | 500 pages | 5,000 | < $2.00 |
| Heavy | 5,000 pages | 50,000 | < $10.00 |
Compare to: Enterprise knowledge base tools at $50-500/month
Advanced Features
Multi-Document Search
Query across all uploaded documents:
"Compare the vacation policies in the US and UK handbooks"
→ Retrieves chunks from both documents
→ Synthesizes comparison answer
Conversation Memory
Maintain context across questions:
User: "What's the refund policy?"
Bot: "Refunds are available within 30 days..."
User: "What about for digital products?"
Bot: "For digital products specifically, the policy states..."
Custom Prompts
Tune the AI's response style:
System Prompt Options:
├── Concise: Brief, bullet-point answers
├── Detailed: Comprehensive explanations
├── Technical: Include technical details
├── Simple: Plain language, no jargon
└── Formal: Professional business tone
Source Highlighting
Show exactly where answers come from:
Answer with highlights:
"According to Section 4.2 of the Employee Handbook:
'[Employees are entitled to 15 days of paid vacation]
per calendar year, with [5 additional days granted]
after three years of continuous employment.'"
Security Best Practices
API Key Protection
- Store keys in environment variables only
- Never commit keys to version control
- Rotate keys periodically
- Set usage limits in OpenAI dashboard
Document Security
- User authentication required
- Documents isolated per user
- Encryption at rest
- Secure file handling
Production Checklist
- Change JWT secret to strong random value
- Set CORS to your domain only
- Enable rate limiting
- Use HTTPS everywhere
- Monitor API usage
- Set up database backups
Comparing Solutions
| Feature | Cocoding + OpenAI | ChatGPT Plus | Enterprise RAG |
|---|---|---|---|
| Custom documents | Yes | Limited | Yes |
| Source citations | Yes | Sometimes | Yes |
| Data privacy | Your servers | OpenAI servers | Your servers |
| Monthly cost | < $10 | $20/user | $1000s |
| Customization | Full | None | Limited |
| Setup time | Minutes | N/A | Weeks |
Getting Started Today
Build your knowledge assistant:
- Sign up at cocoding.ai
- Get OpenAI and Groq API keys
- Describe your RAG chatbot needs
- Generate your application
- Upload documents and start asking questions
Conclusion
The era of AI hallucinations doesn't have to be your reality. With Cocoding AI's OpenAI integration, you can build a RAG chatbot that only answers from your documents—no made-up facts, no outdated information, just accurate answers with source citations.
Whether you're building a customer support bot, internal knowledge base, research assistant, or document Q&A system, RAG technology ensures your AI is grounded in truth.
For less than the cost of a coffee per month, you can have an AI assistant that knows your documents inside and out.
Stop hallucinating. Start retrieving.
Ready to build your document-powered chatbot? Our team is here to help you create the perfect RAG solution.