ElevenLabs Voice Agent + RAG PDF Upload

Backend serves signed URLs for ElevenLabs agents and PDF upload to Databricks for RAG.

Setup

Backend

pip install -r requirements.txt
cp .env.example .env
# Edit .env with required keys

Frontend

cd frontend
npm install

Run

Terminal 1 – Backend:

python main.py
# or: uvicorn main:app --reload --port 8000

Terminal 2 – Frontend:

cd frontend && npm run dev

Open http://localhost:5173 for voice agent.

API Endpoints

Voice

• GET /signed-url – ElevenLabs signed URL for WebSocket connection

Note: The ElevenLabs WebSocket can close after inactivity or network hiccups. The frontend auto-reconnects once on disconnect and shows a "Reconnect" button if the connection fails.

RAG PDF

• POST /api/cases/{case_id}/pdf – Upload PDF; stores in nyusw.buildathon.rag_pdfs and embeddings in nyusw.buildathon.rag_chunks

Conversation

• POST /conversation-upload – Store conversation text, save to conversation_raw, embed and save to rag_chunks (one chunk per role)

Request body:

{
  "case_id": "case-123",
  "thread_id": "1",
  "Defence": "concatenated defendant arguments...",
  "Plaintiff": "concatenated plaintiff arguments..."
}

Example:

curl -X POST "http://localhost:8000/api/conversation" \
  -H "Content-Type: application/json" \
  -d '{"case_id":"c1","thread_id":"t1","Defence":"defendant text","Plaintiff":"plaintiff text"}'

RAG Query

• POST /api/query – Fetch case chunks, pass to LLM as context, return answer

Request body:

{"case_id": "case-123", "query": "What did the defendant argue?"}

Response: answer (LLM response), chunks_used (case chunks provided as context)

Env vars

ElevenLabs

• ELEVENLABS_API_KEY – ElevenLabs API key
• ELEVENLABS_AGENT_ID – Agent ID

Databricks

• DATABRICKS_HOST – e.g. https://xxx.cloud.databricks.com
• DATABRICKS_TOKEN – Personal access token
• DATABRICKS_HTTP_PATH – SQL warehouse HTTP path (e.g. /sql/1.0/warehouses/xxx)

Gemini

• GEMINI_API_KEY – For 768-d embeddings

Other

• PORT – Backend port (default 8000)

Databricks

Create volume and tables if needed:

• Volume: nyusw.buildathon.rag_pdfs (for PDF storage)
• Table: nyusw.buildathon.rag_chunks (chunk_id, doc_id, source_type, source_uri, chunk_index, chunk_text, updated_at, metadata_json, vector_embeddings array)
• Table: nyusw.buildathon.conversation_raw (event_id, thread_id, tenant_id, role, text, created_at, metadata_json, case_id)