EchoAcess

Inspiration

Over 1.5 million Canadians live with significant vision loss, and the forms they depend on most (banking, transit, government benefits) are the hardest to use. Screen readers choke on nested dropdowns. Tab order is unpredictable. One wrong keystroke resets an entire application.

We asked ourselves: "Who am I leaving out?"

The answer was clear. EchoAccess was built on one idea: filling out a form should be as simple as having a conversation.

How We Built IT

Voice Conversation Pipeline

STEP	SERVICE	WHAT HAPPENS
1. Parse Form	Gemini 2.5 Flash	Raw HTML ingested, every field extracted with plain-English labels, input types identified (text, select, radio, date, etc.)
2. Create Session	Supabase + Backboard.io	Per-user assistant and thread created with row-level security. Returning users matched to existing memory profile
3. Generate Question	Gemini 2.5 Flash	Next unanswered field converted into a natural, conversational voice prompt with context from previous answers
4. Speak Question	ElevenLabs TTS	Question synthesized into natural-sounding speech using ElevenLabs text-to-speech API
5. Capture Answer	ElevenLabs STT	User's spoken response transcribed in real-time. Text input supported as fallback
6. Store Answer	Backboard.io Memory	Answer persisted to long-term user profile. Carries across all forms (bank, transit, CRA)
7. Read Summary	ElevenLabs TTS + Gemini	All answers compiled into plain-English summary, read aloud for confirmation before submissio

Inspiration

We asked ourselves: "Who am I leaving out?"

The answer was clear. EchoAccess was built on one idea: filling out a form should be as simple as having a conversation.

How We Built IT

Voice Conversation Pipeline

STEP	SERVICE	WHAT HAPPENS
1. Parse Form	Gemini 2.5 Flash	Raw HTML ingested, every field extracted with plain-English labels, input types identified (text, select, radio, date, etc.)
2. Create Session	Supabase + Backboard.io	Per-user assistant and thread created with row-level security. Returning users matched to existing memory profile
3. Generate Question	Gemini 2.5 Flash	Next unanswered field converted into a natural, conversational voice prompt with context from previous answers
4. Speak Question	ElevenLabs TTS	Question synthesized into natural-sounding speech using ElevenLabs text-to-speech API
5. Capture Answer	ElevenLabs STT	User's spoken response transcribed in real-time. Text input supported as fallback
6. Store Answer	Backboard.io Memory	Answer persisted to long-term user profile. Carries across all forms (bank, transit, CRA)
7. Read Summary	ElevenLabs TTS + Gemini	All answers compiled into plain-English summary, read aloud for confirmation before submissio

Per-User Memory System

LAYER	TECHNOLOGY	WHAT IT DOES
Auth	Supabase Auth (JWT)	Secure sign-up/login with email. JWT passed on every API call
Session Mapping	Supabase PostgreSQL	`user_sessions` table maps each `user_id` to a dedicated Backboard `assistant_id` + `thread_id`
Isolation	Row-Level Security	Supabase RLS ensures no user can access another user's session or memory
Long-Term Memory	Backboard.io REST API	Stores name, email, address, SIN, and more. Pre-fills across forms on return visits

Frontend Accessibility Stack

FEATURE	IMPLEMENTATION	WHY IT MATTERS
Voice Output	ElevenLabs TTS API	Every question read aloud in natural-sounding voice, no screen required
Voice Input	ElevenLabs STT API	Hands-free answering by speaking naturally
Screen Reader Support	`aria-live="polite"` on chat log	New messages announced automatically
Interactive Labels	`aria-label` on all controls	Every button and input identified by assistive tech
Progress Tracking	`aria-current="step"` on active field	Users always know where they are in the form
Keyboard Navigation	Full Tab + Enter support	Entire flow navigable without a mouse
Animations	Framer Motion	Smooth transitions that respect `prefers-reduced-motion`

Tech Stack

LAYER	TECHNOLOGY
Backend	FastAPI + Python 3.11+
Auth	Supabase Auth (JWT)
Database	Supabase PostgreSQL
LLM	Gemini 2.5 Flash
Voice	ElevenLabs TTS + STT
Memory	Backboard.io
Frontend	React 19 + Vite + TypeScript
UI	shadcn/ui + Tailwind CSS v4 + Framer Motion

Demo Forms

FORM	FIELDS	REAL-WORLD USE CASE
TD Bank Account Application	Personal info, SIN, employment, income	Opening a bank account without visiting a branch
TTC Disability Discount Card	Accessibility needs, ODSP/CPP status, disability type	Applying for subsidized transit in Toronto
CRA Benefits Application	SIN, marital status, dependents, direct deposit	Claiming government benefits independently

API Reference

METHOD	ENDPOINT	AUTH	DESCRIPTION
GET	`/api/health`	—	Health check + Backboard status
GET	`/api/forms`	Bearer	List available forms
POST	`/api/parse-form`	Bearer	Parse form HTML into structured fields
POST	`/api/new-session`	Bearer	Create or reuse per-user Backboard thread
POST	`/api/chat`	Bearer	Generate next conversational question
POST	`/api/save-answer`	Bearer	Save field answer to memory
GET	`/api/user-profile`	Bearer	Retrieve stored user profile
POST	`/api/submit-form`	Bearer	Generate plain-English summary

Per-User Memory System

LAYER	TECHNOLOGY	WHAT IT DOES
Auth	Supabase Auth (JWT)	Secure sign-up/login with email. JWT passed on every API call
Session Mapping	Supabase PostgreSQL	`user_sessions` table maps each `user_id` to a dedicated Backboard `assistant_id` + `thread_id`
Isolation	Row-Level Security	Supabase RLS ensures no user can access another user's session or memory
Long-Term Memory	Backboard.io REST API	Stores name, email, address, SIN, and more. Pre-fills across forms on return visits

Frontend Accessibility Stack

FEATURE	IMPLEMENTATION	WHY IT MATTERS
Voice Output	ElevenLabs TTS API	Every question read aloud in natural-sounding voice, no screen required
Voice Input	ElevenLabs STT API	Hands-free answering by speaking naturally
Screen Reader Support	`aria-live="polite"` on chat log	New messages announced automatically
Interactive Labels	`aria-label` on all controls	Every button and input identified by assistive tech
Progress Tracking	`aria-current="step"` on active field	Users always know where they are in the form
Keyboard Navigation	Full Tab + Enter support	Entire flow navigable without a mouse
Animations	Framer Motion	Smooth transitions that respect `prefers-reduced-motion`

Tech Stack

LAYER	TECHNOLOGY
Backend	FastAPI + Python 3.11+
Auth	Supabase Auth (JWT)
Database	Supabase PostgreSQL
LLM	Gemini 2.5 Flash
Voice	ElevenLabs TTS + STT
Memory	Backboard.io
Frontend	React 19 + Vite + TypeScript
UI	shadcn/ui + Tailwind CSS v4 + Framer Motion

Demo Forms

FORM	FIELDS	REAL-WORLD USE CASE
TD Bank Account Application	Personal info, SIN, employment, income	Opening a bank account without visiting a branch
TTC Disability Discount Card	Accessibility needs, ODSP/CPP status, disability type	Applying for subsidized transit in Toronto
CRA Benefits Application	SIN, marital status, dependents, direct deposit	Claiming government benefits independently

API Reference

METHOD	ENDPOINT	AUTH	DESCRIPTION
GET	`/api/health`	—	Health check + Backboard status
GET	`/api/forms`	Bearer	List available forms
POST	`/api/parse-form`	Bearer	Parse form HTML into structured fields
POST	`/api/new-session`	Bearer	Create or reuse per-user Backboard thread
POST	`/api/chat`	Bearer	Generate next conversational question
POST	`/api/save-answer`	Bearer	Save field answer to memory
GET	`/api/user-profile`	Bearer	Retrieve stored user profile
POST	`/api/submit-form`	Bearer	Generate plain-English summary

more projects

Spotify Case Study

DATA SCIENCE / MACHINE LEARNING

Spotify Case Study

DATA SCIENCE / MACHINE LEARNING

Machine Learning Prediction of Age and Alzheimer's Disease from DNA Methylation Profiles

DATA SCIENCE / MACHINE LEARNING

Machine Learning Prediction of Age and Alzheimer's Disease from DNA Methylation Profiles

DATA SCIENCE / MACHINE LEARNING