diff --git a/Dockerfile b/Dockerfile index 2684887..38b5634 100644 --- a/Dockerfile +++ b/Dockerfile @@ -9,7 +9,7 @@ COPY website/ . RUN mkdir -p ../backend && bun run build # Backend and final image -FROM python:3.9-slim +FROM python:3.10-slim RUN apt-get update && \ apt-get install -y ffmpeg libgl1 libglib2.0-0 && \ @@ -22,6 +22,7 @@ COPY backend/requirements.txt . RUN pip install --no-cache-dir -r requirements.txt COPY backend/ . +COPY scripts/ /scripts/ # Copy static build from builder COPY --from=builder /project/backend/static /app/static diff --git a/README.md b/README.md index db77cc7..2323a83 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,7 @@
-HushMap bridges the gap between hardware sensors and top-tier artificial intelligence pipelines (e.g. **Terp AI**, **ElevenLabs**, **YOLOv8**), delivering a seamless real-time learning assistant combined with live noise and occupancy metrics. +HushMap bridges the gap between hardware sensors and top-tier artificial intelligence pipelines (e.g. **Terp AI**, **ElevenLabs**, **YOLOv8**, and **Gemini**), delivering a seamless real-time learning assistant combined with live noise and occupancy metrics. --- @@ -34,8 +34,9 @@ A responsive, high-fidelity PWA frontend written in Svelte 5 and styled seamless A blazing fast asynchronous HTTP server facilitating audio chunking and sensor metrics logic over full-duplex sockets. * **Core Capabilities**: * **Voice Socket Pipelining**: WebSockets (`/ws/voice`) that hook incoming 16-bit PCM arrays into `faster-whisper`. - * **LLM Context Augmentation**: Seamlessly aggregates live MongoDB noise statistics (Decibel levels per location) to feed contextual history to the TerpAI engine! + * **LLM Context Augmentation**: Seamlessly aggregates live noise statistics (Decibel levels per location) to feed contextual history to the AI engine. Uses **TerpAI** with a seamless fallback to **Gemini 2.5 Flash** if TerpAI is unavailable! * **Computer Vision Endpoint**: Exposes a `YOLOv8` tensor API (`/api/vision/room-status`) to parse webcam imagery, pinpoint seating capacities, and locate available chairs algorithmically. + * **Dynamic Data Source**: Data can either be fetched in real-time from a **MongoDB** database, or simulated on-the-fly via an in-memory generator depending on the `USE_DB` environment flag. * **Setup**: ```bash cd backend @@ -62,4 +63,4 @@ docker-compose up --build --- -For IoT clients, update `/m5go/main.py` explicitly to broadcast to your running router IP namespace matching your specific VLAN. +For IoT clients, update `/m5go/main.py` explicitly to broadcast to your running router IP namespace matching your specific VLAN. \ No newline at end of file diff --git a/backend/README.md b/backend/README.md index 0ef4464..053d3da 100644 --- a/backend/README.md +++ b/backend/README.md @@ -14,7 +14,7 @@ ## Setup Instructions ### Prerequisites -1. **Python 3.9+** is strictly recommended to support asynchronous typing paradigms. +1. **Python 3.10+** is strictly recommended to support asynchronous typing paradigms. 2. **FFmpeg** must be successfully registered onto your OS PATH environments. This engine handles the core conversions decoding MP3 output arrays into 16-bit, 16kHz Mono arrays natively required for browser contexts: - **Ubuntu/Debian**: `sudo apt install ffmpeg` - **macOS**: `brew install ffmpeg` @@ -33,16 +33,20 @@ pip install -r requirements.txt ### Configuration Tokens -Provide runtime keys securely targeting TerpAI context queues and ElevenLabs synthesized avatars within a `.env` dotfile: +Provide runtime keys securely targeting TerpAI context queues, Gemini Fallback, and ElevenLabs synthesized avatars within a `.env` dotfile: ```ini ELEVENLABS_API_KEY=sk_... ELEVENLABS_VOICE_ID=JBFqnCBsd6RMkjVDRZzb TERP_AI_BEARER_TOKEN=eyJhbGciOiJSUz... TERP_AI_CONVERSATION_ID=37fa27cc-... +GEMINI_API_KEY=AIza... MONGODB_URI=mongodb+srv://... +USE_DB=false ``` +*Note: `USE_DB` controls whether the application connects to MongoDB (`true`) or uses on-the-fly generated in-memory data for demonstrations (`false`).* + To invoke the engine, simply execute Uvicorn across your `0.0.0.0` loopback: ```bash @@ -58,7 +62,7 @@ uvicorn server:app --host 0.0.0.0 --port 8000 This WebSocket proxy establishes a fully integrated multi-turn communication bridge seamlessly interacting between Edge node Hardware APIs (ESP32/M5GO/Browsers) and NLP architectures. 1. **Int16 Byte Array Exchange**: Devices connect to `ws://:8000/ws/voice` and push raw binary frames asynchronously over the socket. -2. **Contextual Augmentation**: The server waits for the `"stop_listening"` payload event to signify a completed audio snippet. That float array is cast through `faster-whisper` and combined seamlessly with real-time `MongoDB` decibel tracking telemetry parameters natively attached into the `TerpAI` user conversation chunk. +2. **Contextual Augmentation**: The server waits for the `"stop_listening"` payload event to signify a completed audio snippet. That float array is cast through `faster-whisper` and combined seamlessly with real-time decibel tracking telemetry parameters natively attached into the AI user conversation chunk. We utilize **Terp AI** with an automatic, seamless fallback to **Gemini 2.5 Flash** if the primary Terp service is unavailable. 3. **TTS Pipeline Rendering**: Output predictions are caught instantly, forwarded natively into the `ElevenLabs` TTS interface rendering `pcm_16000` wav codecs, and alerted back down to clients using a `tts_ready` dispatcher. ### Tensor Vision Endpoints (`/api/vision/room-status`) @@ -90,8 +94,8 @@ Leveraging OpenCV bindings layered beneath a YOLOv8-driven bounding box topology ## Database Registries -* `GET /api/study-rooms/history`: Pulls the active global repository of logged architectural noise measurements captured universally within the preceding 24 hours. Data payloads correspond geographically mapping `GeoJSON` nodes to front-end Mapbox topologies. -* `GET /api/study-rooms`: Pulls generic unstructured noise lists directly unfiltered from Cosmos bounds. +* `GET /api/study-rooms/history`: Pulls the active global repository of logged architectural noise measurements captured universally within the preceding 24 hours. (Uses MongoDB or in-memory generated data based on the `USE_DB` flag). +* `GET /api/study-rooms`: Pulls generic unstructured noise lists. > [!IMPORTANT] -> The browser frontend strictly configures standard Web Audio API's `ScriptProcessorNode` interfaces routing data synchronously to this backend! Wait to close down pipelines until *after* all WS queues have successfully been delivered. +> The browser frontend strictly configures standard Web Audio API's `ScriptProcessorNode` interfaces routing data synchronously to this backend! Wait to close down pipelines until *after* all WS queues have successfully been delivered. \ No newline at end of file diff --git a/backend/requirements.txt b/backend/requirements.txt index 5e2ca77..c33ae4b 100644 --- a/backend/requirements.txt +++ b/backend/requirements.txt @@ -9,4 +9,6 @@ ultralytics opencv-python-headless scipy pymongo -pydantic +certifi +google-genai +pydantic \ No newline at end of file diff --git a/backend/server.py b/backend/server.py index 9baf13b..14fadad 100644 --- a/backend/server.py +++ b/backend/server.py @@ -8,9 +8,16 @@ import requests import json import base64 import urllib.request +import time +import sys -# import ssl -# ssl._create_default_https_context = ssl._create_unverified_context +# Add scripts directory to path to import fake data generator +sys.path.append(os.path.join(os.path.dirname(os.path.dirname(__file__)), 'scripts')) +try: + from generate_fake_data import get_fake_data +except ImportError: + print("Warning: Could not import get_fake_data from scripts/generate_fake_data.py") + def get_fake_data(locations): return [] from fastapi import FastAPI, WebSocket, WebSocketDisconnect, File, UploadFile from fastapi.staticfiles import StaticFiles @@ -21,6 +28,7 @@ from datetime import datetime, timedelta from pymongo import MongoClient from dotenv import load_dotenv from vision import analyze_room_image +from google import genai from fastapi.middleware.cors import CORSMiddleware @@ -37,11 +45,30 @@ app.add_middleware( ) +USE_DB = os.getenv("USE_DB", "false").lower() == "true" + import certifi -MONGO_URI = os.getenv("MONGODB_URI", "mongodb://localhost:27017/") -mongo_client = MongoClient(MONGO_URI, tlsCAFile=certifi.where()) -db = mongo_client.study_buddy_db -study_rooms_collection = db.study_rooms +if USE_DB: + MONGO_URI = os.getenv("MONGODB_URI", "mongodb://localhost:27017/") + mongo_client = MongoClient(MONGO_URI, tlsCAFile=certifi.where()) + db = mongo_client.study_buddy_db + study_rooms_collection = db.study_rooms +else: + print("Running in in-memory mode. MongoDB is disabled. Set USE_DB=true to enable.") + +# Fake data cache +_fake_data_cache = None +_fake_data_cache_time = 0 + +def _get_cached_fake_data(): + global _fake_data_cache, _fake_data_cache_time + now = time.time() + # Cache for 5 minutes (300 seconds) + if _fake_data_cache is None or now - _fake_data_cache_time > 300: + # Assuming UMD_LOCATIONS is defined further down, but we can just use the global + _fake_data_cache = get_fake_data(UMD_LOCATIONS) + _fake_data_cache_time = now + return _fake_data_cache class GeoJSONPoint(BaseModel): @@ -159,7 +186,7 @@ def _transcribe_pcm(pcm_data: bytes, sample_rate: int = SAMPLE_RATE) -> str: os.remove(tmp_path) def get_terp_ai_response(message: str) -> str: - """Send text to Terp AI and return the full response.""" + """Send text to Terp AI and return the full response. Fallback to Gemini if needed.""" url = f"https://terpai.umd.edu/api/internal/userConversations/{CONVERSATION_ID}/segments" payload = { "question": message, @@ -175,7 +202,7 @@ def get_terp_ai_response(message: str) -> str: full_response = "" event = None try: - resp = requests.post(url, json=payload, headers=HEADERS, stream=True, timeout=30, verify=False) + resp = requests.post(url, json=payload, headers=HEADERS, stream=True, timeout=10, verify=False) resp.raise_for_status() for line in resp.iter_lines(decode_unicode=True): if not line: @@ -188,13 +215,28 @@ def get_terp_ai_response(message: str) -> str: if event == "response-updated": full_response += decoded resp.close() + if full_response: + return full_response + else: + raise Exception("Empty response from Terp AI") except Exception as e: - print(f"Terp AI error: {e}") - return "I am sorry, there was an error connecting to Terp AI." + print(f"Terp AI error, falling back to Gemini: {e}") + try: + api_key = os.getenv("GEMINI_API_KEY") + if not api_key or api_key == "your_gemini_api_key": + return "Terp AI is unavailable and Gemini fallback is not configured." + client = genai.Client(api_key=api_key) + response = client.models.generate_content( + model='gemini-2.5-flash', + contents=message + ) + return response.text + except Exception as gemini_e: + print(f"Gemini fallback error: {gemini_e}") + return "I am sorry, both Terp AI and the Gemini fallback encountered an error." - return full_response -def _convert_to_pcm(audio_data: bytes, input_format: str = "mp3") -> bytes | None: +def _convert_to_pcm(audio_data: bytes, input_format: str = "mp3") -> Optional[bytes]: """Convert audio data to 16-bit 16 kHz mono PCM using ffmpeg.""" try: result = subprocess.run( @@ -225,7 +267,7 @@ def _convert_to_pcm(audio_data: bytes, input_format: str = "mp3") -> bytes | Non print("ffmpeg conversion timed out") return None -def _generate_tts(text: str) -> bytes | None: +def _generate_tts(text: str) -> Optional[bytes]: """Generate speech audio from text using ElevenLabs TTS API.""" api_key = os.getenv("ELEVENLABS_API_KEY") voice_id = os.getenv("ELEVENLABS_VOICE_ID", "JBFqnCBsd6RMkjVDRZzb") @@ -373,23 +415,36 @@ UMD_LOCATIONS = [ def get_latest_locations_context() -> str: """Fetch the latest stats for each known location to feed as AI context.""" - twenty_four_hours_ago = datetime.utcnow() - timedelta(hours=24) - pipeline = [ - {"$match": {"date": {"$gte": twenty_four_hours_ago}}}, - {"$sort": {"date": -1}}, - {"$group": { - "_id": "$room_id", - "latest_db": {"$first": "$db"}, - "time": {"$first": "$date"} - }} - ] - latest_stats = list(study_rooms_collection.aggregate(pipeline)) + room_dict = {loc["id"]: loc["name"] for loc in UMD_LOCATIONS} + + if USE_DB: + twenty_four_hours_ago = datetime.utcnow() - timedelta(hours=24) + pipeline = [ + {"$match": {"date": {"$gte": twenty_four_hours_ago}}}, + {"$sort": {"date": -1}}, + {"$group": { + "_id": "$room_id", + "latest_db": {"$first": "$db"}, + "time": {"$first": "$date"} + }} + ] + latest_stats = list(study_rooms_collection.aggregate(pipeline)) + else: + fake_data = _get_cached_fake_data() + latest_stats_map = {} + # Data is naturally sorted chronologically in our generator, so reverse it + for d in reversed(fake_data): + if d["room_id"] not in latest_stats_map: + latest_stats_map[d["room_id"]] = { + "_id": d["room_id"], + "latest_db": d["db"], + "time": d["date"] + } + latest_stats = list(latest_stats_map.values()) if not latest_stats: return "No recent location noise stats available today." - room_dict = {loc["id"]: loc["name"] for loc in UMD_LOCATIONS} - lines = ["Latest Study Room Stats:"] for stat in latest_stats: room_id = stat.get("_id") @@ -426,8 +481,12 @@ async def create_study_room_data(data: StudyRoomData): if not data.date: data.date = datetime.utcnow() doc = data.dict() - result = study_rooms_collection.insert_one(doc) - return {"id": str(result.inserted_id), "room_id": data.room_id, "status": "success"} + + if USE_DB: + result = study_rooms_collection.insert_one(doc) + return {"id": str(result.inserted_id), "room_id": data.room_id, "status": "success"} + else: + return {"id": "dummy_id", "room_id": data.room_id, "status": "success (in-memory, not saved)"} @app.get("/api/study-rooms") async def get_study_room_data(): @@ -437,11 +496,17 @@ async def get_study_room_data(): @app.get("/api/study-rooms/history") async def get_study_room_history(): """Get all study room data from the last 24 hours.""" - twenty_four_hours_ago = datetime.utcnow() - timedelta(hours=24) - rooms = list(study_rooms_collection.find( - {"date": {"$gte": twenty_four_hours_ago}}, - {"_id": 0} - ).sort("date", -1)) + if USE_DB: + twenty_four_hours_ago = datetime.utcnow() - timedelta(hours=24) + rooms = list(study_rooms_collection.find( + {"date": {"$gte": twenty_four_hours_ago}}, + {"_id": 0} + ).sort("date", -1)) + else: + rooms = _get_cached_fake_data() + # Ensure we don't leak ObjectIds or non-serializable stuff + # Dates are naturally sorted but let's reverse them to match MongoDB behavior (newest first) + rooms = list(reversed(rooms)) return {"data": rooms} @app.get("/{full_path:path}") diff --git a/example.env b/example.env index 2fc80e3..b3c705e 100644 --- a/example.env +++ b/example.env @@ -1 +1,6 @@ PORT=3000 +USE_DB=false +GEMINI_API_KEY=your_gemini_api_key +ELEVENLABS_API_KEY=your_elevenlabs_api_key_here +ELEVENLABS_VOICE_ID=JBFqnCBsd6RMkjVDRZzb +TERP_AI_BEARER_TOKEN=your_terp_ai_token diff --git a/scripts/generate_fake_data.py b/scripts/generate_fake_data.py index 1b5c67e..2fa65b1 100644 --- a/scripts/generate_fake_data.py +++ b/scripts/generate_fake_data.py @@ -1,24 +1,6 @@ import random from datetime import datetime, timedelta -from pymongo import MongoClient - - -MONGO_URI = "mongodb+srv://SarayuJ:[EMAIL_ADDRESS]/testing" -client = MongoClient(MONGO_URI) -db = client.study_buddy_db -collection = db.study_rooms - -UMD_LOCATIONS = [ - { "id": 'esj', "name": 'Edward St. John (ESJ)', "lng": -76.94209511596014, "lat": 38.987133359608755 }, - { "id": 'mckeldin', "name": 'McKeldin Library', "lng": -76.94494907523277, "lat": 38.986021017749366 }, - { "id": 'hornbake', "name": 'Hornbake Library', "lng": -76.94161787005467, "lat": 38.988233373664826 }, - { "id": 'stem', "name": 'STEM Library', "lng": -76.93942003731279, "lat": 38.988991437126195 }, - { "id": 'clarice', "name": 'Clarice Library', "lng": -76.9500912552473, "lat": 38.990547823732285 }, - { "id": 'yahentamitsi', "name": 'Yahentamitsi', "lng": -76.9448027183373, "lat": 38.99108961575231 }, - { "id": 'iribe', "name": 'Iribe', "lng": -76.93643838603555, "lat": 38.98933701397555 }, - { "id": 'reckord', "name": 'Reckord Armory', "lng": -76.93897470250619, "lat": 38.98609556181066 }, - { "id": 'stamp', "name": 'Stamp Student Union', "lng": -76.94473083972326, "lat": 38.988130238874874 } -] +import os def get_db_for_time_and_location(hour, loc_id): """ @@ -28,7 +10,6 @@ def get_db_for_time_and_location(hour, loc_id): base_db = 40.0 if loc_id in ['mckeldin', 'esj']: - if 10 <= hour <= 16: base_db = 75.0 elif 17 <= hour <= 22: @@ -37,7 +18,6 @@ def get_db_for_time_and_location(hour, loc_id): base_db = 45.0 elif loc_id in ['stem', 'iribe']: - if 14 <= hour <= 20: base_db = 70.0 elif 9 <= hour <= 13: @@ -46,7 +26,6 @@ def get_db_for_time_and_location(hour, loc_id): base_db = 42.0 elif loc_id == 'stamp': - if 12 <= hour <= 14 or 17 <= hour <= 19: base_db = 85.0 elif 10 <= hour <= 21: @@ -55,33 +34,24 @@ def get_db_for_time_and_location(hour, loc_id): base_db = 50.0 else: - - if 9 <= hour <= 18: base_db = 60.0 else: base_db = 45.0 - noise = random.uniform(-5.0, 5.0) return max(30.0, min(100.0, base_db + noise)) -def generate_fake_data(): - print("Clearing existing study room data...") - collection.delete_many({}) - +def get_fake_data(locations): now = datetime.utcnow() start_time = now - timedelta(hours=24) - docs_to_insert = [] - - print("Generating 24 hours of fake data with patterns...") - + docs = [] current_time = start_time while current_time <= now: hour = current_time.hour - for loc in UMD_LOCATIONS: + for loc in locations: db_level = get_db_for_time_and_location(hour, loc["id"]) # Estimate people based on noise level. @@ -99,9 +69,36 @@ def generate_fake_data(): "people": people_count, "date": current_time } - docs_to_insert.append(doc) + docs.append(doc) current_time += timedelta(minutes=15) + + return docs + +def generate_fake_data(): + from pymongo import MongoClient + MONGO_URI = os.getenv("MONGODB_URI", "mongodb+srv://SarayuJ:[EMAIL_ADDRESS]/testing") + client = MongoClient(MONGO_URI) + db = client.study_buddy_db + collection = db.study_rooms + + UMD_LOCATIONS = [ + { "id": 'esj', "name": 'Edward St. John (ESJ)', "lng": -76.94209511596014, "lat": 38.987133359608755 }, + { "id": 'mckeldin', "name": 'McKeldin Library', "lng": -76.94494907523277, "lat": 38.986021017749366 }, + { "id": 'hornbake', "name": 'Hornbake Library', "lng": -76.94161787005467, "lat": 38.988233373664826 }, + { "id": 'stem', "name": 'STEM Library', "lng": -76.93942003731279, "lat": 38.988991437126195 }, + { "id": 'clarice', "name": 'Clarice Library', "lng": -76.9500912552473, "lat": 38.990547823732285 }, + { "id": 'yahentamitsi', "name": 'Yahentamitsi', "lng": -76.9448027183373, "lat": 38.99108961575231 }, + { "id": 'iribe', "name": 'Iribe', "lng": -76.93643838603555, "lat": 38.98933701397555 }, + { "id": 'reckord', "name": 'Reckord Armory', "lng": -76.93897470250619, "lat": 38.98609556181066 }, + { "id": 'stamp', "name": 'Stamp Student Union', "lng": -76.94473083972326, "lat": 38.988130238874874 } + ] + + print("Clearing existing study room data...") + collection.delete_many({}) + + print("Generating 24 hours of fake data with patterns...") + docs_to_insert = get_fake_data(UMD_LOCATIONS) print(f"Inserting {len(docs_to_insert)} records into MongoDB...") collection.insert_many(docs_to_insert) diff --git a/website/src/lib/components/Footer.svelte b/website/src/lib/components/Footer.svelte new file mode 100644 index 0000000..ce3fead --- /dev/null +++ b/website/src/lib/components/Footer.svelte @@ -0,0 +1,22 @@ + + + diff --git a/website/src/routes/+layout.svelte b/website/src/routes/+layout.svelte index 595964d..437aeee 100644 --- a/website/src/routes/+layout.svelte +++ b/website/src/routes/+layout.svelte @@ -5,6 +5,7 @@ import { page } from '$app/stores'; import VoiceButton from '$lib/components/VoiceButton.svelte'; import LogoBadge from '$lib/components/LogoBadge.svelte'; + import Footer from '$lib/components/Footer.svelte'; let { children } = $props(); @@ -15,7 +16,7 @@ -
+
{#if showVoiceButton} @@ -56,7 +63,9 @@ {/if} -
+
{@render children()}
+ +
diff --git a/website/src/routes/info/+page.svelte b/website/src/routes/info/+page.svelte new file mode 100644 index 0000000..50d0af2 --- /dev/null +++ b/website/src/routes/info/+page.svelte @@ -0,0 +1,171 @@ + + +
+
+ +
+

+ HushMap +

+
+ + Best UI/UX Bitcamp 2026 +
+

+ A real-time study buddy platform integrating vision AI, speech recognition, and map data to optimize campus space utilization. +

+
+ + +
+
+

+ + The Problem: Campus Noise and Crowds +

+
+
+

Sensory Overload

+

Unexpectedly loud environments can trigger severe sensory overload for neurodivergent students or other students with high sensitivity.

+
+
+

Awkward Confrontation

+

Neither librarians nor other students want to initiate uncomfortable confrontations when noise levels spike.

+
+
+

Wasted Time

+

Students burn time walking to study spots only to find them packed and loud, wishing they knew before leaving their dorm.

+
+
+
+ + +
+
+

+ + Our Solution: HushMap +

+
+
+

Real-Time Mapping

+

We track noise levels across campus as they happen.

+
+
+

Historical Trends

+

We analyze past data to predict the best times to study.

+
+
+

Active Control

+

We use smart devices to keep noise levels within acceptable limits.

+
+
+ + +
+ +
+

1. The Interactive Map

+
    +
  • 24hr Data Storage: Database with the past 24 hrs' worth of sound volume data saved for analysis.
    • +
  • Live Updates: Real-time updates directly from campus-wide noise sensors.
    • +
  • Noise Legend: Visual legend and graph describing noise levels and thresholds.
    • +
+
+ +
+

2. TerpAI

+
    +
  • Multichannel Access: Talk to Terp AI assistant in the website or in person with the sensors.
    • +
  • Study Recommendations: TerpAI will let you know the best spots to study based on current noise data.
    • +
+
+ +
+

3. Accessibility Settings

+

HushMap ensures usability for all students through integrated accessibility tools.

+
    +
  • Color Blind Mode: Optimized palette for color vision deficiencies.
    • +
  • Language Translation: Multi-language support for international users.
    • +
  • High Contrast Mode: Enhanced legibility for low-vision accessibility.
    • +
+
+ +
+

4. The On-site Librarians

+
    +
  • Automated Noise Management: Nodes monitor noise levels and react to noise spikes by telling students to quiet down.
    • +
  • Interactive Assistance: Students can interact directly by asking the librarians questions in real-time.
    • +
  • Privacy & Analytics: No audio recorded—only noise data. Camera footage determines occupancy by comparing seats vs. people.
    • +
+
+
+
+ + +
+ +
+
+

+ + Frontend +

+
    +
  • ▹ SvelteKit 5
    • +
  • ▹ Tailwind CSS 4
    • +
  • ▹ MapLibre GL JS
    • +
  • ▹ Chart.js
    • +
+
+ + +
+
+

+ + Backend +

+
    +
  • ▹ Python & FastAPI
    • +
  • ▹ WebSockets
    • +
  • ▹ MongoDB / In-Memory Mock Data
    • +
+
+ + +
+
+

+ + AI Integration +

+
+
    +
  • ▹ Terp AI (Primary Conversational Agent)
    • +
  • ▹ Gemini 2.5 Flash (Seamless Fallback)
    • +
  • ▹ Faster-Whisper (On-device Speech-to-Text)
    • +
+
    +
  • ▹ ElevenLabs (Text-to-Speech Voice)
    • +
  • ▹ Yolo v8 Vision (Room Image Analysis)
    • +
+
+
+
+ + +
+
+

Project Team

+
+ Gagan (Adith) Manjunatha + Sameera Nageshwar + Jolie Wu + Sarayu Jilludumudi +
+
+
+
\ No newline at end of file