Hackathons/HushMap
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

M5 Update and Web Voice Agent using Terp AI.

Browse Source
This commit is contained in:
SirBlob
2026-04-12 08:33:28 -04:00
parent 88de831198
commit e6c7ed230b
12 changed files with 1038 additions and 524 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+9 -6
View File
@@ -1,4 +1,4 @@
# BitCamp 2026 - AI Study Buddy & Room Monitor # HUSHMAP - AI Study Buddy & Room Monitor
This project is a comprehensive solution featuring an M5GO smart device integration, an AI Voice and Vision Backend, and a Svelte frontend dashboard. It connects physical hardware to advanced AI models (Terp AI, ElevenLabs, YOLOv8) to provide a real-time study buddy experience and a study room occupancy monitor. This project is a comprehensive solution featuring an M5GO smart device integration, an AI Voice and Vision Backend, and a Svelte frontend dashboard. It connects physical hardware to advanced AI models (Terp AI, ElevenLabs, YOLOv8) to provide a real-time study buddy experience and a study room occupancy monitor.
@@ -18,7 +18,8 @@ bun run dev --open
### 2. AI Backend Services (`/backend`) ### 2. AI Backend Services (`/backend`)
A FastAPI backend providing two core capabilities: A FastAPI backend providing two core capabilities:
- **Real-time Voice WebSockets (`/ws/voice`)**: Connects the M5GO device to STT (faster-whisper), an LLM (Terp AI), and TTS (ElevenLabs). It streams audio bytes natively over WebSockets. - **Real-time Voice WebSockets (`/ws/voice`)**: Connects the M5GO device AND the web dashboard to STT (faster-whisper), LLMs (Terp AI), and TTS (ElevenLabs). It streams 16-bit PCM audio bytes natively over WebSockets in full-duplex.
- **Context DB Aggregation**: TerpAI automatically queries the MongoDB `study_rooms_collection` to gather live hardware decibel readings globally before answering your prompt.
- **Vision Occupancy API (`/api/vision/room-status`)**: Uses YOLOv8 object detection to identify people and chairs in a room image, determining if a study room is fully occupied and pairing the closest person to an available chair. - **Vision Occupancy API (`/api/vision/room-status`)**: Uses YOLOv8 object detection to identify people and chairs in a room image, determining if a study room is fully occupied and pairing the closest person to an available chair.
**Developing:** **Developing:**
@@ -51,11 +52,13 @@ Make sure you set up your `.env` variables before running the Docker containers
Create a `.env` in the `/backend` folder: Create a `.env` in the `/backend` folder:
```ini ```ini
ELEVENLABS_API_KEY=your_elevenlabs_api_key_here ELEVENLABS_API_KEY=sk_...
ELEVENLABS_VOICE_ID=JBFqnCBsd6RMkjVDRZzb ELEVENLABS_VOICE_ID=JBFqnCBsd6RMkjVDRZzb
TERP_AI_BEARER_TOKEN=your_jwt_token_here TERP_AI_BEARER_TOKEN=eyJhbGciOiJ...
TERP_AI_CONVERSATION_ID=5e752e56-06c6-ec73-1f13-456029ce1299 TERP_AI_CONVERSATION_ID=37fa27cc-542a-c8a8-9c31-9d1954fdc1d2
MONGODB_URI=mongodb_url_here MONGODB_URI=mongodb+srv://...
``` ```
Update your `.env` to match the exact `authorization: Bearer` and `parentSegmentId` context from TerpAI if timeouts occur.
Update the `/m5go/main.py` file to include your Wi-Fi credentials and the correct local IP for the WebSocket (`WS_URL`). Update the `/m5go/main.py` file to include your Wi-Fi credentials and the correct local IP for the WebSocket (`WS_URL`).
backend/README.md
+20 -9
View File
@@ -26,11 +26,14 @@ This directory contains the FastAPI backend for the AI Voice Agent, facilitating
### Configuration ### Configuration
Update the `.env` file in this directory with your ElevenLabs credentials: Update the `.env` file in this directory with your credentials:
```ini ```ini
ELEVENLABS_API_KEY=your_elevenlabs_api_key_here ELEVENLABS_API_KEY=sk_...
ELEVENLABS_VOICE_ID=JBFqnCBsd6RMkjVDRZzb ELEVENLABS_VOICE_ID=JBFqnCBsd6RMkjVDRZzb
TERP_AI_BEARER_TOKEN=eyJhbGciOiJSUz...
TERP_AI_CONVERSATION_ID=37fa27cc-542a-c8a8-9c31-9d1954fdc1d2
MONGODB_URI=mongodb+srv://...
``` ```
## Running the Server ## Running the Server
@@ -60,12 +63,18 @@ This is the primary WebSocket endpoint used by the M5GO device for real-time voi
} }
``` ```
4. **Processing (Server):** Upon receiving the `stop_listening` event, the server executes the AI pipeline: 4. **Processing (Server):** Upon receiving the `stop_listening` event, the server executes the AI pipeline:
- Transcribes the accumulated PCM audio using `faster-whisper`. - Transcribes the accumulated Int16 PCM audio organically using `faster-whisper`.
- Sends the transcribed text to the Terp AI conversational endpoint and waits for the full response. - Injects a MongoDB aggregate map of the latest 24hr Campus Location noise levels seamlessly into the LLM system prompt.
- Sends the Terp AI response text to ElevenLabs TTS. - Sends the transcribed text & location context to the Terp AI conversational endpoint and waits for the full response.
- Converts the received TTS audio to 16-bit 16kHz Mono PCM. - Streams the Terp AI response text directly to ElevenLabs TTS and demands `pcm_16000` via URL flags natively!
5. **Streaming Response (Server -> Client):** The server sends the converted PCM audio back to the client as binary frames. 5. **TTS Endpoint Notification**: The server saves the TTS audio buffer and pushes a JSON:
6. **End of Response (Server -> Client):** The server sends an empty binary frame (`b""`) to signal that playback is complete. ```json
{
"event": "tts_ready",
"size": 105000
}
```
6. **Audio Callback**: Client queries `GET /api/tts-audio` to play the binary wav response.
## REST Endpoints ## REST Endpoints
@@ -126,7 +135,9 @@ Returns a list of all recorded study room data from the last 24 hours, sorted by
## Client Integration Notes ## Client Integration Notes
For the ESP32/M5GO client (`m5go/main.py`), ensure you update the `WS_URL` variable to point to the correct local IP address of the machine running this backend server. For the ESP32/M5GO hardware client (`m5go/main.py`), ensure you update the `WS_URL` variable to point to the correct internal server IP.
For the Web Frontend (`VoiceButton.svelte`), it uses standard Web Audio API's `ScriptProcessorNode` to bridge the Float32 arrays strictly into 16-Bit Mono over a dynamic WebSocket tunnel automatically.
```python ```python
# In m5go/main.py # In m5go/main.py
backend/server.py
+154 -50
View File
@@ -3,13 +3,17 @@ import io
import struct import struct
import tempfile import tempfile
import subprocess import subprocess
import asyncio
import requests import requests
import json import json
import base64 import base64
import urllib.request import urllib.request
# import ssl
# ssl._create_default_https_context = ssl._create_unverified_context
from fastapi import FastAPI, WebSocket, WebSocketDisconnect, File, UploadFile from fastapi import FastAPI, WebSocket, WebSocketDisconnect, File, UploadFile
from fastapi.staticfiles import StaticFiles from fastapi.staticfiles import StaticFiles
from fastapi.responses import FileResponse from fastapi.responses import FileResponse, Response
from pydantic import BaseModel from pydantic import BaseModel
from typing import List, Optional from typing import List, Optional
from datetime import datetime, timedelta from datetime import datetime, timedelta
@@ -53,22 +57,38 @@ SAMPLE_RATE = 16000
BITS_PER_SAMPLE = 16 BITS_PER_SAMPLE = 16
NUM_CHANNELS = 1 NUM_CHANNELS = 1
CONVERSATION_ID = os.getenv("TERP_AI_CONVERSATION_ID", "5e752e56-06c6-ec73-1f13-456029ce1299") CONVERSATION_ID = os.getenv("TERP_AI_CONVERSATION_ID", "37fa27cc-542a-c8a8-9c31-9d1954fdc1d2")
HEADERS = { HEADERS = {
"accept": "*/*", "accept": "*/*",
"accept-language": "en-US,en;q=0.9", "accept-language": "en-US,en;q=0.9,de-DE;q=0.8,de;q=0.7",
"authorization": f"Bearer {os.getenv('TERP_AI_BEARER_TOKEN', '')}", "authorization": f"Bearer {os.getenv('TERP_AI_BEARER_TOKEN', '')}",
"baggage": "sentry-environment=TerpAI,sentry-release=2.2605.4472,sentry-public_key=c41f6dfb98d5bed12037e17e78c2c5d3,sentry-trace_id=250c82a03041415b99422d838ccc7003,sentry-org_id=4504359075840000,sentry-sampled=false,sentry-sample_rand=0.34017479518051186,sentry-sample_rate=0",
"content-type": "application/json", "content-type": "application/json",
"origin": "https://patriotai.gmu.edu", "origin": "https://terpai.umd.edu",
"referer": f"https://patriotai.gmu.edu/chat/8c3fc7f0-7c8b-4f2f-849c-5e2a45915066/{CONVERSATION_ID}", "priority": "u=1, i",
"user-agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36", "referer": f"https://terpai.umd.edu/chat/1eaa95ea-9b73-4850-8534-d1552401513a/{CONVERSATION_ID}",
"sec-ch-ua": "\"Chromium\";v=\"146\", \"Not-A.Brand\";v=\"24\", \"Microsoft Edge\";v=\"146\"",
"sec-ch-ua-mobile": "?0",
"sec-ch-ua-platform": "\"Windows\"",
"sec-fetch-dest": "empty",
"sec-fetch-mode": "cors",
"sec-fetch-site": "same-origin",
"sentry-trace": "250c82a03041415b99422d838ccc7003-9a9ec11d7fd0293b-0",
"user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/146.0.0.0 Safari/537.36 Edg/146.0.0.0",
"x-cosmos-session-281286": "0:-1#10931",
"x-cosmos-session-295334": "0:-1#749854",
"x-cosmos-session-317755": "0:-1#191601",
"x-cosmos-session-382299": "0:-1#265024",
"x-cosmos-session-418988": "0:-1#4058856",
"x-cosmos-session-793952": "0:-1#14004",
"x-request-id": "6a128b8a-7f63-4f97-a40b-bfd31b4a376e",
"x-timezone": "America/New_York", "x-timezone": "America/New_York",
} }
def _write_wav_to_buffer(pcm_data: bytes) -> bytes: def _write_wav_to_buffer(pcm_data: bytes, sample_rate: int = SAMPLE_RATE) -> bytes:
"""Wrap raw PCM data in a WAV header and return the full WAV bytes.""" """Wrap raw PCM data in a WAV header and return the full WAV bytes."""
data_size = len(pcm_data) data_size = len(pcm_data)
byte_rate = SAMPLE_RATE * NUM_CHANNELS * (BITS_PER_SAMPLE // 8) byte_rate = sample_rate * NUM_CHANNELS * (BITS_PER_SAMPLE // 8)
block_align = NUM_CHANNELS * (BITS_PER_SAMPLE // 8) block_align = NUM_CHANNELS * (BITS_PER_SAMPLE // 8)
buf = io.BytesIO() buf = io.BytesIO()
@@ -79,7 +99,7 @@ def _write_wav_to_buffer(pcm_data: bytes) -> bytes:
buf.write(struct.pack("<I", 16)) buf.write(struct.pack("<I", 16))
buf.write(struct.pack("<H", 1)) # PCM buf.write(struct.pack("<H", 1)) # PCM
buf.write(struct.pack("<H", NUM_CHANNELS)) buf.write(struct.pack("<H", NUM_CHANNELS))
buf.write(struct.pack("<I", SAMPLE_RATE)) buf.write(struct.pack("<I", sample_rate))
buf.write(struct.pack("<I", byte_rate)) buf.write(struct.pack("<I", byte_rate))
buf.write(struct.pack("<H", block_align)) buf.write(struct.pack("<H", block_align))
buf.write(struct.pack("<H", BITS_PER_SAMPLE)) buf.write(struct.pack("<H", BITS_PER_SAMPLE))
@@ -88,20 +108,50 @@ def _write_wav_to_buffer(pcm_data: bytes) -> bytes:
buf.write(pcm_data) buf.write(pcm_data)
return buf.getvalue() return buf.getvalue()
def _transcribe_pcm(pcm_data: bytes) -> str: def _transcribe_pcm(pcm_data: bytes, sample_rate: int = SAMPLE_RATE) -> str:
"""Transcribe raw PCM audio using faster-whisper via a temp WAV file.""" """Transcribe raw PCM audio using faster-whisper via a temp WAV file."""
from faster_whisper import WhisperModel from faster_whisper import WhisperModel
wav_data = _write_wav_to_buffer(pcm_data) print(f" Using sample rate: {sample_rate} Hz")
# Debug: analyze PCM audio quality
num_samples = len(pcm_data) // 2
if num_samples > 0:
samples = list(struct.unpack(f"<{num_samples}h", pcm_data[:num_samples * 2]))
min_s, max_s = min(samples), max(samples)
mean_s = sum(samples) / num_samples
rms = (sum(s * s for s in samples) / num_samples) ** 0.5
print(f" PCM stats (raw): {num_samples} samples, min={min_s}, max={max_s}, mean={mean_s:.1f}, RMS={rms:.1f}")
# Remove DC offset (center audio at 0)
dc_offset = int(round(mean_s))
samples = [max(-32768, min(32767, s - dc_offset)) for s in samples]
pcm_data = struct.pack(f"<{num_samples}h", *samples)
# Stats after correction
rms_fixed = (sum(s * s for s in samples) / num_samples) ** 0.5
print(f" PCM stats (fixed): DC offset removed={dc_offset}, RMS={rms_fixed:.1f}")
wav_data = _write_wav_to_buffer(pcm_data, sample_rate=sample_rate)
tmp_fd, tmp_path = tempfile.mkstemp(suffix=".wav") tmp_fd, tmp_path = tempfile.mkstemp(suffix=".wav")
try: try:
with os.fdopen(tmp_fd, "wb") as f: with os.fdopen(tmp_fd, "wb") as f:
f.write(wav_data) f.write(wav_data)
# Save a debug copy so we can listen
debug_path = os.path.join(os.path.dirname(__file__), "debug_audio.wav")
with open(debug_path, "wb") as df:
df.write(wav_data)
print(f" Debug WAV saved to: {debug_path}")
# Initialize the model (using base model for speed) # Initialize the model (using base model for speed)
model = WhisperModel("base", device="cpu", compute_type="int8") model = WhisperModel("base", device="cpu", compute_type="int8")
segments, _ = model.transcribe(tmp_path, beam_size=5) segments, info = model.transcribe(tmp_path, beam_size=5)
text = " ".join([segment.text for segment in segments]) seg_list = list(segments)
print(f" Whisper: {len(seg_list)} segments, language={info.language}, prob={info.language_probability:.2f}")
for i, seg in enumerate(seg_list):
print(f" Seg {i}: [{seg.start:.1f}s-{seg.end:.1f}s] '{seg.text}'")
text = " ".join([seg.text for seg in seg_list])
return text.strip() return text.strip()
finally: finally:
if os.path.exists(tmp_path): if os.path.exists(tmp_path):
@@ -109,34 +159,34 @@ def _transcribe_pcm(pcm_data: bytes) -> str:
def get_terp_ai_response(message: str) -> str: def get_terp_ai_response(message: str) -> str:
"""Send text to Terp AI and return the full response.""" """Send text to Terp AI and return the full response."""
url = f"https://patriotai.gmu.edu/api/internal/userConversations/{CONVERSATION_ID}/segments" url = f"https://terpai.umd.edu/api/internal/userConversations/{CONVERSATION_ID}/segments"
data = json.dumps({ payload = {
"question": message, "question": message,
"visionImageIds": [], "visionImageIds": [],
"attachmentIds": [], "attachmentIds": [],
"segmentTraceLogLevel": "NonPersisted" "segmentTraceLogLevel": "NonPersisted",
}).encode("utf-8") "lineage": {
"parentSegmentId": "83f997ca-5089-4568-ae23-fb2d5a6d5855",
req = urllib.request.Request(url, data=data, method="POST") "lineageType": "Question"
for key, value in HEADERS.items(): }
req.add_header(key, value) }
full_response = "" full_response = ""
event = None event = None
try: try:
with urllib.request.urlopen(req) as response: resp = requests.post(url, json=payload, headers=HEADERS, stream=True, timeout=30, verify=False)
while True: resp.raise_for_status()
line = response.readline() for line in resp.iter_lines(decode_unicode=True):
if not line: if not line:
break continue
line = line.decode("utf-8").strip() if line.startswith("event: "):
if line.startswith("event: "): event = line[7:]
event = line[7:] elif line.startswith("data: "):
elif line.startswith("data: "): data = line[6:]
data = line[6:] decoded = base64.b64decode(data).decode("utf-8")
decoded = base64.b64decode(data).decode("utf-8") if event == "response-updated":
if event == "response-updated": full_response += decoded
full_response += decoded resp.close()
except Exception as e: except Exception as e:
print(f"Terp AI error: {e}") print(f"Terp AI error: {e}")
return "I am sorry, there was an error connecting to Terp AI." return "I am sorry, there was an error connecting to Terp AI."
@@ -182,12 +232,13 @@ def _generate_tts(text: str) -> bytes | None:
print("ElevenLabs API key not configured") print("ElevenLabs API key not configured")
return None return None
url = f"https://api.elevenlabs.io/v1/text-to-speech/{voice_id}" # Request PCM directly — no ffmpeg needed
url = f"https://api.elevenlabs.io/v1/text-to-speech/{voice_id}?output_format=pcm_16000"
headers = { headers = {
"xi-api-key": api_key, "xi-api-key": api_key,
"Content-Type": "application/json", "Content-Type": "application/json",
"Accept": "audio/mpeg", "Accept": "application/octet-stream",
} }
payload = { payload = {
@@ -204,18 +255,28 @@ def _generate_tts(text: str) -> bytes | None:
try: try:
resp = requests.post(url, json=payload, headers=headers, timeout=30) resp = requests.post(url, json=payload, headers=headers, timeout=30)
resp.raise_for_status() resp.raise_for_status()
mp3_data = resp.content pcm_data = resp.content
if not mp3_data: if not pcm_data:
return None return None
# Convert MP3 to 16-bit 16 kHz mono PCM print(f"TTS: received {len(pcm_data)} bytes of PCM audio")
return _convert_to_pcm(mp3_data, input_format="mp3") return pcm_data
except requests.exceptions.RequestException as e: except requests.exceptions.RequestException as e:
print(f"ElevenLabs TTS error: {e}") print(f"ElevenLabs TTS error: {e}")
return None return None
# Latest TTS WAV stored in memory for HTTP download by M5GO
_latest_tts_wav = None
@app.get("/api/tts-audio")
async def get_tts_audio():
global _latest_tts_wav
if _latest_tts_wav is None:
return Response(status_code=404, content=b"No audio available")
return Response(content=_latest_tts_wav, media_type="audio/wav")
@app.websocket("/ws/voice") @app.websocket("/ws/voice")
async def websocket_voice(websocket: WebSocket): async def websocket_voice(websocket: WebSocket):
await websocket.accept() await websocket.accept()
@@ -235,8 +296,9 @@ async def websocket_voice(websocket: WebSocket):
if msg.get("event") == "stop_listening": if msg.get("event") == "stop_listening":
pcm_data = bytes(audio_buffer) pcm_data = bytes(audio_buffer)
audio_buffer = bytearray() # Reset for next time audio_buffer = bytearray() # Reset for next time
device_sample_rate = msg.get("sample_rate", SAMPLE_RATE)
print(f"Received stop_listening event. Buffer size: {len(pcm_data)} bytes.") print(f"Received stop_listening event. Buffer size: {len(pcm_data)} bytes, sample_rate: {device_sample_rate} Hz")
if len(pcm_data) < 3200: if len(pcm_data) < 3200:
print("Audio too short, ignoring.") print("Audio too short, ignoring.")
@@ -245,20 +307,22 @@ async def websocket_voice(websocket: WebSocket):
# Step 1: Speech to Text # Step 1: Speech to Text
print("Transcribing...") print("Transcribing...")
user_text = _transcribe_pcm(pcm_data) user_text = _transcribe_pcm(pcm_data, sample_rate=device_sample_rate)
if not user_text: if not user_text:
print("Transcription failed or empty.") print("Transcription failed or empty.")
await websocket.send_bytes(b"") await websocket.send_text(json.dumps({"event": "error", "msg": "No speech detected"}))
continue continue
print(f"User said: {user_text}") print(f"User said: {user_text}")
# Step 2: Terp AI # Step 2: Terp AI
print("Sending to Terp AI...") print("Sending to Terp AI...")
ai_response_text = get_terp_ai_response(user_text) context_str = get_latest_locations_context()
augmented_prompt = f"USER ASKS: {user_text}\n\n[SYSTEM CONTEXT - LATEST UMD ROOM STATS TO HELP YOU ANSWER IF ASKED]:\n{context_str}"
ai_response_text = get_terp_ai_response(augmented_prompt)
if not ai_response_text: if not ai_response_text:
print("No response from Terp AI.") print("No response from Terp AI.")
await websocket.send_bytes(b"") await websocket.send_text(json.dumps({"event": "error", "msg": "No AI response"}))
continue continue
print(f"Terp AI response: {ai_response_text}") print(f"Terp AI response: {ai_response_text}")
@@ -268,18 +332,24 @@ async def websocket_voice(websocket: WebSocket):
tts_pcm = _generate_tts(ai_response_text) tts_pcm = _generate_tts(ai_response_text)
if tts_pcm: if tts_pcm:
print(f"Sending {len(tts_pcm)} bytes of PCM back to device.") # Save as WAV for HTTP download by M5GO
await websocket.send_bytes(tts_pcm) global _latest_tts_wav
_latest_tts_wav = _write_wav_to_buffer(tts_pcm)
print(f"TTS WAV ready: {len(_latest_tts_wav)} bytes, serving via /api/tts-audio")
await websocket.send_text(json.dumps({
"event": "tts_ready",
"size": len(_latest_tts_wav)
}))
else: else:
print("TTS failed.") print("TTS failed.")
await websocket.send_bytes(b"") await websocket.send_text(json.dumps({"event": "error", "msg": "TTS failed"}))
except json.JSONDecodeError: except json.JSONDecodeError:
pass pass
except Exception as e: except Exception as e:
print(f"Error processing message: {e}") print(f"Error processing message: {e}")
await websocket.send_bytes(b"") await websocket.send_text(json.dumps({"event": "error", "msg": str(e)[:100]}))
except WebSocketDisconnect: except (WebSocketDisconnect, RuntimeError):
print("Device disconnected.") print("Device disconnected.")
@app.post("/api/vision/room-status") @app.post("/api/vision/room-status")
@@ -300,6 +370,40 @@ UMD_LOCATIONS = [
{ "id": 'stamp', "name": 'Stamp Student Union', "lng": -76.94473083972326, "lat": 38.988130238874874 } { "id": 'stamp', "name": 'Stamp Student Union', "lng": -76.94473083972326, "lat": 38.988130238874874 }
] ]
def get_latest_locations_context() -> str:
"""Fetch the latest stats for each known location to feed as AI context."""
twenty_four_hours_ago = datetime.utcnow() - timedelta(hours=24)
pipeline = [
{"$match": {"date": {"$gte": twenty_four_hours_ago}}},
{"$sort": {"date": -1}},
{"$group": {
"_id": "$room_id",
"latest_db": {"$first": "$db"},
"time": {"$first": "$date"}
}}
]
latest_stats = list(study_rooms_collection.aggregate(pipeline))
if not latest_stats:
return "No recent location noise stats available today."
room_dict = {loc["id"]: loc["name"] for loc in UMD_LOCATIONS}
lines = ["Latest Study Room Stats:"]
for stat in latest_stats:
room_id = stat.get("_id")
name = room_dict.get(room_id, room_id)
db = stat.get("latest_db", 0.0)
status = "Quiet"
if isinstance(db, (int, float)):
if db >= 65: status = "Loud"
elif db >= 55: status = "Moderate"
lines.append(f"- {name}: Noise Level {db:.1f} dB ({status})")
return "\n".join(lines)
@app.post("/api/study-rooms") @app.post("/api/study-rooms")
async def create_study_room_data(data: StudyRoomData): async def create_study_room_data(data: StudyRoomData):
# Check if the coordinates match one of the known locations (with small tolerance) # Check if the coordinates match one of the known locations (with small tolerance)
m5go/face_assets.py
-177
View File
@@ -1,177 +0,0 @@
# --- FACE ASSETS ---
C = {
'0': 0x222222,
'Y': 0xFFFF00,
'R': 0xFF0000,
'W': 0xFFFFFF,
'B': 0x000000,
'P': 0xFF8888,
'D': 0x555555
}
f_s_o = [
"0000000000000000",
"0000000000000000",
"000WWW0000WWW000",
"00WWBW0000WWBW00",
"00WWBW0000WWBW00",
"000WWW0000WWW000",
"0000000000000000",
"0000000000000000",
"0000000000000000",
"YY000000000000YY",
"0YY0000000000YY0",
"00YY00000000YY00",
"000YYYYYYYYYY000",
"00000YYYYYY00000",
"0000000000000000",
"0000000000000000"
]
f_s_h = [
"0000000000000000",
"0000000000000000",
"0000000000000000",
"000WWW0000WWW000",
"00WWBW0000WWBW00",
"000WWW0000WWW000",
"0000000000000000",
"0000000000000000",
"0000000000000000",
"YY000000000000YY",
"0YY0000000000YY0",
"00YY00000000YY00",
"000YYYYYYYYYY000",
"00000YYYYYY00000",
"0000000000000000",
"0000000000000000"
]
f_s_c = [
"0000000000000000",
"0000000000000000",
"0000000000000000",
"0000000000000000",
"0000000000000000",
"00WWWW0000WWWW00",
"0000000000000000",
"0000000000000000",
"0000000000000000",
"YY000000000000YY",
"0YY0000000000YY0",
"00YY00000000YY00",
"000YYYYYYYYYY000",
"00000YYYYYY00000",
"0000000000000000",
"0000000000000000"
]
f_a_o = [
"0000000000000000",
"0DDDD000000DDDD0",
"00DDDD0000DDDD00",
"000DDDD00DDDD000",
"000WWW0000WWW000",
"00WWBB0000BBWW00",
"000WWW0000WWW000",
"0000000000000000",
"0000000000000000",
"0000000000000000",
"000000RRRR000000",
"0000RRRRRRRR0000",
"00RRRR0000RRRR00",
"0RRR00000000RRR0",
"0000000000000000",
"0000000000000000"
]
f_a_h = [
"0000000000000000",
"0DDDD000000DDDD0",
"00DDDD0000DDDD00",
"000DDDD00DDDD000",
"0000000000000000",
"000WWW0000WWW000",
"00WWBB0000BBWW00",
"0000000000000000",
"0000000000000000",
"0000000000000000",
"000000RRRR000000",
"0000RRRRRRRR0000",
"00RRRR0000RRRR00",
"0RRR00000000RRR0",
"0000000000000000",
"0000000000000000"
]
f_a_c = [
"0000000000000000",
"0DDDD000000DDDD0",
"00DDDD0000DDDD00",
"000DDDD00DDDD000",
"0000000000000000",
"0000000000000000",
"000WW000000WW000",
"0000000000000000",
"0000000000000000",
"0000000000000000",
"000000RRRR000000",
"0000RRRRRRRR0000",
"00RRRR0000RRRR00",
"0RRR00000000RRR0",
"0000000000000000",
"0000000000000000"
]
f_t_1 = [
"0000000000000000",
"0000000000000000",
"0000000000000000",
"0000000000000000",
"000WWW0000WWW000",
"00WWWW0000WWWW00",
"000WWW0000WWW000",
"0000000000000000",
"0000000000000000",
"0000000000000000",
"0000000000000000",
"0000000000000000",
"000YYYYYYYYYY000",
"0000000000000000",
"0000000000000000",
"0000000000000000"
]
f_t_2 = [
"0000000000000000",
"0000000000000000",
"00DD00000000DD00",
"000DD000000DD000",
"0000000000000000",
"000WWW0000WWW000",
"00WWBW0000WWBW00",
"000WWW0000WWW000",
"0000000000000000",
"0000000000000000",
"0000000000000000",
"0000RRRRRRRR0000",
"000RR000000RR000",
"0000000000000000",
"0000000000000000",
"0000000000000000"
]
def d_s(lcd, f, s_x, s_y, p_s):
for r in range(16):
c = 0
while c < 16:
s_c = c
v = f[r][c]
while c < 16 and f[r][c] == v:
c += 1
w = c - s_c
x_p = s_x + (s_c * p_s)
y_p = s_y + (r * p_s)
lcd.fillRect(x_p, y_p, w * p_s, p_s, C[v])
m5go/main.py
+515 -267
View File
@@ -3,73 +3,277 @@ import time
import machine import machine
import json import json
import math import math
import _thread import usocket
import websocket import ubinascii
import os
import gc
from m5stack import * from m5stack import *
from m5ui import * from m5ui import *
from uiflow import * from uiflow import *
C = { '0': 0x222222, 'Y': 0xFFFF00, 'R': 0xFF0000, 'W': 0xFFFFFF, 'B': 0x000000, 'D': 0x555555 }
f_s_o = "00000000000000000000000000000000000WWW0000WWW00000WWBW0000WWBW0000WWBW0000WWBW00000WWW0000WWW000000000000000000000000000000000000000000000000000YY000000000000YY0YY0000000000YY000YY00000000YY00000YYYYYYYYYY00000000YYYYYY0000000000000000000000000000000000000"
f_s_c = "0000000000000000000000000000000000000000000000000000000000000000000000000000000000WWWW0000WWWW00000000000000000000000000000000000000000000000000YY000000000000YY0YY0000000000YY000YY00000000YY00000YYYYYYYYYY00000000YYYYYY0000000000000000000000000000000000000"
f_t_1 = "0000000000000000000000000000000000000000000000000000000000000000000WWW0000WWW00000WWWW0000WWWW00000WWW0000WWW00000000000000000000000000000000000000000000000000000000000000000000000000000000000000YYYYYYYYYY000000000000000000000000000000000000000000000000000"
f_t_2 = "0000000000000000000000000000000000DD00000000DD00000DD000000DD0000000000000000000000WWW0000WWW00000WWBW0000WWBW00000WWW0000WWW0000000000000000000000000000000000000000000000000000000RRRRRRRR0000000RR000000RR000000000000000000000000000000000000000000000000000"
f_listen = "0000000000000000000000000000000000WWWW0000WWWW0000WBBW0000WBBW0000WBBW0000WBBW0000WWWW0000WWWW0000000000000000000000000000000000000000000000000000000YYYYYY000000000YY0000YY00000000YY0000YY000000000YYYYYY00000000000000000000000000000000000000000000000000000"
f_speak = "00000000000000000000000000000000000WWW0000WWW00000WWBW0000WWBW0000WWBW0000WWBW00000WWW0000WWW0000000000000000000000000000000000000000000000000000000YYYYYYYY0000000YY000000YY000000YY000000YY0000000YYYYYYYY0000000000000000000000000000000000000000000000000000"
f_angry = "00000000000000000DDDD000000DDDD000DDDD0000DDDD00000DDDD00DDDD000000WWW0000WWW00000WWBB0000BBWW00000WWW0000WWW000000000000000000000000000000000000000000000000000000000RRRR0000000000RRRRRRRR000000RRRR0000RRRR000RRR00000000RRR000000000000000000000000000000000"
def d_s(lcd, f, s_x, s_y, p_s):
for r in range(16):
c = 0
i = r * 16
while c < 16:
s_c = c
v = f[i + c]
while c < 16 and f[i + c] == v:
c += 1
w = c - s_c
lcd.fillRect(s_x + s_c * p_s, s_y + r * p_s, w * p_s, p_s, C[v])
setScreenColor(0x222222) setScreenColor(0x222222)
# --- CONFIGURATION --- # ==========================================
# CONFIGURATION
# ==========================================
WIFI_SSID = "Blobby" WIFI_SSID = "Blobby"
WIFI_PASS = "73556088" WIFI_PASS = "73556088"
# Update this to the IP address of your backend server WS_URL = "ws://192.168.137.1:8000/ws/voice"
WS_URL = "ws://192.168.137.1:8000/ws/voice"
# ---------------------
def draw_status(status, color): # Location Settings for Noise Monitoring
CURRENT_ROOM_ID = "mckeldin"
CURRENT_LAT = 38.986021
CURRENT_LNG = -76.944949
# Audio Settings
TARGET_SAMPLE_RATE = 8000 # Voice recording sample rate
AUDIO_CHUNK_SIZE = 2048
# ==========================================
# WEBSOCKET CLIENT
# ==========================================
class WSClient:
def __init__(self, sock):
self._sock = sock
def send(self, data):
if isinstance(data, str):
data = data.encode()
opcode = 0x1
else:
opcode = 0x2
length = len(data)
mask_key = os.urandom(4)
header = bytearray()
header.append(0x80 | opcode)
if length < 126:
header.append(0x80 | length)
elif length < 65536:
header.append(0x80 | 126)
header.append((length >> 8) & 0xFF)
header.append(length & 0xFF)
else:
header.append(0x80 | 127)
for i in range(7, -1, -1):
header.append((length >> (8 * i)) & 0xFF)
header.extend(mask_key)
masked = bytearray(data)
for i in range(length):
masked[i] ^= mask_key[i % 4]
self._sock.send(header + masked)
def recv(self):
hdr = self._recv_exact(2)
if not hdr or len(hdr) < 2:
return None
opcode = hdr[0] & 0x0F
is_masked = (hdr[1] & 0x80) != 0
length = hdr[1] & 0x7F
if length == 126:
ext = self._recv_exact(2)
length = (ext[0] << 8) | ext[1]
elif length == 127:
ext = self._recv_exact(8)
length = 0
for b in ext:
length = (length << 8) | b
mask_key = self._recv_exact(4) if is_masked else None
payload = self._recv_exact(length) if length > 0 else b""
if is_masked and mask_key and payload:
payload = bytearray(payload)
for i in range(len(payload)):
payload[i] ^= mask_key[i % 4]
payload = bytes(payload)
if opcode == 0x8:
return None
if opcode == 0x9:
self._send_pong(payload)
return self.recv()
if opcode == 0x1:
return payload.decode() if payload else ""
return payload
def _send_pong(self, data):
mask_key = os.urandom(4)
length = len(data) if data else 0
header = bytearray([0x8A, 0x80 | length])
header.extend(mask_key)
if data:
masked = bytearray(data)
for i in range(length):
masked[i] ^= mask_key[i % 4]
self._sock.send(header + masked)
else:
self._sock.send(header)
def _recv_exact(self, n):
buf = bytearray(n)
pos = 0
while pos < n:
chunk = self._sock.recv(n - pos)
if not chunk:
return None
buf[pos:pos + len(chunk)] = chunk
pos += len(chunk)
return bytes(buf)
def close(self):
try:
self._sock.send(bytearray([0x88, 0x80, 0, 0, 0, 0]))
except:
pass
try:
self._sock.close()
except:
pass
def ws_connect(url):
if url.startswith("ws://"):
rest = url[5:]
else:
raise ValueError("Only ws:// supported")
if "/" in rest:
host_port = rest.split("/", 1)[0]
path = "/" + rest.split("/", 1)[1]
else:
host_port = rest
path = "/"
if ":" in host_port:
host = host_port.split(":")[0]
port = int(host_port.split(":")[1])
else:
host = host_port
port = 80
addr = usocket.getaddrinfo(host, port)[0][-1]
sock = usocket.socket()
sock.connect(addr)
sock.settimeout(15)
key = ubinascii.b2a_base64(os.urandom(16)).strip().decode()
req = (
"GET %s HTTP/1.1\r\n"
"Host: %s\r\n"
"Upgrade: websocket\r\n"
"Connection: Upgrade\r\n"
"Sec-WebSocket-Key: %s\r\n"
"Sec-WebSocket-Version: 13\r\n"
"\r\n"
) % (path, host_port, key)
sock.send(req.encode())
resp = b""
while b"\r\n\r\n" not in resp:
b = sock.recv(1)
if not b:
sock.close()
raise Exception("Closed during handshake")
resp += b
status_line = resp.split(b"\r\n")[0]
if b"101" not in status_line:
sock.close()
raise Exception("Upgrade failed: " + status_line.decode())
return WSClient(sock)
# ==========================================
# UI & UTILITY FUNCTIONS
# ==========================================
_cur_face = None
def set_face(face):
global _cur_face
if face and face != _cur_face:
d_s(lcd, face, 80, 25, 10)
_cur_face = face
def draw_status(status, color, face=None):
lcd.fillRect(0, 220, 320, 20, 0x222222) lcd.fillRect(0, 220, 320, 20, 0x222222)
lcd.print(status, int((320 - len(status) * 8) / 2), 220, color) lcd.print(status, int((320 - len(status) * 8) / 2), 220, color)
if face:
set_face(face)
lcd.print("Connecting to WiFi...", 0, 0, 0xFFFFFF) def connect_wifi():
wlan = network.WLAN(network.STA_IF) lcd.clear()
wlan.active(True) lcd.print("Connecting to WiFi...", 0, 0, 0xFFFFFF)
wlan.connect(WIFI_SSID, WIFI_PASS) wlan = network.WLAN(network.STA_IF)
wlan.active(True)
if not wlan.isconnected():
wlan.connect(WIFI_SSID, WIFI_PASS)
attempts = 0
while not wlan.isconnected() and attempts < 20:
time.sleep(0.5)
attempts += 1
if wlan.isconnected():
lcd.clear()
lcd.print("WiFi Connected!", 0, 0, 0x00FF00)
lcd.print(wlan.ifconfig()[0], 0, 20, 0x00FF00)
time.sleep(1)
lcd.clear()
else:
lcd.clear()
lcd.print("WiFi Failed", 0, 0, 0xFF0000)
time.sleep(2)
lcd.clear()
return wlan.isconnected()
# Simple connection loop # ==========================================
attempts = 0 # AUDIO I/O
while not wlan.isconnected() and attempts < 20: # ==========================================
time.sleep(0.5)
attempts += 1
if wlan.isconnected(): def get_adc():
lcd.clear()
lcd.print("WiFi Connected!", 0, 0, 0x00FF00)
lcd.print(wlan.ifconfig()[0], 0, 20, 0x00FF00)
time.sleep(1)
lcd.clear()
else:
lcd.clear()
lcd.print("WiFi Failed", 0, 0, 0xFF0000)
time.sleep(2)
lcd.clear()
try:
adc = machine.ADC(34)
adc.atten(machine.ADC.ATTN_11DB)
except:
try: try:
adc = machine.ADC(machine.Pin(34)) a = machine.ADC(34)
adc.atten(machine.ADC.ATTN_11DB) a.atten(machine.ADC.ATTN_11DB)
return a
except: except:
adc = None try:
a = machine.ADC(machine.Pin(34))
a.atten(machine.ADC.ATTN_11DB)
return a
except:
return None
def get_db(): def get_db(adc_obj):
if not adc: return 30 if not adc_obj: return 30
sum_v = 0 sum_v = 0
sum_sq = 0 sum_sq = 0
count = 0 count = 0
end_t = time.ticks_ms() + 40 end_t = time.ticks_ms() + 40
while time.ticks_ms() < end_t: while time.ticks_ms() < end_t:
try: try:
v = adc.read() v = adc_obj.read()
sum_v += v sum_v += v
sum_sq += v * v sum_sq += v * v
count += 1 count += 1
except: except:
pass pass
if count == 0: return 30 if count == 0: return 30
mean = sum_v / count mean = sum_v / count
@@ -79,249 +283,293 @@ def get_db():
amp = math.sqrt(variance) amp = math.sqrt(variance)
if amp <= 1: return 30 if amp <= 1: return 30
# Use your specific calculation formula to properly scale to human DB # Scale to human DB
db = 20 * math.log10(amp) + 25 return 20 * math.log10(amp) + 25
return db
def init_mic(): def init_manual_spk():
try: try:
if hasattr(machine.I2S, "RX"): # M5GO Core speaker sits on DAC 1 (Pin 25)
audio_in = machine.I2S( # Therefore we MUST strictly use DAC_BUILT_IN. Digital I2S on this pin is not supported!
0, if hasattr(machine.I2S, "MODE_DAC_BUILT_IN"):
sck=machine.Pin(0), mode = getattr(machine.I2S, "MODE_MASTER", 1) | getattr(machine.I2S, "MODE_TX", 2) | getattr(machine.I2S, "MODE_DAC_BUILT_IN", 0)
ws=machine.Pin(0),
sd=machine.Pin(34),
mode=machine.I2S.RX,
bits=16,
format=machine.I2S.MONO,
rate=16000,
ibuf=4096
)
return audio_in
else:
# Fallback for old Micropython (e.g. M5Stack UIFlow)
mode = getattr(machine.I2S, "MODE_MASTER", 1) | getattr(machine.I2S, "MODE_RX", 2)
if hasattr(machine.I2S, "MODE_PDM"):
mode |= getattr(machine.I2S, "MODE_PDM", 0)
cfmt = getattr(machine.I2S, "CHANNEL_FMT_ALL_LEFT", 1)
dfmt = getattr(machine.I2S, "FORMAT_I2S", 1)
try_args = [
([getattr(machine.I2S, "NUM0", 0), mode, 16000, 16, cfmt, dfmt], {}),
([getattr(machine.I2S, "NUM0", 0), mode, 16000, 16], {}),
([getattr(machine.I2S, "NUM0", 0)], {"mode": mode, "sample_rate": 16000, "bits": 16, "channel_format": cfmt, "data_format": dfmt}),
([getattr(machine.I2S, "NUM0", 0)], {"mode": mode, "sample_rate": 16000, "bits": 16}),
([getattr(machine.I2S, "NUM0", 0)], {"mode": mode, "bck": 0, "ws": 0, "sd": 34, "sample_rate": 16000, "bits": 16}),
([], {"mode": mode, "sample_rate": 16000, "bits": 16}),
([getattr(machine.I2S, "NUM0", 0)], {"mode": mode}),
([], {"mode": mode}),
]
audio_in = None
last_e = None
for args, kwargs in try_args:
try:
audio_in = machine.I2S(*args, **kwargs)
break
except Exception as e:
last_e = e
if audio_in is None:
raise Exception("Mic fallback failed: " + str(last_e))
return audio_in
except Exception as e:
print("Mic I2S Error:", repr(e))
lcd.print("Mic Err: " + str(e)[:20], 0, 40, 0xFF0000)
return None
def init_spk():
try:
if hasattr(machine.I2S, "TX"):
audio_out = machine.I2S(
1,
sck=machine.Pin(12),
ws=machine.Pin(0),
sd=machine.Pin(25), # M5Stack Core/GO speaker is typically on Pin 25
mode=machine.I2S.TX,
bits=16,
format=machine.I2S.MONO,
rate=16000,
ibuf=8192
)
return audio_out
else:
mode = getattr(machine.I2S, "MODE_MASTER", 1) | getattr(machine.I2S, "MODE_TX", 2)
if hasattr(machine.I2S, "MODE_DAC_BUILT_IN"):
mode |= machine.I2S.MODE_DAC_BUILT_IN
cfmt = getattr(machine.I2S, "CHANNEL_FMT_RIGHT_LEFT", 1) cfmt = getattr(machine.I2S, "CHANNEL_FMT_RIGHT_LEFT", 1)
dfmt = getattr(machine.I2S, "FORMAT_I2S_MSB", 1) dfmt = getattr(machine.I2S, "FORMAT_I2S_MSB", 1)
try_args = [ i2s_id = getattr(machine.I2S, "NUM0", 0)
([getattr(machine.I2S, "NUM1", 1), mode, 16000, 16, cfmt, dfmt], {}),
([getattr(machine.I2S, "NUM1", 1), mode, 16000, 16], {}), # Different MicroPython versions vary wildly on kwarg vs positional I2S init structure
([getattr(machine.I2S, "NUM1", 1)], {"mode": mode, "sample_rate": 16000, "bits": 16, "channel_format": cfmt, "data_format": dfmt}), try_sigs = [
([getattr(machine.I2S, "NUM1", 1)], {"mode": mode, "sample_rate": 16000, "bits": 16}), ([i2s_id], {"mode": mode, "rate": 16000, "bits": 16, "format": cfmt, "ibuf": 2048}),
([getattr(machine.I2S, "NUM1", 1)], {"mode": mode, "bck": 12, "ws": 0, "sd": 25, "sample_rate": 16000, "bits": 16}), ([i2s_id, mode, 16000, 16, cfmt, dfmt], {}),
([i2s_id, mode, 16000, 16], {}),
([i2s_id], {"mode": mode, "sample_rate": 16000, "bits": 16, "channel_format": cfmt, "data_format": dfmt}),
([i2s_id], {"mode": mode, "sample_rate": 16000, "bits": 16}),
([], {"mode": mode, "sample_rate": 16000, "bits": 16}), ([], {"mode": mode, "sample_rate": 16000, "bits": 16}),
([getattr(machine.I2S, "NUM1", 1)], {"mode": mode}), ([i2s_id], {"mode": mode, "rate": 16000, "bits": 16})
([], {"mode": mode}),
] ]
audio_out = None last_err = None
last_e = None for args, kwargs in try_sigs:
for args, kwargs in try_args:
try: try:
audio_out = machine.I2S(*args, **kwargs) return machine.I2S(*args, **kwargs)
break
except Exception as e: except Exception as e:
last_e = e last_err = e
if audio_out is None:
raise Exception("Spk fallback failed: " + str(last_e)) print("I2S Fallback err (exhausted):", last_err)
return audio_out return None
else:
print("ERR: DAC_BUILT_IN missing on this firmware")
return None
except Exception as e: except Exception as e:
print("Spk I2S Error:", repr(e)) print("I2S Init err:", e)
lcd.print("Spk Err: " + str(e)[:20], 0, 60, 0xFF0000)
return None return None
def deinit_i2s(i2s_obj): def stream_http_audio(url):
if i2s_obj: print("Streaming I2S direct from HTTP...")
try: audio_out = init_manual_spk()
if hasattr(i2s_obj, 'deinit'): if not audio_out:
i2s_obj.deinit() print("Speaker init failed, cannot play audio via DAC")
except Exception as e:
print("I2S Deinit Error:", e)
ws = None
def connect_ws():
global ws
try:
ws = websocket.WebSocket()
ws.connect(WS_URL)
return True
except Exception as e:
draw_status("WS Connection Error", 0xFF0000)
return False return False
draw_status("Hold Button A to Talk", 0xFFFFFF)
buf = bytearray(1024)
# Noise monitoring variables
l_db_s = ""
s_db = 30.0
# Location settings - change this depending on where the M5GO is placed
CURRENT_ROOM_ID = "mckeldin"
CURRENT_LAT = 38.986021
CURRENT_LNG = -76.944949
last_db_post_time = 0
while True:
# m5stack core button check
if btnA.isPressed():
if not ws:
draw_status("Connecting...", 0xFFFF00)
if not connect_ws():
time.sleep(1)
continue
draw_status("Listening...", 0x0000FF) if url.startswith("http://"):
rest = url[7:]
# Read and send audio while button is held
audio_in = init_mic()
while btnA.isPressed():
try:
if audio_in:
num_read = audio_in.readinto(buf)
if num_read and num_read > 0 and ws:
ws.send(buf[:num_read])
except Exception as e:
pass
deinit_i2s(audio_in)
audio_in = None
# Button released
draw_status("Thinking...", 0xFFFF00)
try:
if ws:
ws.send(json.dumps({"event": "stop_listening"}))
# Wait for response audio
draw_status("Speaking...", 0x00FF00)
audio_out = init_spk()
while True:
resp = ws.recv()
if resp and isinstance(resp, bytes):
if len(resp) == 0:
break # End of audio transmission
# Briefly pause face animation updates while speaker is playing
# to prevent dropping packets or stuttering
if audio_out:
audio_out.write(resp)
else:
break # Empty or non-bytes response means end
deinit_i2s(audio_out)
audio_out = None
except Exception as e:
draw_status("Error during playback", 0xFF0000)
ws = None # force reconnect next time
draw_status("Hold Button A to Talk", 0xFFFFFF)
# ---------------------------------------------
# NOISE MONITORING LOOP
# ---------------------------------------------
r_db = get_db()
s_db = (s_db * 0.8) + (r_db * 0.2)
db = int(s_db)
if db < 40:
i_c = 0x89b4fa
elif db < 55:
i_c = 0x94e2d5
elif db < 65:
i_c = 0xf9e2af
elif db < 80:
i_c = 0xfab387
else: else:
i_c = 0xf38ba8 raise ValueError("Only http:// supported")
lcd.fillRect(0, 0, 320, 4, i_c)
db_s = "Noise: %d dB" % db
if db_s != l_db_s:
lcd.fillRect(0, 4, 150, 15, 0x222222)
lcd.print(db_s, 5, 4, i_c)
l_db_s = db_s
# --------------------------------------------- if "/" in rest:
# POST DB DATA PERIODICALLY host_port = rest.split("/", 1)[0]
# --------------------------------------------- path = "/" + rest.split("/", 1)[1]
try: else:
now_ms = time.ticks_ms() host_port = rest
if time.ticks_diff(now_ms, last_db_post_time) > 15000: # Every 15 seconds path = "/"
last_db_post_time = now_ms
try:
payload = {
"room_id": CURRENT_ROOM_ID,
"location": {
"type": "Point",
"coordinates": [CURRENT_LNG, CURRENT_LAT]
},
"db": float(db)
}
# Convert ws url from ws://ip:port/ws/voice to http://ip:port/api/study-rooms
http_url = WS_URL.replace("ws://", "http://").replace("/ws/voice", "/api/study-rooms")
import urequests
res = urequests.post(http_url, json=payload)
res.close()
except Exception as e:
print("Failed to post db data:", e)
except Exception as main_e:
pass
time.sleep(0.02) if ":" in host_port:
host = host_port.split(":")[0]
port = int(host_port.split(":")[1])
else:
host = host_port
port = 80
addr = usocket.getaddrinfo(host, port)[0][-1]
sock = usocket.socket()
sock.connect(addr)
sock.settimeout(30)
req = "GET %s HTTP/1.0\r\nHost: %s\r\n\r\n" % (path, host_port)
sock.send(req.encode())
hdr = b""
while b"\r\n\r\n" not in hdr:
b = sock.recv(1)
if not b:
sock.close()
return False
hdr += b
# Read WAV header
header_left = 44
while header_left > 0:
chunk = sock.recv(header_left)
if not chunk: break
header_left -= len(chunk)
in_buf = bytearray(1024)
out_buf = bytearray(2048)
while True:
n = 0
while n < 1024:
chunk = sock.recv(1024 - n)
if not chunk: break
in_buf[n:n+len(chunk)] = chunk
n += len(chunk)
if n == 0: break
samples = n // 2
for j in range(samples):
idx = j * 2
# Read 16-bit Signed LE
s = in_buf[idx] | (in_buf[idx + 1] << 8)
if s >= 32768: s -= 65536
# Convert to Unsigned + Center Offset for DAC
u = (s + 32768) & 0xFFFF
u_lo = u & 0xFF
u_hi = u >> 8
# Map Stereo for built-in MSB
o_idx = j * 4
out_buf[o_idx] = u_lo
out_buf[o_idx + 1] = u_hi
out_buf[o_idx + 2] = u_lo
out_buf[o_idx + 3] = u_hi
try:
audio_out.write(out_buf[:samples * 4])
except Exception as e:
print("Write err:", e)
break
sock.close()
if hasattr(audio_out, 'deinit'): audio_out.deinit()
return True
# ==========================================
# MAIN LOOP
# ==========================================
def run_main():
if not connect_wifi():
return
adc = get_adc()
ws = None
l_db_s = ""
s_db = 30.0
last_db_post_time = time.ticks_ms()
draw_status("Hold Button A to Talk", 0xFFFFFF, f_s_o)
def maintain_ws():
nonlocal ws
if not ws:
draw_status("Connecting...", 0xFFFF00, f_t_2)
try:
ws = ws_connect(WS_URL)
draw_status("Hold Button A to Talk", 0xFFFFFF, f_s_o)
except Exception as e:
print("WS Err:", e)
ws = None
return ws is not None
while True:
gc.collect()
# --- Voice Interaction ---
if btnA.isPressed():
if maintain_ws():
draw_status("Listening...", 0x0000FF, f_listen)
send_buf = bytearray(AUDIO_CHUNK_SIZE)
buf_pos = 0
total_samples = 0
rec_start_us = time.ticks_us()
# Fastest possible analog capture loop
while btnA.isPressed():
try:
raw = adc.read() if adc else 2048
sample = (raw - 2048) * 16
# Fast clamp
if sample > 32767: sample = 32767
elif sample < -32768: sample = -32768
send_buf[buf_pos] = sample & 0xFF
send_buf[buf_pos + 1] = (sample >> 8) & 0xFF
buf_pos += 2
total_samples += 1
if buf_pos >= AUDIO_CHUNK_SIZE:
if ws: ws.send(bytes(send_buf))
buf_pos = 0
except Exception as e:
print("Rec Err:", e)
break
if buf_pos > 0 and ws:
try:
ws.send(bytes(send_buf[:buf_pos]))
except:
pass
draw_status("Thinking...", 0xFFFF00, f_t_1)
actual_rate = (total_samples * 1000000) // time.ticks_diff(time.ticks_us(), rec_start_us)
print("Captured at", actual_rate, "Hz")
try:
ws.send(json.dumps({"event": "stop_listening", "sample_rate": actual_rate}))
# Wait for TTS ready
resp = ws.recv()
if resp and isinstance(resp, str):
msg = json.loads(resp)
if msg.get("event") == "tts_ready":
# Start direct TCP stream immediately
draw_status("Speaking...", 0x00FF00, f_speak)
http_base = WS_URL.replace("ws://", "http://").replace("/ws/voice", "")
audio_url = http_base + "/api/tts-audio"
try:
stream_http_audio(audio_url)
except Exception as e:
print("Stream Err:", e)
draw_status("Play Failed", 0xFF0000, f_s_c)
time.sleep(1)
elif msg.get("event") == "error":
draw_status("Error: " + msg.get("msg", "")[:10], 0xFF0000, f_s_c)
time.sleep(2)
except Exception as e:
print("Comm Err:", e)
try: ws.close()
except: pass
ws = None
draw_status("Hold Button A to Talk", 0xFFFFFF, f_s_o)
# --- Noise Monitoring ---
r_db = get_db(adc)
s_db = (s_db * 0.8) + (r_db * 0.2)
db_val = int(s_db)
i_c = 0x89b4fa if db_val < 40 else (0x94e2d5 if db_val < 55 else (0xf9e2af if db_val < 65 else (0xfab387 if db_val < 80 else 0xf38ba8)))
lcd.fillRect(0, 0, 320, 4, i_c)
db_s = "Noise: %d dB" % db_val
if db_s != l_db_s:
lcd.fillRect(0, 4, 150, 15, 0x222222)
lcd.print(db_s, 5, 4, i_c)
l_db_s = db_s
# Periodic DB Posting
try:
now_ms = time.ticks_ms()
if time.ticks_diff(now_ms, last_db_post_time) > 15000:
last_db_post_time = now_ms
payload_str = '{"room_id":"%s","location":{"type":"Point","coordinates":[%s,%s]},"db":%s}' % (CURRENT_ROOM_ID, CURRENT_LNG, CURRENT_LAT, db_val)
http_url = WS_URL.replace("ws://", "http://").replace("/ws/voice", "/api/study-rooms")
# Raw socket post
h_p = http_url.split("://")[1].split("/")[0]
p_th = "/" + http_url.split("://")[1].split("/", 1)[1] if "/" in http_url.split("://")[1] else "/"
h_b = h_p.split(":")[0]
p_r = int(h_p.split(":")[1]) if ":" in h_p else 80
addr = usocket.getaddrinfo(h_b, p_r)[0][-1]
s = usocket.socket()
s.settimeout(5)
s.connect(addr)
req = "POST %s HTTP/1.0\r\nHost: %s\r\nContent-Type: application/json\r\nContent-Length: %d\r\n\r\n%s" % (p_th, h_p, len(payload_str), payload_str)
s.send(req.encode())
s.close()
del s, req, payload_str
except:
pass
# Idle Face Blinking logic
if db_val > 67:
set_face(f_angry)
else:
t = time.ticks_ms() % 5000
if t < 200:
set_face(f_s_c)
else:
set_face(f_s_o)
time.sleep(0.02)
if __name__ == "__main__":
try:
run_main()
except Exception as e:
print("Fatal error:", e)
lcd.print("Error: " + str(e), 0, 100, 0xFF0000)
website/package.json
+4 -4
View File
@@ -14,14 +14,14 @@
"type": "module", "type": "module",
"dependencies": { "dependencies": {
"@auth0/auth0-spa-js": "^2.19.0", "@auth0/auth0-spa-js": "^2.19.0",
"maplibre-gl": "^5.0.1", "maplibre-gl": "^5.22.0",
"chart.js": "^4.4.0", "chart.js": "^4.5.1",
"chartjs-adapter-date-fns": "^3.0.0", "chartjs-adapter-date-fns": "^3.0.0",
"date-fns": "^2.30.0" "date-fns": "^2.30.0"
}, },
"devDependencies": { "devDependencies": {
"@iconify/svelte": "^5.2.1", "@iconify/svelte": "^5.2.1",
"@sveltejs/adapter-node": "^5.2.10", "@sveltejs/adapter-node": "^5.5.4",
"@sveltejs/adapter-static": "^3.0.10", "@sveltejs/adapter-static": "^3.0.10",
"@sveltejs/kit": "^2.57.1", "@sveltejs/kit": "^2.57.1",
"@sveltejs/vite-plugin-svelte": "^7.0.0", "@sveltejs/vite-plugin-svelte": "^7.0.0",
@@ -29,7 +29,7 @@
"@tailwindcss/typography": "^0.5.19", "@tailwindcss/typography": "^0.5.19",
"@tailwindcss/vite": "^4.2.2", "@tailwindcss/vite": "^4.2.2",
"@types/node": "^25.6.0", "@types/node": "^25.6.0",
"@vite-pwa/sveltekit": "^0.6.6", "@vite-pwa/sveltekit": "^0.6.8",
"svelte": "^5.55.3", "svelte": "^5.55.3",
"svelte-check": "^4.4.6", "svelte-check": "^4.4.6",
"tailwindcss": "^4.2.2", "tailwindcss": "^4.2.2",
website/src/lib/components/AICallModal.svelte
+316
View File
@@ -0,0 +1,316 @@
<script lang="ts">
import { onDestroy, onMount } from 'svelte';
let { onClose }: { onClose: () => void } = $props();
type CallState = 'idle' | 'listening' | 'processing' | 'speaking';
let callState = $state<CallState>('idle');
let ws: WebSocket | null = null;
let stream: MediaStream | null = null;
let audioContext: AudioContext | null = null;
let processor: ScriptProcessorNode | null = null;
let currentAudio: HTMLAudioElement | null = null;
let hasSpoken = false;
let silenceTime = 0;
let errorMessage = $state<string>('');
let hardwareSampleRate = 16000;
// Visualizer data
let currentRms = $state<number>(0);
function cleanupMic() {
try {
if (processor) {
processor.disconnect();
processor = null;
}
if (stream) {
stream.getTracks().forEach((track) => track.stop());
stream = null;
}
if (audioContext && audioContext.state !== 'closed') {
if (typeof audioContext.close === 'function') {
audioContext.close();
}
audioContext = null;
}
} catch(e) {
console.warn('Silent mic cleanup error:', e);
}
}
function playTTS(url: string) {
callState = 'speaking';
currentAudio = new Audio(url + "?t=" + Date.now());
currentRms = 0.06; // Set a much smaller safe static visualizer size
currentAudio.onended = () => {
currentRms = 0;
if (callState === 'speaking') {
// AI is done talking, start listening automatically!
startListeningPhase();
}
};
// Some browsers require explicit play tracking
const playPromise = currentAudio.play();
if (playPromise !== undefined) {
playPromise.catch(e => {
console.error('Audio play blocked:', e);
errorMessage = 'Audio playback blocked by browser.';
endCall();
});
}
}
async function startListeningPhase() {
if (callState === 'idle') return;
callState = 'listening';
hasSpoken = false;
silenceTime = 0;
errorMessage = '';
currentRms = 0;
try {
// Synchronous audio context resume
if (audioContext && audioContext.state === 'suspended') {
await audioContext.resume();
}
if (!stream) {
stream = await navigator.mediaDevices.getUserMedia({ audio: true });
if (!audioContext) return;
const source = audioContext.createMediaStreamSource(stream);
processor = audioContext.createScriptProcessor(2048, 1, 1);
processor.onaudioprocess = (e) => {
if (!ws || ws.readyState !== WebSocket.OPEN || callState !== 'listening') return;
const float32 = e.inputBuffer.getChannelData(0);
const int16 = new Int16Array(float32.length);
let sumSq = 0;
for (let i = 0; i < float32.length; i++) {
const s = Math.max(-1, Math.min(1, float32[i]));
int16[i] = s < 0 ? s * 0x8000 : s * 0x7fff;
sumSq += s * s;
}
ws.send(int16.buffer);
const rms = Math.sqrt(sumSq / float32.length);
currentRms = rms; // Drive the visualizer UI
if (rms > 0.035) {
hasSpoken = true;
silenceTime = 0;
} else if (hasSpoken) {
silenceTime += float32.length / hardwareSampleRate;
if (silenceTime > 1.2) {
finishUtterance();
}
}
};
const dummy = audioContext.createGain();
dummy.gain.value = 0;
source.connect(processor);
processor.connect(dummy);
dummy.connect(audioContext.destination);
}
} catch (err: any) {
console.error('Mic error:', err);
errorMessage = err.message || 'Microphone blocked';
endCall();
}
}
function finishUtterance() {
callState = 'processing';
currentRms = 0;
if (ws && ws.readyState === WebSocket.OPEN) {
ws.send(JSON.stringify({ event: 'stop_listening', sample_rate: hardwareSampleRate }));
}
// Do NOT cleanup hardware mic here as the session is perfectly continuous!
}
function startCall() {
if (callState !== 'idle') return;
callState = 'listening'; // transition state instantly
try {
// MUST CREATE AUDIO CONTEXT SYNCHRONOUSLY IN CLICK HANDLER
const AC = window.AudioContext || (window as any).webkitAudioContext;
audioContext = new AC();
hardwareSampleRate = audioContext.sampleRate;
} catch (e: any) {
errorMessage = "Audio system initialization failed: " + e.message;
return;
}
const protocol = window.location.protocol === 'https:' ? 'wss:' : 'ws:';
const host = window.location.hostname;
ws = new WebSocket(`${protocol}//${host}:8000/ws/voice`);
ws.onmessage = (e) => {
const data = JSON.parse(e.data);
if (data.event === 'tts_ready') {
playTTS(`${window.location.protocol}//${host}:8000/api/tts-audio`);
} else if (data.event === 'error') {
console.error('Agent Error:', data.msg);
errorMessage = data.msg;
endCall();
}
};
ws.onclose = () => {
if (callState !== 'idle') endCall();
};
startListeningPhase();
}
function endCall() {
callState = 'idle';
currentRms = 0;
cleanupMic();
if (ws) {
ws.close();
ws = null;
}
if (currentAudio) {
currentAudio.pause();
currentAudio = null;
}
}
function handleAction() {
if (callState === 'idle') {
startCall();
} else if (callState === 'listening') {
// Force manual send
hasSpoken = true;
finishUtterance();
} else {
endCall();
onClose();
}
}
function handleHangUp() {
endCall();
onClose();
}
onMount(() => {
// Auto-start the call when modal opens
startCall();
});
onDestroy(() => {
endCall();
});
</script>
<div class="fixed inset-0 z-[100] flex items-center justify-center p-4 bg-black/60 backdrop-blur-sm animate-fade-in">
<div class="glass-panel bg-crust/95 border border-white/10 rounded-3xl p-8 max-w-sm w-full shadow-2xl flex flex-col items-center">
<!-- Header -->
<h2 class="text-white text-xl font-display font-medium mb-1">Live AI Assistant</h2>
<p class="text-slate-400 text-sm mb-8 font-medium">
{#if callState === 'idle'}
Disconnected
{:else if callState === 'listening'}
<span class="text-neon-primary animate-pulse">Listening...</span>
{:else if callState === 'processing'}
<span class="text-blue-400 animate-pulse">Agent is thinking...</span>
{:else if callState === 'speaking'}
<span class="text-white">Agent is speaking...</span>
{/if}
</p>
<!-- Visualizer Circle -->
<div class="relative w-32 h-32 flex items-center justify-center mb-10">
<!-- Animated rings based on RMS volume -->
{#if callState === 'listening' || callState === 'speaking'}
<div
class="absolute inset-0 rounded-full transition-all duration-75 {callState === 'speaking' ? 'animate-pulse' : ''}"
class:bg-neon-primary={callState === 'listening'}
class:bg-blue-500={callState === 'speaking'}
style={`opacity: ${0.15 + (currentRms * 6)}; transform: scale(${1 + (currentRms * 8)});`}
></div>
<div
class="absolute inset-2 rounded-full transition-all duration-150 {callState === 'speaking' ? 'animate-pulse' : ''}"
class:bg-neon-primary={callState === 'listening'}
class:bg-blue-400={callState === 'speaking'}
style={`opacity: ${0.25 + (currentRms * 8)}; transform: scale(${1 + (currentRms * 6)});`}
></div>
{/if}
<div class="z-10 w-20 h-20 rounded-full bg-surface0 border-[3px] shadow-inner flex items-center justify-center
{callState === 'listening' ? 'border-neon-primary' : callState === 'processing' ? 'border-blue-500 border-dashed animate-spin-slow' : callState === 'speaking' ? 'border-blue-400' : 'border-surface1'}">
{#if callState === 'processing'}
<!-- Searching/Thinking icon -->
<svg xmlns="http://www.w3.org/2000/svg" class="w-8 h-8 text-blue-400" viewBox="0 0 24 24"><path fill="currentColor" d="M12 2A10 10 0 1 0 22 12A10 10 0 0 0 12 2Zm0 18a8 8 0 1 1 8-8A8 8 0 0 1 12 20Z" opacity="0.3"/><path fill="currentColor" d="M12 2a10 10 0 0 0-10 10h2a8 8 0 0 1 8-8Z"/></svg>
{:else}
<!-- Mic icon -->
<svg xmlns="http://www.w3.org/2000/svg" width="32" height="32" viewBox="0 0 24 24" class="text-white">
<path fill="currentColor" d="M12 14q-1.25 0-2.125-.875T9 11V5q0-1.25.875-2.125T12 2t2.125.875T15 5v6q0 1.25-.875 2.125T12 14m-1 7v-3.075q-2.6-.35-4.3-2.325T5 11h2q0 2.075 1.463 3.538T12 16t3.538-1.463T17 11h2q0 2.625-1.7 4.6t-4.3 2.325V21z"/>
</svg>
{/if}
</div>
</div>
{#if errorMessage}
<div class="bg-red-500/20 text-red-300 text-sm p-3 rounded mb-6 text-center border border-red-500/50 w-full shadow px-4">
{errorMessage}
</div>
{/if}
<!-- Controls -->
<div class="flex gap-4 w-full justify-center">
{#if callState === 'idle'}
<button
onclick={handleAction}
class="bg-blue-600 hover:bg-blue-500 text-white py-3 px-6 rounded-xl font-display font-medium shadow-lg transition-colors flex-1 border border-white/5">
Start Call
</button>
{:else if callState === 'listening'}
<button
onclick={handleAction}
title="Force process audio"
class="bg-surface0 hover:bg-surface1 border border-white/10 text-neon-primary rounded-xl p-4 font-display font-medium shadow-lg transition-colors">
<svg xmlns="http://www.w3.org/2000/svg" width="24" height="24" viewBox="0 0 24 24"><path fill="currentColor" d="M2.01 21L23 12L2.01 3L2 10l15 2l-15 2z"/></svg>
</button>
{/if}
<button
onclick={handleHangUp}
class="bg-red-600 hover:bg-red-500 text-white py-3 px-6 rounded-xl font-display font-medium shadow-lg transition-colors flex-1 flex items-center justify-center gap-2 border border-red-500/50">
<svg xmlns="http://www.w3.org/2000/svg" width="20" height="20" viewBox="0 0 24 24"><path fill="currentColor" d="M12 9c-1.6 0-3.15.25-4.6.72v3.1c0 .39-.23.74-.56.9c-.98.49-1.87 1.12-2.66 1.85c-.18.18-.43.28-.7.28c-.28 0-.53-.11-.71-.29L.29 13.08a.956.956 0 0 1 0-1.4C3.36 8.42 7.46 6.5 12 6.5s8.64 1.92 11.71 5.18c.39.39.39 1.02 0 1.41l-2.48 2.48c-.18.18-.43.29-.71.29c-.27 0-.52-.11-.7-.28c-.79-.74-1.69-1.36-2.67-1.85c-.33-.16-.56-.5-.56-.9v-3.1C15.15 9.25 13.6 9 12 9z"/></svg>
Hang Up
</button>
</div>
</div>
</div>
<style>
.animate-fade-in {
animation: fadeIn 0.2s ease-out forwards;
}
@keyframes fadeIn {
from { opacity: 0; }
to { opacity: 1; }
}
.animate-spin-slow {
animation: spin 3s linear infinite;
}
@keyframes spin {
from { transform: rotate(0deg); }
to { transform: rotate(360deg); }
}
</style>
website/src/lib/components/VoiceButton.svelte
+15 -6
View File
@@ -1,12 +1,18 @@
<script lang="ts"> <script lang="ts">
let pressed = $state(false); import AICallModal from './AICallModal.svelte';
let isModalOpen = $state(false);
function handleClick() { function handleClick() {
pressed = !pressed; isModalOpen = true;
}
function closeModal() {
isModalOpen = false;
} }
</script> </script>
<div class="voice-btn-wrapper"> <div class="voice-btn-wrapper z-40">
<div class="halo-ring halo-ring-1"></div> <div class="halo-ring halo-ring-1"></div>
<div class="halo-ring halo-ring-2"></div> <div class="halo-ring halo-ring-2"></div>
<div class="halo-ring halo-ring-3"></div> <div class="halo-ring halo-ring-3"></div>
@@ -17,7 +23,7 @@
</div> </div>
<button <button
id="voice-assistant-btn" id="voice-assistant-btn"
class="voice-btn {pressed ? 'pressed' : ''}" class="voice-btn {isModalOpen ? 'pressed' : ''}"
onclick={handleClick} onclick={handleClick}
aria-label="Voice Assistant" aria-label="Voice Assistant"
title="Voice Assistant" title="Voice Assistant"
@@ -26,7 +32,7 @@
xmlns="http://www.w3.org/2000/svg" xmlns="http://www.w3.org/2000/svg"
viewBox="0 0 24 24" viewBox="0 0 24 24"
fill="none" fill="none"
class="chat-icon {pressed ? 'icon-active' : ''}" class="chat-icon {isModalOpen ? 'icon-active' : ''}"
> >
<path <path
d="M20 2H4C2.9 2 2 2.9 2 4V22L6 18H20C21.1 18 22 17.1 22 16V4C22 2.9 21.1 2 20 2Z" d="M20 2H4C2.9 2 2 2.9 2 4V22L6 18H20C21.1 18 22 17.1 22 16V4C22 2.9 21.1 2 20 2Z"
@@ -39,12 +45,15 @@
</button> </button>
</div> </div>
{#if isModalOpen}
<AICallModal onClose={closeModal} />
{/if}
<style> <style>
.voice-btn-wrapper { .voice-btn-wrapper {
position: fixed; position: fixed;
bottom: 5.5rem; bottom: 5.5rem;
left: 1.5rem; left: 1.5rem;
z-index: 60;
display: flex; display: flex;
align-items: center; align-items: center;
justify-content: center; justify-content: center;
website/src/routes/+page.svelte
+1 -1
View File
@@ -6,7 +6,7 @@
</script> </script>
<svelte:head> <svelte:head>
<title>EchoNode | Live Map</title> <title>HushMap | Live Map</title>
</svelte:head> </svelte:head>
<div class="relative w-full h-full bg-crust border-l border-white/5"> <div class="relative w-full h-full bg-crust border-l border-white/5">
website/src/routes/history/+page.svelte
+1 -1
View File
@@ -55,7 +55,7 @@
</script> </script>
<svelte:head> <svelte:head>
<title>EchoNode | History</title> <title>HushMap | History</title>
</svelte:head> </svelte:head>
<div class="relative w-full h-full bg-crust border-l border-white/5"> <div class="relative w-full h-full bg-crust border-l border-white/5">
website/src/routes/settings/+page.svelte
+1 -1
View File
@@ -41,7 +41,7 @@
</script> </script>
<svelte:head> <svelte:head>
<title>EchoNode | Settings</title> <title>HushMap | Settings</title>
</svelte:head> </svelte:head>
<div class="relative w-full h-full bg-crust border-l border-white/5 p-6 md:p-12 overflow-y-auto duration-500 transition-colors"> <div class="relative w-full h-full bg-crust border-l border-white/5 p-6 md:p-12 overflow-y-auto duration-500 transition-colors">
website/vite.config.ts
+2 -2
View File
@@ -9,8 +9,8 @@ export default defineConfig({
sveltekit(), sveltekit(),
SvelteKitPWA({ SvelteKitPWA({
manifest: { manifest: {
name: 'EchoNode', name: 'HushMap',
short_name: 'EchoNode', short_name: 'HushMap',
description: 'Campus noise mapping and intervention.', description: 'Campus noise mapping and intervention.',
theme_color: '#0f172a', /* slate-900 */ theme_color: '#0f172a', /* slate-900 */
background_color: '#0f172a', background_color: '#0f172a',