HushMap: AI Services API

The central nervous system linking physical M5GO devices, external Computer Vision tensors, and Conversational NLP APIs synchronously.

--- ## Setup Instructions ### Prerequisites 1. **Python 3.9+** is strictly recommended to support asynchronous typing paradigms. 2. **FFmpeg** must be successfully registered onto your OS PATH environments. This engine handles the core conversions decoding MP3 output arrays into 16-bit, 16kHz Mono arrays natively required for browser contexts: - **Ubuntu/Debian**: `sudo apt install ffmpeg` - **macOS**: `brew install ffmpeg` - **Windows**: Install globally via the [FFmpeg website](https://ffmpeg.org/download.html). ### Environment Initialization Bootstrap the virtual environment and initialize project dependencies: ```bash cd backend python -m venv venv source venv/bin/activate # Windows: .\venv\Scripts\activate pip install -r requirements.txt ``` ### Configuration Tokens Provide runtime keys securely targeting TerpAI context queues and ElevenLabs synthesized avatars within a `.env` dotfile: ```ini ELEVENLABS_API_KEY=sk_... ELEVENLABS_VOICE_ID=JBFqnCBsd6RMkjVDRZzb TERP_AI_BEARER_TOKEN=eyJhbGciOiJSUz... TERP_AI_CONVERSATION_ID=37fa27cc-... MONGODB_URI=mongodb+srv://... ``` To invoke the engine, simply execute Uvicorn across your `0.0.0.0` loopback: ```bash uvicorn server:app --host 0.0.0.0 --port 8000 ``` --- ## Gateway Pipelines ### Full-Duplex Subroutines (`/ws/voice`) This WebSocket proxy establishes a fully integrated multi-turn communication bridge seamlessly interacting between Edge node Hardware APIs (ESP32/M5GO/Browsers) and NLP architectures. 1. **Int16 Byte Array Exchange**: Devices connect to `ws://:8000/ws/voice` and push raw binary frames asynchronously over the socket. 2. **Contextual Augmentation**: The server waits for the `"stop_listening"` payload event to signify a completed audio snippet. That float array is cast through `faster-whisper` and combined seamlessly with real-time `MongoDB` decibel tracking telemetry parameters natively attached into the `TerpAI` user conversation chunk. 3. **TTS Pipeline Rendering**: Output predictions are caught instantly, forwarded natively into the `ElevenLabs` TTS interface rendering `pcm_16000` wav codecs, and alerted back down to clients using a `tts_ready` dispatcher. ### Tensor Vision Endpoints (`/api/vision/room-status`) Leveraging OpenCV bindings layered beneath a YOLOv8-driven bounding box topology detector, this `POST` API analyzes raw camera image buffers returning capacity logic natively. > [!NOTE] > This API calculates euclidean distances algorithmically detecting adjacent proximities between "person" classifiers and untaken "chair" bounding frames to accurately diagnose available seats inside crowded architectures! **Response Output Protocol:** ```json { "room_status": "full", "counts": { "people": 2, "chairs": 2 }, "pairs": [ { "person_index": 0, "chair_index": 1, "distance": 150.5 } ] } ``` --- ## Database Registries * `GET /api/study-rooms/history`: Pulls the active global repository of logged architectural noise measurements captured universally within the preceding 24 hours. Data payloads correspond geographically mapping `GeoJSON` nodes to front-end Mapbox topologies. * `GET /api/study-rooms`: Pulls generic unstructured noise lists directly unfiltered from Cosmos bounds. > [!IMPORTANT] > The browser frontend strictly configures standard Web Audio API's `ScriptProcessorNode` interfaces routing data synchronously to this backend! Wait to close down pipelines until *after* all WS queues have successfully been delivered.