Patriot_Hacks-24/README.md

Patriot Hacks 2024

PatriotHacks 24 - 🏆 Microsoft X Cloudforce & 🏆 Save the World

Repository overview

This repository contains the team's work for PatriotHacks 2024. It is divided into three main folders:

• AI Training/ — Python scripts and small dataset utilities used to prepare and train ML models for trash classification.
• File Converter/ — a small Node.js express service that converts PDF documents into images.
• Project/ — a Next.js web application (called "Carbin") for tracking building carbon footprints and providing a trash-detection interface.

Below are concise descriptions and quick start instructions for each subfolder so you (or another team member) can run or continue development.

AI Training/

Purpose

• Utilities and small scripts used to: extract images from a HF parquet dataset, upload images to Azure Custom Vision, and run model inference examples.

Key files

• hf.py — shows how to load a Hugging Face image classification model (AutoImageProcessor + AutoModelForImageClassification) for inference.
• train.py — script that uploads images from a Hugging Face parquet dataset into an Azure Custom Vision project and triggers training. NOTE: the current script contains hard-coded Azure endpoint/API keys and project IDs; replace these with environment variables before sharing or running.
• ml.py — extracts images from a Hugging Face parquet dataset and writes images into the AI Training/images/ folders by label (biodegradable, cardboard, glass, metal, paper, plastic).
• main.py — another example showing uploading images to Custom Vision and iterating the dataset to create images by tag.
• pdf.py — (small utility) sample code to work with pdf-to-image flows referenced in other parts of the project.

Quick start (local, Python)

1. Create and activate a virtual environment (recommended):

python -m venv .venv
source .venv/bin/activate

2. Install common dependencies used by the scripts (adjust versions as needed):

pip install dask[complete] azure-cognitiveservices-vision-customvision msrest transformers opencv-python

3. Important: set Azure Custom Vision credentials and the project id as environment variables or edit the scripts to load them securely (do not commit keys):

export AZURE_CUSTOMVISION_ENDPOINT="https://..."
export AZURE_CUSTOMVISION_TRAINING_KEY="<your-training-key>"
export AZURE_CUSTOMVISION_PREDICTION_KEY="<your-prediction-key>"
export CUSTOMVISION_PROJECT_ID="<project-id>"

4. Run the scripts as needed. For example, to extract images from the HF parquet source:

python "AI Training/ml.py"

Notes and security

• Some scripts currently contain hard-coded keys and endpoints. Replace them with environment variables or a secrets store before running in any shared environment.
• The code expects a Hugging Face dataset accessible via an hf:// parquet path. Ensure you have appropriate HF credentials or local parquet files when running.

File Converter/

Purpose

• Small Node.js Express service that accepts an uploaded PDF and returns PNG/JPEG frames generated via pdf-to-img.

Key files

• index.js — Express server exposing /convert POST endpoint that accepts a file upload (express-fileupload) and converts the PDF into images with pdf-to-img.
• package.json — lists dependencies required by the service.

Quick start (Node.js)

1. From the File Converter folder, install dependencies:

cd "File Converter"
npm install

2. Run the server (the package.json in this folder does not define a start script, so run node directly):

node index.js

3. The server listens on port 5000 (see index.js). POST a multipart/form-data request with the pdf file field to http://localhost:5000/convert to receive converted images in the response.

Notes

• Consider adding a start script to package.json and proper error handling and file-size limits before production use.

Project/ (Carbin)

Purpose

• A Next.js (v14) application named "Carbin" for tracking building carbon footprints. It includes:
  • Realtime trash detection UI using an inference worker and webcam access
  • Building list and detail pages
  • Firebase-backed data persistence for building metrics

Key files & directories (high level)

• package.json — lists dependencies and provides dev, build, and start scripts.
• app/ — Next.js app routes and pages including API routes under app/api/ (e.g. buildings, chat, pdf-to-image).
• components/ — reusable UI components (navbar, trash-detection UI, theme switch, etc.).
• lib/firebase.ts — Firebase initialization (reads config from NEXT_PUBLIC_* env vars).
• lib/useBuildingData.ts — React Query hooks for fetching and updating building data.
• config/ — app config (site metadata and trash item definitions used for co2 calculations).

Quick start (Next.js)

1. From the Project/ folder, install dependencies and run dev:

cd Project
npm install
npm run dev

2. Required environment variables
   • The app uses Firebase and expects the following env vars (set these before running):

export NEXT_PUBLIC_FIREBASE_API_KEY="..."
export NEXT_PUBLIC_FIREBASE_AUTH_DOMAIN="..."
export NEXT_PUBLIC_FIREBASE_PROJECT_ID="..."
export NEXT_PUBLIC_FIREBASE_STORAGE_BUCKET="..."
export NEXT_PUBLIC_FIREBASE_MESSAGING_SENDER_ID="..."
export NEXT_PUBLIC_FIREBASE_APP_ID="..."

3. Visit http://localhost:3000 after running npm run dev.

Notes and next steps

• The project contains an inference worker integration (inferencejs) used by the trashDetection component to run a model in the browser/worker; you will need the correct worker name/id and API key for that service.
• The config/trashItems.ts file contains per-item CO2e estimates used by the app — adjust values as appropriate.

Contribution & development tips

• Replace any hard-coded secrets (Azure keys, Firebase secrets) with environment variables or a vault before committing.
• Add start/dev scripts where missing (e.g., the File Converter folder) for a consistent DX.
• Consider adding minimal README files inside each subfolder (AI Training/, File Converter/, Project/) if you plan to expand the team.

Contact / Source

Repository maintained for PatriotHacks 2024. The Next.js app links to the GitHub org in Project/config/site.ts.

---

If you want, I can:

• add per-folder README files
• create a requirements.txt for the Python scripts and a start script for the File Converter
• remove or externalize hard-coded secrets into env var usage in the Python scripts

Tell me which of those you'd like next and I'll implement it.