Why I Built This
Searching surveillance footage for a specific person usually means scrubbing through video by hand. I wanted to replace that with a sentence: "the person in a red shirt carrying a backpack" — typed, or spoken in Vietnamese — and have the system point at the right person in the feed.
The project was built at Dai Nam University between 2025 and 2026 and won Second Prize and Most Favorite Project at the Technology Innovation Competition.
How It Works
- Detect — YOLO11 finds every person in the campus camera streams (the user can narrow by camera location and date).
- Describe & match — each detection is cropped and encoded with CLIP ViT-L/14; the text description is encoded in the same space and results are ranked by similarity (e.g. 90%, 83%, 79%) with the time and camera where each person appeared.
- Listen — PhoWhisper transcribes Vietnamese speech so the description can be spoken instead of typed.
Results and query history are stored in SQLite, and the whole thing is driven from a lightweight HTML/CSS/JavaScript web interface. From any result, "Bám theo bằng drone" hands the target off to the UAV tracking system, which then follows the person autonomously.
Key Decisions
- Vision–language matching over hand-written attribute classifiers. Encoding the description and the image crops into one embedding space meant any description works, not just a fixed list of colours and items.
- PhoWhisper for speech. A model trained on Vietnamese gave far better transcriptions than general-purpose alternatives for the target users.
- SQLite and a plain web front end. The focus was the AI pipeline; keeping the surrounding stack simple let me iterate on the models quickly and demo on any machine.
What I Learned
Combining several models into one usable product is a different skill from training any one of them. Latency budgets, failure cases (no match, several matches) and a clear UI mattered as much as model accuracy — and they were what the judges reacted to.