About

XWalk Keyboards uses video feeds from NYC's network of traffic cameras to transform crosswalks into piano keyboards. As pedestrians step on the white stripes of the crosswalk, the app plays the corresponding notes.

HOW IT STARTED

This project won first place at AI Tinkerers' NYC Vision Hack v.2 in Aug 2026. The challenge: leverage AI, computer vision via Roboflow, and deploy via Google Cloud Run to transform raw NYC data into intelligent vision agents.

HOW IT WORKS

The web app uses Roboflow to detect pedestrians in a 511NY traffic cam video in real time. An AI agent running on Cloud Run orchestrates a calibration cycle and uses Roboflow to detect crosswalk stripes on regular intervals. When the web app detects a pedestrian inside a crosswalk stripe boundary, it plays the note corresponding to the stripe's position on a piano keyboard.

MORE DETAILS

Get more details on how the app works at vinceallen.com!