A Synesthetic Experience 

An Extension of Our Sense: A Synesthetic Experience | 2018

Shirley Leung, Karen El Asmar, Zhizhen (Jerry) Tan 


One of our most basic human functions is the ability to smell and pick up scents. The way our brain recognizes scent is different from that of machines. Machines detects scent by picking up molecules in the air and then mapping this to a database of human perceptions of scents. Our objective is to explore the relationship between senses in the machine. We’re employing synesthetic relationships between sight and scent to evoke feelings of familiarity. By creating a cross-sensory experience that enhances our visual perception, we’re also extending our research to potentially teach a machine how to smell by pairing computer-visioning with human synesthesia. If humans can sense scents and embed information within this scent, can this be anew way of storing information, or exist as a new form of multimodal communication?

Tools: Arduino + Camera + p5.js+ ml5.js + Peristaltic Liquid Pumps + L298N Dual Full-Bridge Drivers

Using Format