Illy is a Primitive Intelligence
A semi-intelligent listening instrument — tap and hold to speak, release to hear its harmonic reply
As part of:
![ART+TECH Festival [2026]](https://substackcdn.com/image/fetch/$s_!xMBg!,w_140,h_140,c_fill,f_webp,q_auto:good,fl_progressive:steep,g_auto/https%3A%2F%2Fsubstack-post-media.s3.amazonaws.com%2Fpublic%2Fimages%2F8a6d59a4-0891-4053-8aa3-ba2baa30162d_714x391.jpeg)
It understands what it is spoken to in a primitive way and responds to us in a primitive, emotional and simple way both sonically and visually. As opposed to the cutting edge AIs and interfaces like Siri, Alexa, Cortana, OK Google, Illy does not understand natural language, defined commands or even language at all.
It does not respond using fine-tuned intonations to deliver the information gathered from the cloud, it is not integrated with any services. It does not show maps, or pictures about the topic. Instead, as an antithesis, it only tries to understand the sonic properties of the spoken language like the attack, loudness, roughness and pitch. And responds sonically, in a non-language, musical, psychoacoustical way: without the symbolic sounds that we call language but with tonal sounds, timbre, frequency, consonance and rhythm.
Visually, it shapes the visuals focusing on size, speed and color determined by its understanding and creating an abstract visual. Ignoring the symbolic data, like an infant, Primitive Intelligence tries to understand the emotion in the human voice - free from symbols of the language - and responds you with your emotions like sonic mirror to the soul. The interactive experience brings questions about human sound and human-machine interface that we sometimes ignore in this age of information.
lly is a semi-intelligent system powered by Web Audio API and WebGL. It analyzes sonic properties of speech — attack, loudness, roughness, and pitch — rather than language content. It responds through tonal sounds, timbre, frequency, consonance, and rhythm alongside abstract visuals. An emotional mirror, understanding emotion in the human voice free from language symbols.
This version prioritizes faithful cadence and public access inside ARTEX.
How to Interact
Tap and hold anywhere on the canvas to speak or sing into the microphone. Release to hear Illy’s harmonic reply. Headphones recommended. If the captured signal is too weak or unvoiced, the system falls back to a did-not-understand sweep rather than fabricating a response.
Visual Mapping
The iris and outer shell are driven by live or synthesized audio analysis. Loudness expands the core, bass increases ring weight, pitch shifts the internal hue, and interval relationships tint the outer shell. During reply, the visuals follow the synthesized answer — not the microphone.
Original Work
Illy: A Primitive Intelligence was created by illy-pi and exhibited at CODAME ARTOBOTS (2018), SIGGRAPH Digital Art Community (2017), and Bay Area Maker Faire (2017). This ARTEX edition preserves the original listen-and-respond grammar while adapting the interaction for gesture-based control across desktop and mobile.
Next Steps
Future iterations can deepen the harmonic classifier, widen the reply voice palette, and publish additional debug traces for tuning noisy environments.
Artist:
