2026-09-23: Physical Computing, ESP32, MQTT, p5.js, Matter.js

Dust Bunnies to Candy: Connecting Physical Inputs to p5.js with MQTT

This article presents an interactive system that connects physical actions with a digital simulation. Participants sweep a handmade surface to create dust bunnies, press a button to blow them toward a funnel, and collect enough of them to activate a motor that releases candy.

The system combines an ESP32 C3 Super Mini, sound sensors, buttons, and a motor with a p5.js and Matter.js simulation. MQTT carries messages between the physical controller and the browser, allowing actions in the real world to change the digital scene and produce a physical reward.

Instruction

This project began as an exercise in inputs, electricity, and electronics. The course materials introduced sensing and actuation methods that were later combined into one connected interaction.

View the class slides

Electronics Design and Fabrication Course resources for building the electronic system. Notion Sensing and Actuation Course resources for sensors, motors, and physical inputs. Notion

System Overview

The interaction moves between the physical and digital worlds in three steps. All physical objects were made by hand with paper clay, giving the electronic system a playful and imperfect form.

Step 1 — Sweep the Floor to Create Dust Bunnies

Sweeping the black surface creates small dust bunnies on the tablet. A sound sensor inside the handmade object detects the sound of sweeping and sends the input to the digital simulation.

The sound sensor installed inside the handmade sweeping object. — Sound sensor inside the paper-clay sweeping surface

Step 2 — Press the Button to Blow Air Upward

Pressing the button applies an upward force to the digital dust bunnies. The flower-shaped funnel appears to pull them in, turning a simple button press into a visible movement on the screen.

A white paper-clay funnel with red dots and a button in its handle. — Handmade flower-shaped funnel and button controller

Step 3 — Trigger the Motor to Release Candy

When the system collects more than ten dust bunnies, it sends a final message to the ESP32. A motor turns the release mechanism so that a piece of candy rolls out as a physical reward.

A three-dimensional drawing of the motor-driven candy release wheel. — Motor-driven candy release mechanism

ESP32 C3 Super Mini, MQTT, and p5.js

MQTT allows the ESP32 and browser to exchange small messages through a broker. Sensor values are published as topics, while the p5.js sketch and physical machine subscribe to the information they need.

Read the MQTT getting-started tutorial

MQTT Setting

I used public MQTT tools to test the connection, inspect topics, and confirm that messages reached the correct subscriber. This made it easier to debug the communication before connecting the complete physical system.

Arduino Client for MQTT PubSubClient API documentation for the ESP32. PubSubClient shiftr.io The MQTT broker used to exchange messages. shiftr.io MQTT.Cool Test Client A browser client for testing publish and subscribe messages. MQTT.Cool

The MQTT test client showing a subscribed topic and a received message. — Testing an MQTT topic in the browser client

public.cloud.shiftr.io
public
public

Arduino Wiring

The circuit connects the ESP32 to a sound sensor, two buttons, a motor driver, and a DC motor. I first assembled the components as a working prototype and then recorded the connections in a hand-drawn wiring diagram.

The ESP32, sound sensor, buttons, batteries, motor, and wiring arranged on a table. — Physical wiring prototype

A hand-drawn diagram showing the ESP32, sound sensor, buttons, motor driver, batteries, and motor connections. — Hand-drawn wiring diagram

p5.js and Matter.js

The browser scene uses Matter.js to simulate gravity, collisions, and upward forces. Each dust bunny is a small physics body, allowing the group to pile up, scatter, and move toward the funnel in response to physical inputs.

View the Matter.js documentation

The Matter.js documentation page and a simple physics demonstration. — Matter.js provides the two-dimensional physics simulation

Animated dust bunnies moving upward and spreading across the simulation. — Applying an upward force to the dust bunnies

The visual tests explored different dust bunny shapes and collision behaviors. These variations helped establish a readable character while keeping the simulation responsive when many bodies appeared at once.

Gray and black dust bunny characters gathering in a physics simulation. — Dust bunny character test

Simple circular dust bunny bodies stacked along the bottom of the canvas. — Circular collision-body test

Black and gray dust bunny characters forming a pile at the bottom of the canvas. — Mixed character and collision test

References

These references supported the MQTT connection and testing process. They provide additional examples for connecting an ESP32 to a broker and inspecting messages during development.

ESP32 Connects to a Free Public MQTT Broker An introduction to publishing ESP32 data through MQTT. EMQX MQTT Intro Course notes introducing MQTT communication. Notion MQTT Explorer A desktop tool for inspecting MQTT topics and messages. MQTT Explorer MQTT Explorer Tutorial A video tutorial for publishing and subscribing to MQTT messages. YouTube