Final Project

The Idea

For my final project, my goal is to create a live slowly moving painting with LEDs. Inspired by Brian Eno's light boxes (see image below), these "paintings" are defined by their generative patterns or colors which may appear to be static are always changing in calibrated ways over long periods of time.

Unlike Eno's light boxes, however, I'm interested in producing a generative pattern and color palette with the use of a live streaming camera. Focused on a slowly changing scene -- like a skyline, the sky, a street -- the video would be astracted by the number of LEDs and diffused through the light box. These images reflects what might be the effect of this kind of system (as a still)

ESP 32

The ESP 32 CAM is a ~$10 microchip produced by the company Espressif Systems that has wi-fi and bluetooth networking built-in and an adapter for a camera module. I'm working on getting two of these to talk to each other via wi-fi so that I can have one esp send pixel information to the LED array. The ESP 32 has 320 KiB SRAM so it can handle both image processing and wi-fi communications simultaneously. Key to this set up is that I ideally would not need a computer to interface or for any computation and these two microcontrollers could operate independently. The ESP 32 cam requires an FTDI connection for serial to USB, so I made a board for the FTDI connector to connect to the camera.

Unfortunately, this wasn't working as expected, and although the board appeared to be powered (as the board's LED light was on), it would not appear in the arudino IDE... I tried numerous solutions -- reinstalling drivers, making sure connections on the board were secure and soldered well, making sure the correct libraries and software was installed and that the board was reset correctly. I also managed to break two of the boards I milled and stuffed.

Eventually, I got one camera working using the example code provided by espressif.

Two ESPs streaming video to independent IP addresses on the same wi-fi network.

Using Neil's code I attemped some basic image processing on the camera feeds and played around with the code to better understand how do this properly and accessed the feed on my phone.

LED Array

I initially began trying to charliplex my own LED array. However, in a time crunch, I ended up buying a prebuilt Neopixel matrix made my Adafruit. In the future, I'll try to make my own array. This also meant that I had to scale down the size of my light box significantly. But, I thought it would be worth first trying to program the lights and to figure out the wireless connection. I had to wait a few days (within this time crunch) for my ESP boards which I ordered to come in, and used an arduino uno board I had lying around to get started. Neopixels has a good library (which I belive can be used for regular LEDs).

I programmed one LED.

And programed color and animations.

Once the new ESPs came in, I had to solder the LED connections to the board and managed to get the code running

Networking

I ran into another problem when trying to get my ESP to both host a camera server and power the LED. All of a sudden my boards stopped booting... I was eventually able to revive my boards when I realized that ESP sometime has this problem when certain pins are conncected while attempting to load new code. Using the ESP NOW protocol, I set one ESP as a server and one as a client. This way I actually did not need a separate router (I had been using my old android to host a wi-fi network). I need more time to figure out how to get the specific data I want to be read by the ESP connected to the LED array. Taking some time to get my CS brain working again.

Diffusion: Seaweed Bioplastic

As we have been going through the weeks, I've concurrently been developing a seeweed bioplastic material. The ingredients are dried kelp, potato starch, water, vinegar, salt and glycerine.

Using a silicone mold, I bake the mixture in the oven at 300 degrees for 20 minutes so that the sheet has unified, but is still rubbery and flexible. At this point, it can be formed into any geometry and dried. Modulating the ingredients will determine strength, stiffness, color and translucency.

I realized that the seaweed acts a really great light diffuser.Better than using acrylic as I had previously planned, I thought I would be cool to use this material I had made myself.

Assembly

Using left over basswood I previously used as formwork for my seaweed experiments, I lasercut strips to make a box to house my LED array and ESP. The enclosure has a rails for both the LED matrix and a diffuser sheet. I ended up using spacers to hold the diffuser sheet at a distance.

Conclusions:

To be totally honest, I kind of ran out of time in terms of devloping this project to the point I would have liked in the short course of the semester. In future iterations, I would like to expand the size of my LED array (wiring individual LEDs together rather than relying on store bought LED matricies is the way to go). Getting the pixel data from the camera to the LED matrix at this point is a software issue that I can eventually figure out. For the time being, I'll have to be happy with the programed lights of the LED matrix.

More to come...