Some examples of video superposition/collage :
******
Some little experiments I want to try :
With the Chip Shouter, to mess with the Arduino using VGAX and the FPGA doing VGA and HDMI and see if I can create any interesting visual glitches :
With this Geometric Optics Demo we have at the Cyber Campus, to assign different bit depths of a video signal to different lasers, send the lasers around mirrors and lenses etc, then decode it with laser diodes at the other end. I wonder if I can aim at different logic blocks as they are physically placed in the FPGA ?
Good post on the subject : https://electronics.stackexchange.com/questions/359373/how-to-transmit-data-with-a-laser-beam
Laser sensor module with lens and amp : https://www.amazon.fr/Waveshare-Laser-Receiver-Sensor-Transmitter/dp/B00NJNYQ9G
For the HDMI bending experiment :
Do imbalances of different degrees create different glitches ? If so, I should map every different possible imbalanced 8 byte combination to a different pixel to explore what pattern it makes !
**********
The tech workshops and classes I’ve done so far :
In terms of being more capable and employable as a teacher, I’d like to go more towards software, especially video games, renderers, shaders, and AI.
To that end, working on a simple procedurally generated cave exploring video game :
https://editor.p5js.org/merlinmarrs/sketches/pfvBYUSGB
I want to make a workshop on 2D game design, based on basic math principles and early video game hardware (so, no libraries, must be smaller than some size, uses tiny memory for sprites and fonts). Check out this under 13KB game design challenge : https://js13kgames.com/
https://www.gamemath.com/book/vectors.html
****
Just checked out the wiki on Video Synthesizers (https://en.wikipedia.org/wiki/Video_synthesizer) and Video Art (https://en.wikipedia.org/wiki/Video_art) and learned some things :
- Doing things in “Real Time” distinguishes video synths from 3D renderers. Video synthesis in analog and even digital was done before computers.
- In Chicago, there were video experiment sessions called “Electronic Visualization Events”
- “position, brightness, and color were completely interchangeable and could be used to modulate each other during …This led to various interpretations of the multi-modal synesthesia of these aspects of the image in dialogues that extended the McLuhanesque language of film criticism of the time”
- ” Today, address based distortions are more often accomplished by blitter operations moving data in the memory, rather than changes in video hardware addressing patterns. “
****
Cool article on glitches http://beza1e1.tuxen.de/lore/sparkling_tile.html
- The experts solving the problem often are faced with difficult to reproduce glitches. The solution is often to be patient, and observe unexpected correlations systematically. Also deeply understanding the functioning of the systems. Having logs is important to see what happened right before the crash.
*****
Who is my audience ? I am reading lots of posts from hacker news, am I now trying to appear to a more engineering / larger tech literate crowd on the internet ?
How do I make something that people want, how can I build empathy as a designer instead of staying at the selfish research personal project phase ?
I think I am also thinking about how all my interests in tech all boil down to illusions. It is all just man made stuff. I should embrace that it is all a construction and just make pretty things instead of searching for “truth inside the black box”.
***
From this talk ( ), code should be written in the way that that coding language is written in. You shouldn’t try to write python-like code in javascript. Verilog code should look like verilog code.
******
Some mindbending perspective 2D/3D video games :
Viewfinder
Superliminal
Beautiful indie games:
Sword and sorcery
World of Horror, which is made entirely in MS paint :
Papers please :
Thomas was alone (so beautiful how you can play as a pixel, and something so minimal works because of the text overlay and audio):
Pacific Drive (I love the interfaces)
Baba is you (beautiful coding game) :
Minit (love the field of vision with torch effect from Rogue) :
Rainworld :
Keep driving
Some notes :
- The attention to the players experience, and very consciously introducing them to the video game world in a step by step way.
- The importance of sound design in creating atmosphere.
- The fact that it is purely illusion creation, story telling. They let the player live a fantasy.
- There are a finite number of types of games, some include resource foraging, puzzle solving, side scrolling jump based games.
- The importance of lighting, especially glowing lights, shadows, and fog in creating atmosphere.
- 8 bit and 1 bit games are in fashion
I have been looking at indie games and their construction too. Small scale video games marry tech and art in some a unique way. Game Dev Conf (youtube.com/@Gdconf) is cool to see creators talking about their games.
On the rainworld game design (youtube.com/watch?v=sVntwsrjNe4). The artist/developer Joar Jakobsson advocates for using programming to create illusions, finding pragmatic, instead of purist, solutions. For their creatures, they have a physics + AI engine first and then a computationally expensive and more complex stylized layer is added on to that. Performance optimization concerns appear to help the creative development of the game. Another insight is that because they are working with made up creatures, no one can fault their representations (we could tell if a horse was moving funny).
On the Inside rendering (youtube.com/watch?v=RdN06E6Xn9E&t=1885s). talks about various tricks to create the illusion of fog, quickly render flashlight cones, etc. often in a struggle against 8 bit values that leave bands on the screen. Tricks seem to involve adding different kinds of noise and down/up sampling, using smooth minimum functinos for lighting to eliminate these. The connection between the rendering pipeline, and its bandwidth constraints, and the artistic atmosphere are intertwined. Interesting that this 2.5D game fixes the camera and therefore this allows the artists working on the game to know exactly what will be seen by the player. They also mention how many effects they “stole” from other game developers and are sharing theirs in an open source ethos.
A short Hike GDC talk describes how making the project short and easy to develop, keeping it improvisational, testing + sharing in process work and incorporating feedback, and having a deadline, made it all possible.
Return of Obra Dinn GDC talk describes how the creator works with an engineering “problem-solving” mentality, and that chosing a technical constraint becomes a fun problem to solve. Describes himself as OCD, and so a basic mechanical core task (often bureaucratic and ostensibly super boring) organizes the game. Inspired by games of old. Has a long term vision of game design, make something so different from other games that this is the only option, and that this won’t change any time soon. Every time he makes a game, picks a completely new style and theme. Don’t be fixated on making everything perfect, even with bugs a thing can become appreciated and a classic. Do something you would enjoy playing.
The GDC for the design of FireWatch talks about making decisions based on how the game should function based on the priorities. If something isn’t directly in line with the main priority, leave it out of the game. They create a boolean database of decisions made by the player as they move through the game that impacts future possibilities of the game space. “Your design must … survive contact with players who have a lifetime of different experiences and assumptions than you”
GDC for the FAITH horror video game has the creator explain that lox fi pixel art is perfect for the horror medium because it is based on the feeling “OMG WTF is that thing?” which is very achievable with a tiny number of pixels. Also, the importance of using black background to create an open space feeling with objects placed within this field.
GDC for 8bit art talks about Color cylcing (https://en.wikipedia.org/wiki/Color_cycling)
GDC for Baba is you talks about the importance of quick game jams in developement.
GDC on solo development (youtu.be/YaUdstkv1RE?si=psfnotyjUE3g4kGF), one speaker talks about the cone of possibilities arising from the pie chart of constraints :
This made me think about my synth project :
- I need a deadline !
- I need to set limits on what the project is and is not ! (Only taking input video and memory sampling, no sound, with mini screen though because otherwise it’s not clear what it is doing)
- I need an audience or a platform (like Steam, or Playstation games) that I’m aiming for. Perhaps this was what Marc was thinking about regarding the modular synth euro rack standard.
- It doesn’t have to be perfect, just good enough ! Several game devs said this.
- The last 10% clearly will take the most time : 90% !!
- I also need to use other people’s work (and cite it) in the form of designs and libraries.
********
Back to a physical object : I do indeed need a small screen and a battery pack on this device for it to be legible and demo’able to the majority of people. It is not good enough that it works when plugged in to a bunch of things. To explore this avenue, I need to:
- pick up where I left off with the Gameboy style board that I didn’t really test to get the screen working.
- Get the rpi stable-screen sampling working on my current setup.
- Make a new design that incorporates everything I want but is less in the gameboy aesthetic and has more keys laid out tightly together :
Here’s an attempt at a diagram showing the working of the sampler :
********
Stockhausen
And a great series of video by Hainbach, including some rudimentary vinyl recording devices, bar code card samplers, flanging with two synchronized tape players, and a “Stockhausen speed-run” reproducing beautiful sounds with essentially just a sine oscillator, a splicer and a tape recorder. I realized that this is exactly what I was going for with my early prototypes of the video samplers, trying to construct compositions out of simple parts, and giving one hands-on experience of a time based medium.
*****
Checked out the world of hardware video samplers and discovered that most people use their laptops, so this may be an opportunity !
The OG is the Korg Kaptivator :
Snapbeat https://snapbeat.net/snapbeat-simple-lofi-hardware-sampler/
Here is an open-source one called the r_e_c_r, based on a rpi :
www.20.piksel.no/2020/11/21/r_e_c_u_r-an-open-diy-video-sampler/
And Gieskes arduino video sampler : https://gieskes.nl/visual-equipment/?file=gvs1#p3
I also forgot about Teenage Engineering’s (audio) sampler :
****
I’m preparing a submission for the art track of DIS 2025, I am basically proposing slow oscillating VGA monitors. I’m referencing the experimental video art obsession with flowing water and connecting that to the idea of the “oceanic” to line up with the conference theme.
To get most of these effects I used a VCO (like part of a CD4046) and a rudimentary switched capacitor circuit taking in a second out of phase waveform.
Trying to show how visitor’s could interact with the oscillations, and could try to balance them in the middle of the screen :
Showing the circuit for generating the patterns :
****
Watched this teardown of the Formlabs 4 printer, gives an idea of what real product engineering is. There is a big gap between making a quick prototype and making a fully integrated, ready for public use, machine. There is also a big team !
youtube.com/watch?v=cbm03jtWWL0
The conversation turns around the design and manufacturing considerations, the iterations and complexity of the project. It’s a combination of electronics, optics, mechanical engineering, and integration engineering (dissipating heat through the machine, cable management). The design philosophy of making the machine user serviceable is a big constraint. Desgining for things that aren’t supposed to happen, like resin leakage, but may happen. A range of materials (aluminum, stainless steal, magnets, plastics, and manufacturing processes. There are people who work for months on a single component of the machine, with many iterations. They seemed to try to iterate in house before making expensive tooling. Sensors throughout, for motor rotation, resin level, to detect things being in place. Balancing concerns for robustness, weight, cost, stress resistance, leak protection, fire protection, air flow, heat flow, keeping things level and rigid, making everything work with the existing resin range they have. Using video recordings to observe the operation of parts. Lot of decisions to be made when faced with different design options.
****
FPGA project called Gamebub does everything in python running on an FPGA with a small screen and memory I think :
eli.lipsitz.net/posts/introducing-gamebub/
Also check out this interface for programming FPGAs via USB like they are external drives. Uses CRAM programming :
https://github.com/psychogenic/riffpga