Live Coding Video Synth
Anybody remember Tom Demeyer&#x27;s &quot;Image&#x2F;ine&quot; that he developed in 1996 at STEIM (STudio for Electro Instrumental Music) in Amsterdam?<p><a href="https:&#x2F;&#x2F;v2.nl&#x2F;archive&#x2F;works&#x2F;image-ine" rel="nofollow">https:&#x2F;&#x2F;v2.nl&#x2F;archive&#x2F;works&#x2F;image-ine</a><p>&gt;Image&#x2F;ine is a software for realtime video manipulation.<p>&gt;Image&#x2F;ine is a Macintosh program that allows a user to manipulate visual source material in a live performance environment. Video sampling and playback, keying, displacement and other effects are available with video (live and recorded), QuickTime movies, text, scanned images, image files with alpha channels. Unlike digital video editing programs such as Adobe Premiere, Image&#x2F;ine works in real time. There is no compression slowdown, no rendering time - the digital filtering effects are immediate and controllable through Midi.<p>&gt;The software instrument for realtime video manipulation was developed at STEIM by Steina Vasulka and Tom Demeyer (1996-2001).<p><a href="https:&#x2F;&#x2F;image-ine.org&#x2F;" rel="nofollow">https:&#x2F;&#x2F;image-ine.org&#x2F;</a><p>&gt;Image&#x2F;ine, developed at STEIM from 1997, in close collaboration with Steina, was the first piece of software (for normal computers) that allowed users to manipulate uncompressed video in real time. Limited, at the time, to 320x240 pixels at some 10 frames per second (the Macintosh 8600 was the dream machine), it nevertheless proved a point: artistic quality and stage guts made lack of frame rate and image quality of secondary interest; real time video manipulation could be done! Soon others followed, all with their strengths and weaknesses. What set Image&#x2F;ine apart, and sets ImX apart, is that this is software for video people, not for musicians and not for programmers.<p><a href="https:&#x2F;&#x2F;vimeo.com&#x2F;41196405" rel="nofollow">https:&#x2F;&#x2F;vimeo.com&#x2F;41196405</a><p>&gt;Steina &#x2F; Developing Image&#x2F;ine with Tom Demeyer at STEIM, Interest in displacement and morphing, Dislike of rendering &#x2F; 1997NOT YET RATED<p>&gt;Steina talks about developing the real-time image processing software Image&#x2F;ine with Tom Demeyer while acting as Artistic Director at STEIM in Amsterdam in 1996.<p>&gt;(Editor&#x27;s Note) When initially developed in 1996, Image&#x2F;ine performed real-time capture, processing and playback of video on a Macintosh PowerPC 8600 containing the stock video i&#x2F;o card. Though it was the first software of its kind, development has stopped and many aspects of image&#x2F;ine have now been incorporated into the software Isadora, written by Marc Coniglio.<p><a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=22788773" rel="nofollow">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=22788773</a><p>DonHopkins on April 5, 2020 | parent | context | favorite | on: Csound: A sound and music computing system<p>Here are some notes I wrote about some visual programming languages and real time performance tools for music and video that I wrote to the LEV mailing list in 2000 (and some additional notes and email I saved over the years).<p><a href="https:&#x2F;&#x2F;www.donhopkins.com&#x2F;home&#x2F;archive&#x2F;visual-programming&#x2F;bounce-notes.txt" rel="nofollow">https:&#x2F;&#x2F;www.donhopkins.com&#x2F;home&#x2F;archive&#x2F;visual-programming&#x2F;b...</a><p>That link also includes some interesting discussion with Jaron Lanier about visual programming language design.<p>Image&#x2F;ine was a software instrument for realtime video manipulation and MIDI processing from STEIM (Studio for Electro-Instrumental Music) in Amsterdam, by Steina Vasulka and Tom Demeyer (1996-2001). It ran on a Mac, and you could write plug-ins for it.<p><a href="https:&#x2F;&#x2F;steim.org&#x2F;" rel="nofollow">https:&#x2F;&#x2F;steim.org&#x2F;</a><p><a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;STEIM" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;STEIM</a><p><a href="https:&#x2F;&#x2F;v2.nl&#x2F;archive&#x2F;works&#x2F;image-ine" rel="nofollow">https:&#x2F;&#x2F;v2.nl&#x2F;archive&#x2F;works&#x2F;image-ine</a><p>Hookup is a real time visual programming language for controlling MIDI and playing music and rendering graphics, developed by David Levitt (who shared an office with Miller Puckette at MIT), which also incorporated the Macromedia Director MMP player plug-in (so it could read in Director files and play their content under visual program control).<p><a href="http:&#x2F;&#x2F;www.sdela.dds.nl&#x2F;sfd&#x2F;isadora.html" rel="nofollow">http:&#x2F;&#x2F;www.sdela.dds.nl&#x2F;sfd&#x2F;isadora.html</a><p>&gt;Mark Coniglio: Here&#x27;s a bit of history. In 1986 my soon-to-be mentor and Interactor collaborator Mort Subotnick had just come from a residency at MIT where he was using a program called Hookup created by a student there named David Levitt. Hookup was the first program I know of that used the &quot;patch-cord&quot; metaphor, i.e., modules that manipulate data are linked by virtual wires, the connection of which is determined by the user. For those in the world of early analog, patch-cord programmed synthesizers, this was a familiar interface. Mort was using David&#x27;s program to do tempo following of MIDI instruments -- this allowed him to lock hardware MIDI sequences to the tempo of the live performers. I was a composition student at CalArts at the time, and word had gotten around that I was a good programmer. So Mort contacted me to see if I could hardcode some of the ideas he had implemented in Hookup on a Mac, so that he could use them in his next performance. That program (used in Mort&#x27;s 1987 multimedia work &quot;Hungers&quot;) would eventually become Interactor. Mort designed the functionality of the early versions, but I became more influential in the design as time went on. [...]<p>&gt;Mark: Yes, that&#x27;s true and importantly a kind of creative intuition was creeping back in through the development of these new visual interface possibilities for software. Part of the thing I reacted to in Hookup was the way you could easily drop modules into the program and try things; a lot like you could do with the patch-cord synthesizers. I may not have realized it explicitly then, but this ability to program improvisationally allowed for that kind of artful playfulness that is so important. So I set out to make a similar user interface for Interactor. The creation of Isadora was a natural outgrowth of Interactor. In 1996 Troika Ranch had a two-week residency at STEIM, where I first saw Tom Demeyer&#x27;s real-time video processing program Image&#x2F;ine. I first started using Image&#x2F;ine in concert with Interactor, because Image&#x2F;ine didn&#x27;t allow the kind of complicated interactive decision making that I was used to having in Interactor. So, Interactor would process the MIDI data from my interactive sensors, and then tell Image&#x2F;ine what to do. By 1998 I was using Image&#x2F;ine in a major way in my performances with Troika Ranch. [...]<p>&gt;Mark: Isadora and Max both inherit the modules linked by the patch-cord metaphor from Hookup. But unlike Max, each Isadora module shows the parameter names and current values for all of its inputs and outputs, and many modules give real-time graphic feedback about their operation. This is important from the perspective of helping new users understand what&#x27;s going on right away. But perhaps the biggest difference is that Max is a very powerful, open-ended programming language in which you could solve any number of problems. Isadora isn&#x27;t that. It is a lot like Interactor in that each module is essentially a macro that accomplishes some specific function. This approach helps people who are just beginning to do this kind of work, as it means that useful functionality is already embodied for you and it&#x27;s very easy to start doing things and getting interesting results quickly (like with Image&#x2F;ine). Max allows the most flexibility, but may be somewhat more difficult to program because more things have to be built up from scratch. Isadora offers somewhat less flexibility, but is still open-ended enough for the user to imprint his or her aesthetic on the result.<p>While working at VPL, David also integrated the MMP library into Body Electric (below) to make Bounce (also below). The MMP player plug-in is what eventually became Macromedia Shockwave once it was plugged into the web browser (which wasn&#x27;t nearly as fun as plugging it into a full fledged real time interactive visual programming language).<p>Body Electric is a real time visual programming language for VR and music and hardware control, developed at VPL by Chuck Blanchard, which Jaron Lanier and others used to create virtual reality simulations and virtual interactive musical instruments.<p><a href="http:&#x2F;&#x2F;www.jaronlanier.com&#x2F;vpl.html" rel="nofollow">http:&#x2F;&#x2F;www.jaronlanier.com&#x2F;vpl.html</a><p><a href="https:&#x2F;&#x2F;wiki.c2.com&#x2F;?JaronLanier" rel="nofollow">https:&#x2F;&#x2F;wiki.c2.com&#x2F;?JaronLanier</a><p><a href="https:&#x2F;&#x2F;www.vrs.org.uk&#x2F;virtual-reality-profiles&#x2F;vpl-research.html" rel="nofollow">https:&#x2F;&#x2F;www.vrs.org.uk&#x2F;virtual-reality-profiles&#x2F;vpl-research...</a><p><a href="https:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20050228021115&#x2F;http:&#x2F;&#x2F;www.well.com&#x2F;user&#x2F;jaron&#x2F;vr.html" rel="nofollow">https:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20050228021115&#x2F;http:&#x2F;&#x2F;www.well.c...</a><p><a href="https:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20040414174418&#x2F;http:&#x2F;&#x2F;www.well.com&#x2F;user&#x2F;jaron&#x2F;instruments.html" rel="nofollow">https:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20040414174418&#x2F;http:&#x2F;&#x2F;www.well.c...</a><p><a href="https:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20050211182929&#x2F;http:&#x2F;&#x2F;www.well.com&#x2F;user&#x2F;jaron&#x2F;knittalk.html" rel="nofollow">https:&#x2F;&#x2F;web.archive.org&#x2F;web&#x2F;20050211182929&#x2F;http:&#x2F;&#x2F;www.well.c...</a><p>Body Electric supported all kinds of interesting input and output devices, including MIDI, sending and receiving UDP packets over Ethernet, loading Swivel3D 3D skeleton files and animating them, sending their state over the network to a pair of SGI workstations for rendering with the Isaac rendering engine to the VPL &quot;EyePhones&quot; VR headset (one SGI workstation per eye, with a Mac to run the simulation), VR input devices like VPL&#x27;s DataGlove and Body Suit, 3D input devices like the Ascension Flock of Birds, Polhemus, and Spaceball, 3D audio output devices like the Convolvotron, and lots of other cool stuff.<p><a href="https:&#x2F;&#x2F;est-kl.com&#x2F;manufacturer&#x2F;ascension&#x2F;flock-of-birds.html" rel="nofollow">https:&#x2F;&#x2F;est-kl.com&#x2F;manufacturer&#x2F;ascension&#x2F;flock-of-birds.htm...</a><p><a href="https:&#x2F;&#x2F;polhemus.com&#x2F;" rel="nofollow">https:&#x2F;&#x2F;polhemus.com&#x2F;</a><p><a href="http:&#x2F;&#x2F;www-cdr.stanford.edu&#x2F;DesignSpace&#x2F;sponsors&#x2F;Convolvotron.html" rel="nofollow">http:&#x2F;&#x2F;www-cdr.stanford.edu&#x2F;DesignSpace&#x2F;sponsors&#x2F;Convolvotro...</a><p><a href="https:&#x2F;&#x2F;www.researchgate.net&#x2F;publication&#x2F;253921765_The_Convolvotron_Real-time_demonstration_of_reverberant_virtual_acoustic_environments" rel="nofollow">https:&#x2F;&#x2F;www.researchgate.net&#x2F;publication&#x2F;253921765_The_Convo...</a><p>Bounce is a derivative of Body Electric, that David Levitt integrated with the MMP player, and that I helped him develop, and used for some fun projects. Extremely weird and esoteric, but still one of the must productive, delightful visual programming languages I&#x27;ve used!<p><a href="https:&#x2F;&#x2F;medium.com&#x2F;@donhopkins&#x2F;bounce-stuff-8310551a96e3" rel="nofollow">https:&#x2F;&#x2F;medium.com&#x2F;@donhopkins&#x2F;bounce-stuff-8310551a96e3</a><p><a href="https:&#x2F;&#x2F;wiki.c2.com&#x2F;?BounceLanguage" rel="nofollow">https:&#x2F;&#x2F;wiki.c2.com&#x2F;?BounceLanguage</a>