Collage Montage (CM) Sketches and Studies 2025-06-05

Table of Contents

Preface

This is a folio of latest sketches and informal studies using a collage-montage generator algorithm I’ve implemented in Max Jitter and realized typically on a Mac Mini M4 in realtime. These variations are using 3d rotation of photos and video. These are to a large extent technical exercises to understand and demonstrate techniques and not representations of specific works.

A folder of images and videos is supplied and then rendered by the patch.

This combines several areas of interest:

use of replicas, shadow and virtual objects
scalability – algorithms that work at a variety of scale
motion studies
means for collaboration, orchestration and choreograpy
software defined spaces and environments
real-time generation rather then from recordings
framework for spatializing sound

These sketches and studies include audio using a simple technique of using the cross fade variable to control volume from extent video.

This set of examples uses a random walk to control rates, size (scale, aspect ratio distortions), position, rotation, cross fade (a form of transparency). There are some variation to one of the control variable – the length of delay before selecting the next rate and target value for each of the control variables.

Areas for further development:

Use and selection of bespoke rather than “found” images
Improved selection of the images. Currently each of these videos is random selection of files from a folder containing raw files.
Customization and refinement of the control variables. In these current sketches all variable except one share the same range
- adjust the maximum in the range of the size (scale) will increase the overlapping of the images and extent to filling the frame as the size is magnified.
Extend use of the variables used to control position, scale, rotations for audio generation and modification
Use materials without rectangular framing (e.g. transparent backgrounds)
Further work here to show opportunities for interactivity (for performance and installation)
Multiple live camera feed and delay
More articulate control over the range of the variables E.g. modify size and other variables over time including evolution of the selected images.

Mom and Dad 1944-1946

I had longed been fascinated with a collection of family photos from my father and mother ca. 1944-46. This latest adds rotations and variance to aspect ratio which abstract the image from early sketches that were less abstract.

Mexico

These were selected as a variation on color pallette and ready-at-hand Go-pro video.

Nurbs

Around November of 2023, I started working and a series of Jitter generative algorithms that generated shapes and modified those shapes in real-time with co-generation of sound. A live version of one of these configurations was adapted for an interactive installation at the Eclipse Gallery in North Adams for a week end of July 2024.

The sound tracks in these collections are generated in parallel with the images using quantization of the end points defining the meshes mapped to microtonal pitches using various scales. So far, there is little consideration to composition and orchestration.

(missing using the straw fields examples)

Rainforest V

Rainforest V is in the collection of MAC Lyon, Arter Istanbul, MoMA NY and MdM Salzburg. Each variation consists of 20 transduced objects, sound distribution system and sound library. Here a sample of videos from an intial two of the 4 installations are used as input to the collage-montage generator.

RFV Lyon

The audio is spatial recording from a Sony FDR-AX53 with spatial mic here flattened to stereo.

RFV MdM Salzburg

For reason to be determined, this is rendered by gdrive at 360p. Original is 1920×1080.

Collaborative Challenges

There is an intended dual use of the algorithm and implementations sketched out out here. This area of exploration is considered both for individual use in my work and offered as platform for collaboration using sound and images with others. While the examples here are taking materials from exent collaborations, these have not been agreed or discussed with others to date (June 2025).

The techniques here are significant alterations from the original source of the materials and are not intended as improvements or representations of those works.

Additional Technical Notes

Click here for additional notes. These are some technical detail (mostly as aid to memory for myself).

There is an additional video files (cm-rf-lyon-2025-06-05.mov) required 30hours after gdrive upload process (suspect vagaries of gdrive for taking more than 24 hour and completed as of June 7.

Google Drive is used on these web pages as a inexpensive and effective service to store and stream files via the web though not without some difficulties. Usually, gdrive processes uploaded files and makes the video available from sharing within minutes of being uploaded depending of the size of the files. It is then a simple matter to obtain the html embed snippet and add to these pages. Some of these files are still be processed after more than 12 hours in google drive. These are 3+ gb some three times larger that other files. May have to reload to see if these got hung in google due to something buggy.

This is a snapshot of the “worker” patch that displays each of the video and images. This is called from a top level patch cm-masterV1 using the Max polyphonic object (poly~) typically used from multiple “voices” here used as a mechanism for spawning variable number of these image renderers.

The random walk for each of the variables (crossfade, size (scale), horizontal and vertical position) are generated by these objects:

One area of improvement is adding “filigree” to the start and end each segment of the walk which should tend to reduce some of the mechanical nature of the motion.

These are assembled in visual layers by this object:

Other collections to be added

…from photo and video library – other examples ClusterFields, LocaleS, RF5 Salzburg…

Earlier Work – LocaleS3

Here is are some earlier attemps in this direction.

This clip is from documentation for the LocaleS3 project – a tryptich with Michelle Jaffee and David Reeder. I took available video and composited in Jitter. The work uses UWB location tracking headphone system from NousSonic. One of the un-implemented areas of interst stemming from this was being able to use proximity, orientation, and direction as additional compositional variables.