This year is something special! As you’re probably being a witness of, most of the humanity is stuck on self-isolation or even in strict quarantine, which led to unprecedented media consumption. Streaming services usage spiked so high Netflix had to cut their quality to unload their servers. People binge on movies and series and I’m not an exception here, finally watching some classics from the bucket list. This situation reminded me of an idea I’d spotted a couple of years ago. Someone made a beautiful wallpaper from the “cuts” of the “2001: A Space Odyssey”, which is one of my all time favorite movies. Camera work and production design is second to none there, so every frame is really a painting.

I had to try the same idea with other movies and it was clear that doing it manually is not the way to go.
Enter movie-thumbnailer — command line “glue” for ffmpeg and ImageMagick which generates wallpaper from video files laying out a grid of frames with scene changes.

Click for spoilers!

NB: Example pictures are in JPG but the program generates crisp PNGs, so no worries.

How it all works

Movie-thumbnailer depends on two staples of media processing, ffmpeg and ImageMagick. Respective executables ffmpeg and magick must be in your PATH for the program to work. FFmpeg has many tricks up its sleeve and one of them is scene detection algorithm. So, to be honest, you can achieve pretty similar results with barebones ffmpeg, however, if you’re going to set up the resulting image as your desktop/smartphone wallpaper you may want more control, and movie-thumbnailer delivers:

• You can easily generate layouts for multiple resolutions in one pass.
• You can manually check the detected scenes and remove unwanted frames before final collage is generated.
• Program automatically calculates the optimal layout for the given resolution
• There’s an additional step for preprocessing frames leading to increased sharpness and contrast in the resulting collage.
• MT saves the preprocessed frames for future reuse, such as generating wallpapers for additional resolutions.
• The resulting file is a PNG with a transparent background for making the finished pics with any background/additional graphics (see earlier examples).

The program is written in stable Rust using, as usual, the indispensable Clap and the magical Rayon crates which provide the command-line interface and some threading for resource-heavy operations. The code is not pretty, but I never thought I would share it and it works. Yes, I know there’s probably no reason why this couldn’t have been coded in Bash or Python, but I really really like Rust. Compiling small Rust projects is easy, and it’s possible to distribute the binary for benefit of Windows users who have no suitable interpreters.

Usage

Provide movie-thumbnailer with the path to the video file of your choice and the list of wallpaper resolutions you’d like it to generate collages for. The program puts temporary files in the current directory. It creates a folder frames where it puts the result of ffmpeg scene extraction and then pauses. This is the most time consuming part of the process. At this point the user should manually scan the extracted frames and remove unwanted/duplicate/false-identified frames. Upon resuming, movie-thumbnailer uses ImageMagick for preprocessing the remaining frames (mostly for better sharpness and contrast in reduced size) and puts them in the frames_scaled directory. Then optimal size for the thumbnails and the final layout are calculated for each of the resolutions and the collages are created. Resulting files saved as png with transparent background.

The program tries to be “smart” and calculates the layout so there’s some spacing between frames and the composite doesn’t cover all the available space, leaving some margins (6% of the screen width and 18% of the height). At the moment, coefficients are hard-coded in calc_thumbnail_size function. The coefficient for scene detection ffmpeg filter was chosen empirically and set to 0.33. All of this may be added as optional program arguments in future releases, if they ever happen.

Usage example:

movie-thumbnailer -r 2560x1440,1920x1080,1366x768 TheMagickSword.mp4

Full help available on --help switch.

Example wallpaper in the Git repository is generated from the 1962 Bert I. Gordon movie “The Magic Sword” which is in public domain and available from Archive.org.

Download

Again, I provide a single binary for x64 Windows and I encourage everyone to build the program themselves. It’s one cargo build --release away, provided you have Rust installed, which is super simple as well if you use your OS package manager or the official Rust way, the mighty Rustup.

Contributing

Feel free to file a bug report or feature request via Issues.

Possible enhancements:

• Options for margin coefficients
• Option for scene detection threshold
• Ask for removing temporary files on completion
• Open the file browser with the frames directory at the manual thinnig-out stage
• Option for generating the final image with any solid background