Monster Movie: making movie files from microscopy data

What’s the best way to make a movie file from microscopy data?

Maybe you need to generate a movie for the supplementary info for a paper, or insert one into your electronic lab notebook, or to show in a talk. The problem is that the requirements for each of those is different. This situation is compounded by the fact that there are so many options to make movie files and not much guidance on what is the best method. The aim of this post is to present some solutions.

Let’s assume that you have your movie file in Fiji. You’ve added a scale bar, time stamps and any other labels you need. You just want to make a movie file.

To keep things simple, we’ll make an mp4 container file, as it is the most widely used format. In Fiji, your data can be 8-bit, 16-bit, grayscale, color, it doesn’t matter.

The go-to solution

In the lab we use the following recipe:

• Save as AVI – File > Save As… > AVI… – Compression: None. fps: 10.
• Open AVI in Handbrake
• Save as mp4 using Fast 1080p30 preset

For most uses this recipe will give you a movie file that is good quality and a reasonable file size. You may need to adjust fps for the AVI depending on your use case.

This is our go-to recipe because although it involves two programs, it is easy for folks in the lab to do and it produces consistent results.

What to do when this method fails

“Failure” in this context means that the filesize of the mp4 is too big for its intended use. As a rule of thumb:

• Journals require movie files to be less than 20 MB (depends on the journal)
• Our ELN requires 2MB or less
• Powerpoint/Keynote don’t have limits but the larger the file, the more likely it is that you will encounter crashes, so generating 20 MB or less is best.

Solution 1: make the file physically smaller

The simplest thing to do is to crop the movie or resize it in Fiji before you try the “go-to solution” again. If there is less image data then the resulting mp4 will have a smaller filesize.

Example: you have a 900 x 900 x 100 frame stack.

• XY: You could crop to 800 x 800 x 100 if there is an information-free 50 pixel border to your stack. This will be ~79% the size of the original
• Z: Perhaps the full 900 x 900 is needed, but all the action is in the first 80 frames. You can Duplicate frames 1-80 and again be at 80% size of the original
• Z: Perhaps you don’t need to show every frame. Image > Stacks > Tools > Reduce > Reduction factor: 2 will delete every other frame and result in a half size stack.
• XY: You could rescale the movie by 50% to 450 x 450 x 100. Sometimes movies don’t need to be full size. They may be displayed at a smaller size on the screen in the end anyway, so perhaps rescaling is OK.

Remember the goal here is to make something that will display OK and show the important features in the data. It’s not about making a lossless file that can be reanalysed.

While it is simple, it is also the most drastic solution. If you were happy with how things looked in Fiji, you should be able to show that to the world without compromising. So let’s look at some alternatives.

Solution 2: use ffmpeg directly

In the go-to solution, we used Handbrake which is just a wrapper for ffmpeg – a powerful command-line tool for transcoding video and audio. We can use ffmpeg directly to control our output.

On a mac, an easy way to install ffmpeg is via Homebrew (there are lots of guides on how to do this). The most basic command to convert an AVI to mp4 is:

ffmpeg -i movie.avi movie.mp4

This means: use ffmpeg with the input (-i) AVI file and give me an mp4 file called movie.mp4. The program decides what is the best way to do this conversion (and it tells you what settings it has used).

As an example, a 1200 x 1200 x 200 microscopy movie with green and red channels is 1.15 GB. The AVI version is 862.6 MB. The handbrake mp4 is 27.1 MB. The ffmpeg mp4, using defaults is 14 MB.

If your goal was to make something less than 20 MB, this is all that was needed!

What if we need an ELN version (less than 2 MB!)?

There many ways to change the output from ffmpeg (seriously, many ways!). However, a good place to start is to alter the constant rate factor (crf). Values range from 0-51 with 0 being lossless and 51 being worst possible.

ffmpeg -i movie.avi -crf 28 movie.mp4

This gives a file of 587 KB!

The question is, does it look terrible? If it’s not good enough, decrease the number. Note that if the mp4 doesn’t play upon double-clicking in Preview, don’t worry just drag-and-drop into a web browser to play it.

If the command-line is too daunting, there is some mileage in trying different presets in Handbrake. The recommended preset “1080p30” is HD at 30 fps. Dropping the resolution will result in a smaller filesize using the same H.264 codec (which is widely used). Depending on the physical size of the movie, this might not make too much difference. Using ffmpeg directly will give you more control.

Why can’t we just use ffmpeg direct from our TIFF? If we could, this would be one step rather than 3 for the “go-to” method. A disadvantage of ffmpeg is that it can’t take “multipage” TIFF files, i.e. image stacks that we use in microscopy. Even if it could, the scale bars and time stamps may not be preserved; and anyway, there is a 4 GB limit on the TIFF format, meaning that only Fiji could understand your large file. So, it would it be great if we could go direct from Fiji to ffmpeg to generate our file! It would be even simpler than our go-to method.

Solution 3: Fiji to ffmpeg

There are two ways that I know of (feel free to comment if you know other solutions) to go from Fiji to ffmpeg to make an mp4. We are interested in outputting using ffmpeg. Note that a major use case is to use ffmpeg to import movies into Fiji and some solutions are aimed at import rather than export.

The first is the simplest to install. The update site FFMPEG adds I/O (input-output) capability. The output is somewhat restricted though.

The second option is a plugin by Stanislav Chizhik available here (with instructions for installing). This works well for exporting mp4 files direct from Fiji. Select Plugins > Ffmpeg Video > Export Movie Using FFmpeg… in the first dialog type the name including the .mp4 extension.

Then in the next dialog, using “guess by file extension”, there are many options for the export. The dialog allows the input of options using a Key and an Option. So the equivalent to our command above (-crf 28) can be inputted and the mp4 can be saved direct from Fiji!

This resulted in an mp4 that was 609 KB. Similar to the file generated on the command line.

Using this plugin is not without issues. The Fiji-AVI-mp4 workflow works so well because the Fiji to AVI output can handle anything. Here, it’s more fussy. For example, a composite is treated like a single channel, so an RGB version is required. Single channel movies were not a problem. I also found that some settings didn’t give great results. Nonetheless this is a great way to make mp4 files direct from Fiji.

—

The post title comes from “Monster Movie”, an album by Can.