Unlike digital still imagery, which comes in a standardized set of containers and formats regardless of where it was produced geographically around the world, processing video content drawn from across the world presents a vast landscape of different encodings and standards that must be accounted for.
Take the example of an evening broadcast from BBC News London this past January. The actual raw video that is broadcast is in PAL format with a resolution of 704×576 pixels. The typical FFMPEG command to generate still frame thumbnails from a video is:
stime ffmpeg -nostdin -threads 1 -i ./VIDEO.mp4 -vf "fps=1/4" "./FRAMES/FRAME-%06d.jpg"
However, applied to a BBC News broadcast, this yields a squashed image:
Yet when we play this video in any video player, it appears as an ordinary 16:9 widescreen video. So what's going on?
Running FFPROBE on the video yields a clue:
ffprobe -i ./VIDEO.mp4
This outputs:
704x576 [SAR 16:11 DAR 16:9]
The Storage Aspect Ratio (SAR) (the actual pixels stored in the video file itself is quite different from the Display Aspect Ratio (DAR). When a video player plays the video, it loads the raw 704×576 pixel video and rescales it to 1024×576 to display on the screen with the appropriate aspect ratio. In essence, the video is stored with a lower resolution in the file and then resized dynamically when played.
Why doesn't FFMPEG automatically rescale the video for us? The reason is both that scaling irreversibly changes the image and that downstream tasks may expect the original raw resolution, so FFMPEG simply outputs the imagery as-is, leaving it to the user to decide whether they want to rescale the image and how.
We can ask FFMPEG to perform this rescaling for us by simply using a scale filter that keeps the height as-is, but multiplies the width by the SAR ratio:
ffmpeg -nostdin -threads 1 -i ./VIDEO.mp4 -vf "fps=1/4,scale=iw*sar:ih" "./FRAMES/FRAME-%06d.jpg"
This yields the 1024×576 image below, which is what any standard video player will produce from this file.
While the image above is what commercial video players will produce, for some applications reduced filesize is critical, so the inverse can also be done, keeping width the same and downscaling the height based on the SAR ratio:
ffmpeg -nostdin -threads 1 -i ./VIDEO.mp4 -vf "fps=1/4,scale=iw:ih/sar" "./FRAMES/FRAME-%06d.jpg"
This yields the 704×396 image below that retains the same correct aspect ratio, but removes a fair bit of fine detail:
While either approach works, to ensure consistency with how commercial video players display anamorphic PAL video, resizing to 1024×576 is a better approach.