Stacking videos horizontally is easy according to the documentation of FFmpeg using the hstack filter. This filter allows you to create easily build a mosaic from many videos as long as they have the same resolution. However, in practice, there will be cases where the videos won't have the same dimensions. For this article, we will work with these 2 videos recorded from a web camera stream, one from a teacher and the other from a student. As both of them have different cameras that record at a different resolution, both videos have different dimensions:
The idea is to place them in a single video stacked horizontally, while the final video keeps the audio of both as well. In this article, I will explain to you how to stack 2 videos with different resolution horizontally using FFmpeg.
A. Using a complex filter
The first option to stack the videos horizontally will be through a complex filter. The command to do the trick will be the following one:
ffmpeg -i ./left-side-video.mp4 -i ./right-side-video.mp4 -filter_complex "<CUSTOM_FILTER_HERE>" -map "[a]" -ac 2 ./output-video.mp4The command will receive the 2 videos as input and will use a complex filter that needs to be replaced. The custom filter will be:
[0][1]scale2ref='oh*mdar':'if(lt(main_h,ih),ih,main_h)'[0s][1s];
[1s][0s]scale2ref='oh*mdar':'if(lt(main_h,ih),ih,main_h)'[1s][0s];
[0:a][1:a]amerge=inputs=2[a];
[0s][1s]hstack,setsar=1The filter can be breakdown like this. The scale2ref method scales or resizes the input video based on a reference video. scale2ref supports the same but uses the reference video instead of the main input as the basis. scale2ref also supports the following additional constants for the 'w' and 'h' options:
- main_w (width)
- main_h (height)
For the main input video's sample aspect ratio:
- main_dar
- mdar
The second video stream will be scaled to match the main video. The full command to stack both videos using the filter is:
ffmpeg -i ./left-video.mp4 -i ./right-video.mp4 -filter_complex "[0][1]scale2ref='oh*mdar':'if(lt(main_h,ih),ih,main_h)'[0s][1s]; [1s][0s]scale2ref='oh*mdar':'if(lt(main_h,ih),ih,main_h)'[1s][0s];[0:a][1:a]amerge=inputs=2[a]; [0s][1s]hstack,setsar=1" -map "[a]" -ac 2 ./output-video.mp4The output video in this case will have the following layout:
The canvas is quite big as you can see, with almost a 4K resolution, just to preserve the aspect ratio.
B. Resize videos before stacking
The other option is not for everyone though, however, it is useful as well under certain conditions. FFmpeg offers the hstack filter to stack horizontally the videos, the only requirement is that both videos need to have the same dimensions (height and width). So, keeping in mind that you can have a fixed size for the videos, we will proceed with the following task that will be resizing both videos to a common size. In order to keep the aspect ratio of the videos and the same dimension for both videos, the following commands will do the trick:
# Resize Left Side Video
ffmpeg -n -i ./left-video.mp4 -vf "scale=w=960:h=540:force_original_aspect_ratio=1,pad=960:540:(ow-iw)/2:(oh-ih)/2" -c:v libx264 ./left-video_resized.mp4
# Resize Right Side Video
ffmpeg -n -i ./right-video.mp4 -vf "scale=w=960:h=540:force_original_aspect_ratio=1,pad=960:540:(ow-iw)/2:(oh-ih)/2" -c:v libx264 ./right-video_resized.mp4The command will basically resize the input video to a resolution of 960*540 while keeping the aspect ratio. The area that conflicts with the aspect ratio will be blacked out. Remember that filtering and streamcopy cannot be used together, so you will need to specify the codec of the video, in our case we will use libx264. After resizing both videos, you will have 2 new videos that will be used in a new command that will stack them horizontally:
ffmpeg -n -i ./left-video_resized.mp4 -i ./right-video_resized.mp4 -filter_complex "[0:v][1:v]hstack=inputs=2[v]; [0:a][1:a]amerge[a]" -map "[v]" -map "[a]" -ac 2 ./output-video.mp4The command would generate a new video with a max width of 1920 by 540:
Happy coding ❤️!
