'UNSW - Science

UNSW - Science - HPC

Using FFMPEG to convert video files

FFmpeg is a free software project that produces libraries and programs for handling multimedia data that has been installed on Katana. In particular the ffmpeg command line program can be used for transcoding multimedia files.

The basic syntax of the FFmpeg command line is

ffmpeg -i input_file output_file

Fmpeg uses the name of the files that you specify to determine what format your initial file is and what format you want it converted to. If the format that you want to use is not available then email ITServiceCentre@unsw.edu.au indicating that you would like an additional FFMpeg format installed on Katana.

Note: As the files that you will be working with are likely to be too large to fit in your login directory you should save them to global scratch in a subdirectory of /srv/scratch/$USER such as /srv/scratch/z1234567/video_files. As this directory is NOT backed up you should sign up to the UNSW Long Term Research Data Store and then follow the instructions to create an archive version of your original video files.

Converting a Video File Interactively

If you want to use Katana to convert a single MPEG file called input.mp4 to an AVI file output.avi for example then you can start an interactive job on Katana, load the FFmpeg module and then convert the file using the following commands.

qsub -I
cd /srv/scratch/z1234567/video_files
module add ffmpeg/2.1.3
ffmpeg -i input.mp4 output.avi

Some experimentation with parameters has determined that adding the option

-an

to remove sound (assuming that sound isn't required) and

-ar 22050 -b:v 8192k

to adjust the conversion bitrate should give you a reduction of 70% to 80% in file size which still retaining sufficient quality for use so we will use those defaults from now on.

qsub -I
cd /srv/scratch/z1234567/video_files
module add ffmpeg/2.1.3
ffmpeg -i input.mp4 -an -ar 22050 -b:v 8192k  output.avi

If your job will need more than the default job requirements you should check out the resource requirements page to see how to request more memory or CPU time (also known as walltime).

Submitting a Video File for Conversion

Running an interactive job is a good way to make sure that you have the file conversion commands right and everything is working but submitting the job as a batch job means that you don't have to worry about watching it run the conversion.

The following job script requests 4gb of memory and will run for up to 2 hours and will email you when it is finished. For more information about job options have a look at the pages under the HPC jobs section of the site.

#!/bin/bash
 
#PBS -N CONVERT_VIDEO
#PBS -l nodes=1:ppn=1
#PBS -l vmem=4gb
#PBS -l walltime=2:00:00
#PBS -j oe
#PBS -M fred.bloggs@unsw.edu.au
#PBS -m ae
 
cd /srv/scratch/z1234567/video_files
 
module add ffmpeg/2.1.3
 
ffmpeg -i input.mp4 -an -ar 22050 -b:v 8192k  output.avi

If you save this as my_job.pbs you can then submit it by typing

qsub my_job.pbs

Because of how Katana is set up you can submit multiple different jobs at the same time and you will probably not need to wait for one to finish before the next one starts (depending on system load).

Converting Multiple Video Files

Even better than manually submitting 10 conversion jobs you can set up an array job

to automatically submit multiple jobs for you.

For example if you have input files input1.mp4, input2.mp4, ..., input10.mp4 that you want to convert then the following job script will start 10 different jobs each requesting 4gb of memory and will run for up to 2 hours and will email you when it is finished.

#!/bin/bash
 
#PBS -N CONVERT_MULTIPLE_VIDEOS
#PBS -l nodes=1:ppn=1
#PBS -l vmem=4gb
#PBS -l walltime=2:00:00
#PBS -j oe
#PBS -M fred.bloggs@unsw.edu.au
#PBS -m ae
#PBS -t 1-10
 
cd /srv/scratch/z1234567/video_files
 
module add ffmpeg/2.1.3
 
ffmpeg -i input${PBS_ARRAYID}.mp4 -an -ar 22050 -b:v 8192k  output${PBS_ARRAYID}.avi

If you save this as my_multi_job.pbs you can then submit it by typing

qsub my_multi_job.pbs

and your 10 conversion jobs will start.

If your files aren't numbered in way that will work with a standard array job you can still submit multiple conversion jobs at the same time. The simplest way is to set up your job using a variable and then use the BASH command line to submit multiple jobs.

For example if you have movie files input_20150403_7.mp4, input_20150403_27.mp4, input_20150405_11.mp4, input_20150411_35.mp4 then you can still submit all of them for conversion at the same time by creating the following PBS file and calling it multiple.pbs

#!/bin/bash
 
#PBS -N CONVERT_MULTIPLE_VIDEOS
#PBS -l nodes=1:ppn=1
#PBS -l vmem=4gb
#PBS -l walltime=2:00:00
#PBS -j oe
#PBS -M fred.bloggs@unsw.edu.au
#PBS -m ae
 
cd /srv/scratch/z1234567/video_files
 
module add ffmpeg/2.1.3
 
ffmpeg -i $MY_FILES.mp4 -an -ar 22050 -b:v 8192k $MY_FILES.avi

Then you run the following command:

for MY_FILE_LIST in input_20150403_7 input_20150403_27 input_20150405_11 input_20150411_35; 
do export MY_FILES=$MY_FILE_LIST; 
qsub -v MY_FILES multiple.pbs; done;