If you deal with video files on any computer you need to know the what / why / how about codecs!

A "Codec" in simple words:

• A "codec" is an artificial word, composed from the words "Coder" and "Decoder" = "codec".
• The main purpose of a codec is to compress digital media files (audio/video).
• If you would not compress those files they would get extremely big (e.g. a typical video file consists of 30 images per second, each with a size of 720x480 pixels. Storing this data in uncompressed way would result in approximately 36 Gigabytes data per hour! Adding uncompressed audio would easily double this amount!
• A codec consists of typically several software programs of which you do not need to take care of. They are mostly installed through an installer program that comes with that codec. Once installed, the computer’s operating system's media functionalities can make use of that codec. Thus, you can compress media files into that specific format and you can decompress them again to be able to watch the videos again.
• Lots of companies and research institutes continuously work on improving existing audio/video compression algorithms. All those compression formats are based on non-lossless algorithms (data is being deleted from the original data stream to make it smaller). That’s why the main goal for those developers is to find a good trade-off between “image and audio quality” versus “file size”. “Remain as much quality as possible while decreasing file size as much as possible”. … The unsolved question is: What means “good quality” and what means “small file size” …
• Widely spread and well known codecs are “MPEG1”, “MPEG2”, “MPEG4”, “DivX”, “WMV”. (Most common misunderstanding: Be careful, “AVI” is NOT a codec but a container format, which can contain lots of differently coded videos. Thus, there is no “AVI codec”!)
• If there is no more room for improving existing codecs or if there is a business related reason behind it, the companies release new algorithms giving them new names such as “H264”. This is why there are so many codecs available.
• Some people have packed several such codecs together into so called “codec packs”. This makes installation and updating those codecs easier and is very nice if you know what you are doing. If not, don’t install every codec you see in the internet, because they interfere and might lead to a system that cannot play any audio/video file any more …

What codecs do – in simple words:

• The “Coder” part tries to group “similar areas of a video image” and to store this simplified information. The algorithm e.g. tries to reduce slightly different colored areas within each single video image into an evenly colored area (single image points that are close to each other and have similar colors will become a “flat” region of one color)
• The “Coder” also removes information from the audio tracks which is typically too high or too low for a human’s ear.
• Similar information from one image to the following image does not need to be stored twice (e.g. a white wall as part of the video image will always stay in that position and will always look the same). Thus a video files does not store e.g. 30 full images per second but only the differential information between preceding and succeeding images. That’s one major reason why you cannot simply “play backwards” or “move single video images fore and back”!
• … it is really not as simple as that but shows the idea.

That’s why:

• Compressed videos typically have “artifacts”. I.e. smaller or bigger rectangles of colored blocks spread over each single video image.
• Compressed videos typically create a blurred looking image if the video content moves quickly (e.g. a person making quick arm or head movements)
• Compressed videos typically do not allow for frame accurate navigation (some codecs allow, others not)
• The sound might become “noisy”.
• … You won’t see or hear all of this when you simply watch a video / DVD.
• … But you will recognize that if you do “frame by frame” video based analysis.

Summary:

Each coder has its very specific methodology of reducing image contents without losing too much quality.

Thus, it is obvious that a video file having been compressed using a specific Coder needs the accompanying Decoder to unpack all this information again. If you have coded your videos in e.g. DivX format you need a DivX decoder to be able to watch the video. It’s as simple as that!

As mentioned before, this codec issue can cause quite a lot of trouble. This problem has two sides, which you will need to identify in order to get it solved:

Identify the video codec used for compression

If you do not know how and with which Codec the video was created, you will need to take a look 'inside'.

If you are using VISTA, the default AVI properties dialog might tell you all you need to know.

In all other cases the following method is a quite reliable way to investigate:

• Download GSpot http://www.videoanalysis.net/GSpot.zip and start the application.
• Open the video file you are investigating.
• Take a look at the Codec information in the upper right corner, where you find the name of the Codec that was used to store the video. Also the bigger description field at the left side might contain interesting information.
• Make a screenshot of this window to send to us if need be.

Tip: Should you use a video format that is not recognized correctly by GSpot, use the program MediaInfo to find out about the codec used for compression.

Now you know what kind of video it is. If GSpot tells you Codec NOT installed, you will need to install the proper Codec and try again.

If GSpot tells you the Codecs are installed, but things still do not work correctly, your system does use a Codec that is recognized as a proper Codec for that video, but the Codec does not provide the WMP with the proper time information, so INTERACT can not rely on the given information. In this case, you will need to find out what Codec is actually used on your system.

Tips and handy tools

DVD's & VOB's - If you burn videos on a DVD, do NOT burn them as a Video DVD but create a Data DVD! Video DVD's are MPEG2 files packed in a specific VOB format. Sometimes obviously, with the suffix VOB and sometimes hidden, showing just the *.mpg suffix. This will result in single video files that are not divided in logical parts, causing multiple sessions to be joined in one file or, worse, single sessions cut into multiple files. The corresponding DVD typical index files are not recognized by INTERACT.

MPEG2 - A widely spread rather old fashioned coding algorithm used for DVD videos and many hand held recording devices. MPEG2 files need a special codec for accurate controllability. The best codec for this is the Elecard codec. All other codecs that do play mpeg2 videos correctly are not able to reposition the video frame accurately.

Elecard - If you do have VOB or other MPEG2 or MPEG4 videos and you do not know how the videos where created, the Elecard Codec often does a very good job at plying them. A free trial can be downloaded at www.elecard.com.

MangoldVideoConverter Pro - Easy to use free video converter to convert *.H264, *.264, *.MPEG, *.MPG and *.MOD video files into XviD or x264vfw compressed avi files. Free download here.

GraphEdit - Is a utility which is part of the Microsoft DirectShow SDK and it shows you exactly which codec is used for playing the video on your computer. Sometime you might have the correct codec installed, but for some reason Windows uses another codec, this at least shows you what your computer does use.

GSpot - This is a free utility which identifies the codec required to play an AVI file and supplies other information as well, like whether there is a hidden VOB layer in your mpeg files.

VideoCodecTools - You can download the tools GSpot, GraphEdit and DSFM from this link www.videoanalysis.net/VideoCodecTools.zip

VirtualDub - Allows you to convert, edit and manipulate your digitized video in virtually every way you want. You can adjust the frame rate, join avi files or cut them in pieces, get rid of the interlace and many, many more (virtualdub.sourceforge.net).

xMedia recode - This is a freeware conversion tool which enables you to reconvert your video into multiple formats (www.xmedia-recode.de).

SUPER - This is a powerfull freeware conversion tool based on ffmpeg which enables you to reconvert your video into multiple formats (www.erightsoft.com/SUPER.html).

Windows 7 comes with a bunch of default codecs for playing all kinds of video file types, which makes live a lot easier for the average pc user, but not for us!

The problem is that those default codec offer a good playback quality, but unreliable time information, which makes it impossible to control your video frame accurate.

The biggest problem is though, that simply installing a preferred codec does not override the default Windows 7 codecs anymore, as it used to under XP and VISTA!

With Windows 7 you need to tweak your configuration using a great freeware tool called Win7DSFilterTweaker in order to get things 'back to normal' again. Please find below the tweaks needed for XVID/DivX and x264vfw (H.264) videos.

Note: Make sure INTERACT is closed before you perform any of those changes!

• Apply those settings and close the application.
• Start INTERACT and open the video file.
• Check the Video Information dialog to see if the correct Elecard video codec is now used.

NOTE: It is currently not possbile to accurately sync multiple mpeg videos. The codec just is not able to handle that. This is NOT an INTERACT issue!

Should you need to play multiple videos at once while coding, you will need to convert them to XviD as is adviced generally for all mpeg2 videos, simply because it saves you around 60-70% disc space, no commercial codec is required and it is frame accurate whitout additial tweaks. You can use our free Mangold VideoConverterPro to convert your mpeg files.

Frame acurate positioning and synchronizing of mpeg 2 files

Should switching from play to frame by frame movements still result in unexpected jumps (because the video first returns to the last key frame before) or synchronizing multiple mpeg2 videos is still not working, you will need to make some registry adjustments to make things work. We will need to activate the so called 'indexing' option to get get frame-accurate positioning:

• Please close all video applications first.
• Start the 'regedit' utility and open the following branch: HKEY_CURRENT_USER\Software\Elecard\Elecard MPEG Demultiplexer\
• In this branch please delete all folders except the 'Default' folder.
• Double click on the 'Indexing mode' key within this 'Default' folder
• Set the parameter value to '3'.
• Confirm your changes by clicking OK and close the editor.
• Start INTERACT again and check if the problem is solved.

There are numerous ways to digitize your video material. This topic offers you some basic information and general advice. If you do not have many digitized videos, yet, it might prevent you from making a mistake.

Do NOT use video DVD's - One of the biggest problems is caused by this. If you have digitized your video and/or want to save them on a DVD, do NOT create a "Video DVD" but create a "Data DVD" and copy the video files on to that!

The software creating a video DVD will create *.VOB files out of your digitized files and in doing so probably will cut your material into pieces, mixing up your session structure while doing so.

Example: If you have 3 half hour sessions saved in 3 separate video files, burning them onto a video DVD will most often result in multiple *.VOB files in which for instance the last part of session 2 could be within the same VOB file as the beginning of session 3. The contents could only be properly read by a DVD player because in uses the index file that is created as well.

The Windows Media Player and thus INTERACT cannot make use of this index file. INTERACT could play the VOB files one by one, though.

Digitize all recordings with an external encoder - If you do have the time, it is advisable to use an external box like the Pinnacle Movie box to transfer your videos to your hard disk - no matter if you use a miniDV cam or a modern hard/flash disc based camera! Mainly because most disk based cameras record in some MPEG2 variety which results in unnecessary big files. On top of that, most vendors change their compression routines from model to model, leaving you with a mixture of different types of video files. If you use a converter box to save your files, you can select a codec of your choice and digitize all videos of one project into the same format, independent of the camera used for recording.

Test different codecs before digitizing all material - To spare yourself unnecessary frustrations, we advise to make some tests to determine the optimal quality/performance/size combination and test a short sequence in INTERACT as well, checking the control functionality. We know that there are Codecs available that promise high quality pictures at a very small size, but those Codecs usually work with dropped frames to catch up with reality and they are simulating a constant frames per second value, which is never 100% accurate and therefore not suited for frame accurate analysis.

Digitize with the lowest possible frame rate - In most cases you do not need the full 30 (or 25) frames per second with which the video was taped. Reducing the frame rate to 15 fps will reduce the file size noticeably and (in case of software based compression) might even prevent the dropped frame issue mentioned before.

Use a hardware encoder to digitize your videos -This is much faster than with software and also a lot safer, because performance problems on the computer could result in dropped frames, which is not something you see with the naked eye, but loosing 2 frames per second will result in a time difference of 40 seconds for a 10 minutes video sequence digitized with 30 frames per second.

Check the time information - As mentioned in "Why ar codecs so important?" as well, there are a whole lot of video codecs available out there and not all of them are able to provide INTERACT with correct time information. Before you start coding (and best before you start digitizing all material), you should check the following time information:

• The ongoing time code in the INTERACT control panel shows the correct time information.
  TAKE CARE: the format should be HH:MM:SS:FRAMES! If the last 2 digits show you second instead of frames, you need another codec.
• Check the overall duration of the Video. Compare the original length of the recording with the time information of the Windows Media Player, GSpot as well as in INTERACT. If they differ too much, you will also need another Coded OR the problem was cause by dropped frames during the digitizing process!

Software video conversion - If you find yourself in possession of video files that are either uncompressed or use a wrong or uncontrollable video format, and you do not have the possibility to use the previously discussed external conversion method, you can use software for converting your video files. One of many possible tools for that would be Xmedia-recode, which is freeware (www.xmedia-recode.de). BEWARE: Make sure your system has sufficient recourses to complete the conversion without the need of dropping frames!

Check Codec information - If you suffer from a Codec related problem, you could try to find out a bit more about the cause and maybe even the solution, using some third party utilities:

• Use MediaInfo http://mediainfo.sourceforge.net/en/Download to take a look 'inside' the video file. This program will tell you, among other things, what Codec was used, if a compatible codec is installed, how many frames per second, the length of the video, etc.
• Use GraphEdit or MONOGRAM GraphStudio to visualize what Codec is actually used to play that video on the current computer.

Further details can be found in the topic "Identify Codecs".

The file type (or container) is identified by the file extension and does not necessarily reveal any information about the compression routine used inside, as you can see in the table below.

What codec suits you best, depends on your personal demands, which are based on the quality needed, the amount of expected recording time, the available disc/storage space, the recording equipment, etc.

Popular codecs

*) commercial codecs

At this point in time the Xvid compression routine seems to be the best choice. Additionally the x264vfw compression is very good. Also QuickTime movies (*.mov) are working fine. All can be used under Window 7 as well.

Important: Many hard disc handy cams store their videos in some kind of MPEG2 format (DVD quality), which is in fact a rather old fashioned format with a medium compression rate. Please read the topic Digitize video material (pg. 6) for tips about this procedure.

Under Windows XP and Windows Vista, installing the appropriate codec was usually sufficient to make things works. With Windows 7 this is no longer true!

Please read the topic Change default codecs under Windows 7 for further details.

Note: Not all compression routines allow you to control your video frame accurately. Each format needs to be tested excessively first.

Popular file types

Below you will find a short description of the most often used file formats:

AVI: This file format can contain a compressed or un-compressed video. You cannot recognize from the 'outside' what exactly is stored in the file package. This is a typical "Frame format" (also called 'container'), in which every manufacturer / any video software allows many different setting and use varied interpretations. What kind of decoder you will need depends on the format/software used to create the AVI file. We advise you to use either the XviD or X264vfw codec for compression.

DivX and XviD: Two almost identical implementations of the MPEG4-video-codec, which is a further development of the mpeg2 standard. This codec reduces the file size even further while increasing the quality at the same time. To view such files you need a special decoder on your computer (even the onboard Windows 7 codec does a poor job at frame accurate controlling). XviD is an opensource project, DivX is its commercial counterpart, you can find this compression used in avi files as well as files with either an *.xvid or *.divx extension. XVID is, at this point in time, one of the best video codecs to use for accurate frame by frame controlling, also syncing multiple videos works fine and it is free (http://www.koepi.info/xvid.html).

MOV: Quicktime format, originally developed on the macintosh platform. It can contain multiple videao streams and it offers various compression routines like MJPEG or H.264 for high Defintion videos. You need the Apple QuickTime Player to play and control those video files. You can download the Apple QuickTime Player for free from the Apple homepage. INTERACT will use this player instead of the WMP for playing back all *.mov and *.mp4 files. Take care Windows 7 users: The WMP can now handle QuickTime movies, INTERACT relies on the better controllability of the QuickTime player for those formats, so

Tip: QuickTime Pro can be used to create splitscreen videos and can also export divx files, that can be renamed to avi.

MPEG1: Oldfashioned compressed video format. The frame size is 1/4 of the original video. 60 minutes of video will fit on a regular CD (640 MB). Every windows pc (windows 95 or up) is able to play MPEG1 video files.

MPEG2: Widely spread compressed video format. The frame size is usually the same as from the original video. About 90 minutes of film will fit on a DVD (4,6 GB), because this format was originally developed to convert cinema movies to a DVD. You need an adequate decoder to view such video files. Many codecs are able to play MPEG2 files, but most of them give out poor time information, which makes it impossible for INTERACT to properly control such a file. One of the best mpeg2 codecs we know of is provided by Elecard (an older version of this codec is included in this free codec-pack-all-in1). It is highly recommendet to convert those videos into XviD avis for better controllability and to safe 60-70% hard disc space. You can use our free MangoldConverter Pro to do so.

VOB: Compressed video format. As a matter of fact, this is a "packed" MPEG2 video file, spilt into multiple files (chapter), because this in the international commonly used file format on DVDs. You need an appropriate decoder like the Elecard mentioned under MPEG2 to view and control such video files.

MOD: An mpeg programm stream format used by JVC, Panasonic and Canon hard drive comcorders for regular videos (for High Definition videos JVC uses TOD). In general one could say those MOD files are MPEG2 video files with AC3 Audio compression. INTERACT is able to control those videos in combination with the Elecard codec, but conversion is recommended - as it is for regular MPEG2 files. Further details about this format can be found in Wikipedia.

MTS/M2TS: MPEG transport stream clip format for High Definition videos. Often used for live tv recordings, but also used by some hand held high definition recording devices. In most cases it would be best to convert the videos to a reliable controllable format like XviD using a third party converter (e.g. www.aunsoft.com/mts-converter, which offers gut quality settings). The original format can usually be used in INTERACT as they are using the CoreAVC codec. A free trial version can be downloaded at corecodec.com/products/coreavc.

The following table gives you some idea why the file size of digital video files can vary a lot. There are some major factors that dramatically influence files size.

The following table gives some file size estimates based on recording and conversion tests. Please note that the real files size of an individual recorded video is highly dependent on the above factors.