Mangold INTERACT: Video-Based Behavioral Observation at a Scientific Level

In video-based behavioral observation, the real challenge does not begin in front of the camera - it begins afterward. What unfolds in reality as a continuous flow of actions, reactions, and simultaneous events must be translated into a data structure that enables later analysis, without destroying the temporal relationships that lie at the heart of the research question. Mangold INTERACT was developed to solve exactly this problem in a methodologically sound way.

The software has been used for over 35 years in psychology, education, medicine, occupational science, and ethology - as a 360° research software built from the ground up around the demands of behavioral observation. This article describes the methodological decisions behind its design and their practical consequences for everyday research.

What INTERACT Does

At its core, INTERACT is a software for video-based behavioral coding and behavioral analysis. It accompanies the research process from the development of the coding scheme through the actual coding to the analysis - without requiring data to be exported to another program at any point, as long as the research question concerns temporal relationships in the observed behavior.

In concrete terms: INTERACT enables the time-synchronized playback of multiple audio and video files, parallel coding by multiple observers, calculation of inter-rater reliability, sequential and contingency analyses, and the temporal linking of physiological data - such as EEG signals or electrodermal activity measurements - with video material. The important temporal relationship between data points, which is typically lost when exporting to conventional statistical software, is maintained internally throughout - this is a fundamental methodological principle and a key advantage of INTERACT.

Fields of Application

Using software-based analysis tools in observational studies is generally preferable to paper-and-pencil methods - particularly when research questions concern when, how long, and in what relation to other behaviors an event occurs, whether and which behavioral patterns exist, and how the quality of observation can be assessed (e.g., inter-rater reliability). INTERACT is used in, among others:

• Developmental psychology & pediatrics: Mother-child interactions, early childhood development studies, early intervention
• Work & organizational psychology: Team communication, leadership behavior, meeting analysis, motivational interviewing
• Clinical research & therapy: Behavioral observation in intervention studies, pain research, psychiatric diagnostics
• Ethology & animal behavior: Long-term studies with time sampling, behavioral observation in natural and controlled environments
• Usability & human factors: Human-machine interaction, eye tracking integration, user studies
• Sports science: Movement analysis, technique assessment, tactical behavior
• Educational research: Classroom observation, learning behavior, interaction analysis in school settings

The INTERACT Data Model: Flexible and Precise Time Intervals

Most observation tools work with a continuous timeline: each newly recorded event implicitly marks the end of the previous one. This logic is practical for certain ethological study designs but assumes that behavior occurs continuously and in sequential order - an assumption that many research questions do not support.

Anyone who has tried to code overlapping behavioral sequences of multiple individuals in software that internally insists on a continuous timeline knows the problem: the model forces decisions that cannot be methodologically justified. INTERACT resolves this through a different foundational principle.

Every recorded data point - referred to internally as an event - has an explicit start time and an explicit end time. Each event is temporally independent of all others, can be modified at any time, filtered for analyses, and recombined with other data. When start and end time are identical, this corresponds to a momentary value in the sense of momentary time sampling. INTERACT is therefore method-agnostic and prescribes no particular sampling approach - it adapts to the study, not the other way around.

One practical consequence of this event-based approach: multiple coders can work in parallel on the same video material. One observer codes verbal utterances while a colleague simultaneously codes gaze directions. Both datasets can be analyzed independently and in exact temporal synchrony, without either coding influencing the other. For combined analysis, such datasets can simply be merged into a single file, since INTERACT automatically recognizes temporal relationships and supports any number of temporally parallel and overlapping events.

INTERACT stores timestamps with a precision of 100 nanoseconds. This may seem academic but is practically significant. At 25 frames per second (PAL standard), each frame lasts 40 ms; at 30 frames per second (NTSC standard), it is 33.3333333… ms. Systems that store timestamps as floating-point numbers in seconds can accumulate rounding errors that lead to incorrect results in statistical analyses or behavioral pattern searches - errors that remain invisible in the raw data and may only surface when scientific validity is at stake.

From Event to Insight: Iterative Analysis

Frequency and duration of codes are a sensible starting point - but the interesting questions in observational studies usually begin only afterward. What else is happening while two particular behaviors co-occur? How long does it take for a complex action to elicit a response? Does this latency change over the course of a therapy or a learning process? Such questions concern the temporal structure of behavior, not merely its raw frequency.

INTERACT addresses this through an iterative analysis principle: the results of one analysis step - for example, identified co-occurrence intervals - are written back as new, time-stamped events into the data. These secondary events are immediately available for further analysis. One can then ask what happens within the identified contingency intervals, use that result as the basis for yet another analysis, and so on. The temporal relationship is preserved at every step.

A concrete example: “How long does it take, after the simultaneous occurrence of eye contact and a vocal utterance by person A, for person B to initiate contact - and does this latency change over the course of therapy?” This question requires that co-occurrences be identified, translated into time intervals, and used as context for a subsequent analysis. This is difficult to formulate with case-oriented statistical programs, because they typically do not preserve the temporal relationships between data points upon export.

“INTERACT has revolutionized our approach to data analysis, ensuring precision and real-time adaptability.” Heather Henderson, Prof. Ph.D., University of Waterloo, Canada

Coding Schemes: Structure as a Prerequisite for Reliability

A well-constructed coding scheme is not a formality - it determines which research questions can ultimately be answered with the collected data. INTERACT supports the development and application of such schemes, from simple nominal classes to hierarchically nested or recursively linked coding systems. The basic principle: codes are operational behavioral units that describe exactly one observable characteristic and are mutually exclusive (e.g., “eyes open,” “eyes closed” / or “pointing,” “grasping,” “writing”). Classes group thematically related codes together (e.g., “gesture,” “facial expression,” “mode of expression,” “emotion”).

The concept of lexical chains is particularly useful for efficient and effective coding: codes from multiple classes are combined into sequences during input, without requiring every possible combination to be defined in advance. This substantially reduces the cognitive load on coders - and with it the error rate. The Facial Action Coding System (FACS), for example, which encompasses a very large number of combinations of muscle, movement direction, and intensity, can be fully represented in INTERACT using three recursive classes, without requiring the coder to memorize hundreds of individual codes. When coding lexically, the coder simply enters the desired values - muscle, movement, intensity - in sequence, enabling fast and accurate coding of complex expressions.

That codes should be described using concrete behavioral terms - rather than numeric abbreviations that require an additional translation step - may seem obvious, yet is frequently neglected in practice. INTERACT makes it straightforward to define codes with descriptive labels (e.g., “Friendly,” “raises hand,” “open question” rather than “FR,” “RH,” “2”). This greatly facilitates learning (rater training) and practical application. Any descriptive code can be assigned a custom abbreviation or numeric value, so that codes are automatically converted to the desired shorthand upon data export if needed.

Integration into the Research Workflow

Behavioral observation data rarely stand alone - the coding results often need to be related to physiological measurements, eye tracking data, EEG signals, or electrodermal activity. In INTERACT, such external data sources can be temporally linked to the video material, so that behavioral sequences and biosignals can be analyzed on the same time axis.

For recurring analysis processes, the workflow editor allows complex analysis pipelines to be configured and automated. Individual steps - data selection, transformation, analysis, export - are defined as nodes and can be extended with Python scripts. The result is a reproducible, documented process that is automatically available to all project members. Particularly in multi-site studies and in preparing for replicability, this is more than a convenience - it is a methodological necessity.

Choosing the Right Software

Which observation software is appropriate for a given project depends on the data structure the study design requires. Software built on the continuous timeline paradigm works well for designs in which gapless, non-overlapping state sequences describe behavior. Where parallel coding, overlapping codes, non-chronological capture, or the reintegration of analysis results into the data are required, such models encounter structural limitations.

The most useful question before starting a project is therefore not “Which software is most widely used?” but rather: which data structure and capture approach best represents the research question - and which analyses should be possible without data export and loss of information? Correcting this decision after the fact is, in practice, costly.

“Our experience with INTERACT was excellent. The software is friendly, intuitive to use, gives precise results and supports comprehensive behavioral pattern analysis.” Ruth Feldman, Prof. Ph.D., Best Female Scientist 2022, Expertscape World Expert, Herzlia Israel

Use in Research and Teaching

INTERACT is documented in a number of standard references on behavioral observation and group interaction research, including the Cambridge Handbook of Group Interaction Analysis (Glüer & Boos, 2018). The software appears as a data collection instrument in peer-reviewed publications from developmental psychology, organizational psychology, clinical research, and ethology - evidence that the data model genuinely works across disciplines, not just in theory.

Mangold International provides documentation, video tutorials, and technical support.

INTERACT: One Software for Your Entire Observational Research Workflow

From audio/video-based content-coding and transcription to analysis - INTERACT has you covered.

Explore INTERACT

FAQ: Mangold INTERACT - Behavioral Observation and Video Analysis

Frequently asked questions about the software, its data model, and its application in scientific research

What is Mangold INTERACT and what is it used for?

Mangold INTERACT is a software for video-based behavioral coding and behavioral analysis, developed specifically for scientific observational studies. It is used to systematically code behavior in audio and video recordings, assign precise temporal positions to each event, and statistically analyze the results. Unlike general video tools or statistical programs, INTERACT maintains the temporal relationship between data points across all analysis steps - a prerequisite for sequential, contingency, and co-occurrence analyses. The software has been in use for over 35 years in psychology, education, medicine, ethology, and occupational science.

In which research disciplines is INTERACT used?

INTERACT is used across disciplines wherever the temporal occurrence, duration, and sequence of behaviors are the subject of analysis. Documented fields of application include developmental psychology and pediatrics (mother-child interactions, early intervention), work and organizational psychology (team communication, meeting analysis, motivational interviewing), clinical research and therapy (intervention studies, pain research, psychiatric diagnostics), ethology and animal behavior, usability and human factors research, sports science, and educational research. The software is documented in the Cambridge Handbook of Group Interaction Analysis (Glüer & Boos, 2018) as a tool for group interaction research.

How does INTERACT differ from other observation tools?

Most observation tools work with a continuous timeline in which each newly recorded event implicitly marks the end of the previous one. INTERACT uses a different foundational principle: every data point - referred to internally as an event - has an explicit start time and an explicit end time and is temporally independent of all other events. This allows overlapping codes, non-chronological capture, and parallel coding by multiple observers on the same material. In addition, analysis results can be written back as new, time-stamped events into the dataset and used in subsequent analyses - without losing the temporal context that is typically lost when exporting to conventional statistical software.

What sampling methods does INTERACT support?

INTERACT supports all common sampling methods in behavioral observation: event sampling, time sampling, ad libitum sampling, scan sampling, and focal sampling. Because the data model does not prescribe any particular method, these approaches can be combined within the same project. When the start and end time of an event are identical, this automatically corresponds to a momentary value in the sense of momentary time sampling. The choice of method rests entirely with the research team - the software does not enforce any particular sampling paradigm.

Can INTERACT process multiple video and audio files simultaneously?

Yes. INTERACT enables the simultaneous coding of multiple audio and video streams with guaranteed temporal coherence across all streams. This means that camera angles, microphones, and other recording sources can be played back and coded synchronously without losing the temporal relationship between streams. This feature is particularly relevant for studies in which multiple individuals are observed simultaneously, or where behavioral data are to be combined with physiological measurements (EEG, heart rate, electrodermal activity) or eye tracking signals.

How precise is INTERACT?

INTERACT stores timestamps with a resolution of 100 nanoseconds. This is far more than necessary for most observational studies, but becomes relevant for frame-accurate analysis of video material with variable frame rates and for the study of very fast behavioral sequences - such as facial expression analysis, speaker turn changes, or neonatal responses. At 25 fps (PAL), each frame lasts 40 ms; at 30 fps (NTSC), it is 33.3333333… ms. Systems that store timestamps as floating-point numbers in seconds can accumulate rounding errors over long observation periods that remain invisible in the raw data and only become apparent during later statistical analysis.

What is a lexical chain in INTERACT and what is it used for?

Lexical chains are a concept in INTERACT that makes coding more efficient and error-resistant. Instead of pre-defining every possible combination of multiple codes as a separate entry, codes from different classes are dynamically combined into sequences during input. This substantially reduces the cognitive load on coders - and with it the error rate. For example, the Facial Action Coding System (FACS), which encompasses hundreds of combinations of action unit, movement direction, and intensity, can be fully represented in INTERACT using three recursive classes - without requiring coders to memorize hundreds of individual codes.

How does iterative secondary analysis work in INTERACT?

Iterative secondary analysis is a central analytical principle in INTERACT: the results of one analysis step - such as identified co-occurrence or contingency intervals - are written back as new, time-stamped events into the dataset. These secondary events are immediately available for further analysis steps. One can then ask what happens within the identified intervals, use that result as input for a subsequent step, and so on - without exporting data or losing temporal context. This enables the investigation of research questions that are difficult to formulate with case-oriented statistical programs such as SPSS or R.

What analysis functions are available in INTERACT?

INTERACT offers, among others, the following analysis functions: co-occurrence analysis (identification of simultaneously occurring codes and temporal distances), sequential analysis including lag-sequential analysis, contingency analysis with flexibly definable time windows, state-space grids for visualizing dynamic behavioral trajectories (following Hollenstein), inter-rater reliability calculations based on actual code overlaps, and various visualization formats (timelines, chord diagrams, frequency charts). In addition, custom Python scripts can be integrated into the built-in workflow editor - for example, for specialized calculations such as deriving heart rate variability (HRV) from raw data.

Can INTERACT be combined with physiological measurement systems or eye tracking?

Yes. INTERACT is designed to temporally link external data sources - including physiological measurements such as EEG, heart rate, and electrodermal activity - with video material. Behavioral sequences and biosignals then share the same time axis and can be analyzed together. This integration is particularly relevant for multimodal research questions in which behavioral data and biological responses are to be examined in temporal relation to one another.

Does INTERACT support parallel coding by multiple observers?

Yes. Because INTERACT’s data model is based on explicit, mutually independent time intervals, multiple coders can work in parallel on the same video material and subsequently merge their datasets without conflict. One observer codes verbal utterances while a colleague simultaneously codes gaze directions - both datasets can be precisely synchronized and jointly analyzed. INTERACT also calculates inter-rater reliability based on actual code overlaps, independent of interval boundaries.

Can INTERACT be combined with other programs?

Yes. INTERACT exports data in formats that can be processed further by R, SPSS, Python, and Excel. INTERACT can also import data from Excel and CSV files.

Can workflows in INTERACT be automated?

For recurring analysis processes, the integrated workflow editor allows multi-step pipelines to be configured and automated; individual steps can be extended with Python scripts. This enables reproducible, documented analysis processes - a requirement that is increasingly important in multi-site studies and in preparing for replicability.

For what type of study design is INTERACT particularly suited?

INTERACT is particularly well suited for study designs in which the temporal structure of behavior plays a central role: that is, where the research interest is not only in the frequency of a behavior, but in when it occurs, how long it lasts, with which other behaviors it co-occurs, and what sequences or contingencies emerge. Study designs with overlapping codes, parallel coding by multiple observers, multimodal data sources, or iterative analysis workflows benefit especially from the software’s data model.

Can INTERACT be used for mixed methods research?

Yes - INTERACT lends itself well to mixed-methods designs because it brings together qualitative and quantitative data on the same time axis. On the qualitative side, the software allows the capture of transcripts, open descriptions, and contextualized observational notes directly attached to each coded time interval. On the quantitative side, it delivers exact timestamps, frequencies, durations, sequential and contingency statistics, and inter-rater reliability values.

The methodological value lies in the fact that both data layers share the same temporal reference: a qualitative observation - such as a transcript passage or an open annotation - can be directly linked to simultaneously recorded quantitative codes and analyzed jointly. This makes it possible, for example, to contextually interpret numerically identified anomalies (an unusually long latency, a repeated co-occurrence) within the same tool, without switching between programs.

In addition, physiological data can be integrated as a further quantitative layer - so that behavioral observation, verbal data, and biological responses can be analyzed within a single unified time frame. For researchers who want to combine qualitative depth with quantitative precision, this integration is a practical advantage over using separate tools.

Is INTERACT better than other programs?

That depends on what “better” means. INTERACT is in any case the most flexible software on the market, because it allows all recorded data to be edited at any time while the integrity and temporal coherence of the data are always guaranteed. Data from external sources can also be easily imported and exported via copy-and-paste (e.g., transcripts or coding data). Above all, INTERACT supports information mining, enabling answers to questions that remain hidden in purely descriptive analyses. Since INTERACT has been used in research worldwide for over 35 years, it can be regarded as a scientifically grounded and reliable tool.

How is INTERACT documented in the scientific literature?

INTERACT is documented in a number of standard references on behavioral observation and group interaction research, including the Cambridge Handbook of Group Interaction Analysis (Glüer & Boos, 2018). The software also appears as a data collection instrument in peer-reviewed publications from developmental psychology, organizational psychology, clinical research, and ethology. A selection of publications can be found on this website under [Discover more → Research → Publications].

Where can I find more information, tutorials, and trial versions of INTERACT?

Product overview, package information, video tutorials, and access to the trial version are available on the Mangold INTERACT product page .

INTERACT: One Software for Your Entire Observational Research Workflow

From audio/video-based content-coding and transcription to analysis - INTERACT has you covered.

Explore INTERACT