EasyDIAg: A tool for easy determination of interrater agreement

Publikationen: Beitrag in FachzeitschriftZeitschriftenaufsätzeForschungBegutachtung

AutorInnen

Forschungseinrichtungen

Details

Reliable measurements are fundamental for the empirical sciences. In observational research, measurements often consist of observers categorizing behavior into nominal-scaled units. Since the categorization is the outcome of a complex judgment process, it is important to evaluate the extent to which these judgments are reproducible, by having multiple observers independently rate the same behavior. A challenge in determining interrater agreement for timed-event sequential data is to develop clear objective criteria to determine whether two raters' judgments relate to the same event (the linking problem). Furthermore, many studies presently report only raw agreement indices, without considering the degree to which agreement can occur by chance alone. Here, we present a novel, free, and open-source toolbox (EasyDIAg) designed to assist researchers with the linking problem, while also providing chance-corrected estimates of interrater agreement. Additional tools are included to facilitate the development of coding schemes and rater training.
OriginalspracheEnglisch
ZeitschriftBehavior research methods
Jahrgang47
Heft3
Seiten (von - bis)837-847
Seitenumfang11
ISSN1554-351X
DOIs
PublikationsstatusVeröffentlicht - 09.2015

ID: 1446909

DOI

Beziehungsdiagramm anzeigen