EasyDIAg: A tool for easy determination of interrater agreement

Research output: Contribution to journalJournal articlesResearchpeer-review

Authors

Research units

Details

Reliable measurements are fundamental for the empirical sciences. In observational research, measurements often consist of observers categorizing behavior into nominal-scaled units. Since the categorization is the outcome of a complex judgment process, it is important to evaluate the extent to which these judgments are reproducible, by having multiple observers independently rate the same behavior. A challenge in determining interrater agreement for timed-event sequential data is to develop clear objective criteria to determine whether two raters' judgments relate to the same event (the linking problem). Furthermore, many studies presently report only raw agreement indices, without considering the degree to which agreement can occur by chance alone. Here, we present a novel, free, and open-source toolbox (EasyDIAg) designed to assist researchers with the linking problem, while also providing chance-corrected estimates of interrater agreement. Additional tools are included to facilitate the development of coding schemes and rater training.
Original languageEnglish
JournalBehavior research methods
Volume47
Issue number3
Pages (from-to)837-847
Number of pages11
ISSN1554-351X
DOIs
Publication statusPublished - 09.2015

ID: 1446909

View graph of relations