Thumbnail
Access Restriction
Open

Author Qi, Zhen ♦ Tu, Li-ping ♦ Luo, Zhi-yu ♦ Hu, Xiao-juan ♦ Zeng, Ling-zhi ♦ Jiao, Wen ♦ Ma, Xu-xiang ♦ Jing, Cong-cong ♦ Wang, Wei-jian ♦ Zhang, Zhi-feng ♦ Xu, Jia-tuo
Editor Micke, Oliver
Source Hindawi
Content type Text
Publisher Hindawi
File Format PDF
Copyright Year ©2018
Language English
Abstract This study aims at introducing a method for individual agreement evaluation to identify the discordant raters from the experts’ group. We exclude those experts and decide the best experts selection method, so as to improve the reliability of the constructed tongue image database based on experts’ opinions. Fifty experienced experts from the TCM diagnostic field all over China were invited to give ratings for 300 randomly selected tongue images. Gwet’s AC1 (first-order agreement coefficient) was used to calculate the interrater and intrarater agreement. The optimization of the interrater agreement and the disagreement score were put forward to evaluate the external consistency for individual expert. The proposed method could successfully optimize the interrater agreement. By comparing three experts’ selection methods, the interrater agreement was, respectively, increased from 0.53 [0.32-0.75] for original one to 0.64 [0.39-0.80] using method A (inclusion of experts whose intrarater agreement>0.6), 0.69 [0.63-0.81] using method B (inclusion of experts whose disagreement score=“0”), and 0.76 [0.67-0.83] using method C (inclusion of experts whose intrarater agreement>0.6& disagreement score=“0”). In this study, we provide an estimate of external consistency for individual expert, and the comprehensive consideration of both the internal consistency and the external consistency for each expert would be superior to either one in the tongue image construction based on expert opinions.
ISSN 1741427X
Learning Resource Type Article
Publisher Date 2018-10-02
Rights License This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
e-ISSN 17414288
Journal Evidence-Based Complementary and Alternative Medicine
Volume Number 2018
Page Count 9


Open content in new tab

   Open content in new tab