We examined a process for automating the classification of articles in MEDLINE aimed at minimising manual effort without sacrificing accuracy. From 22,808 articles pertaining to 19 antidepressants, 1000 were randomly selected and manually labelled according to article type (including, randomised controlled trials, editorials, etc.). We applied a machine learning approach termed ‘active learning’, where the learner (machine) selects the order in which the user (human) labels examples. Via simulation, we determined the number of articles a user needed to label to produce a classifier with at least 95% recall and 90% precision in three scenarios related to evidence synthesis. We found that the active learning process reduced the number of training instances required by 70%, 19%, and 14% in the three scenarios. The results show that the active learning method may be used in some scenarios to produce accurate classifiers that meet the needs of evidence synthesis tasks and reduce manual effort.
IOS Press, Inc.
6751 Tepper Drive
Clifton, VA 20124
Tel.: +1 703 830 6300
Fax: +1 703 830 2300 firstname.lastname@example.org
(Corporate matters and books only) IOS Press c/o Accucoms US, Inc.
For North America Sales and Customer Service
West Point Commons
Lansdale PA 19446
Tel.: +1 866 855 8967
Fax: +1 215 660 5042 email@example.com