Exploring the impact of automated indexing on completeness of MeSH terms

Abstract

The use of controlled vocabulary to identify relevant articles is a central element of bibliographic database instruction in health sciences. Students learning to search MEDLINE are taught that MeSH yields precise results, and that MeSH indexing increases an article's findability and reliably describes an article's contents.

Indexing for MEDLINE was done completely by human indexers until 2011. Since April 2022, all articles are assigned MeSH via automated indexing (AI). Per the NLM, MeSH assigned by AI are determined based on terms in title, abstract, and terms and indexing of 'neighbouring, related' records, with human review and curation of results "as appropriate".

We explore the following: how well does AI identify key concepts of an article and represent them in MeSH?

Methods: We reviewed a sample of automated records from early 2023 to determine whether their main concepts were adequately represented with MeSH. Working in pairs, our team used spreadsheets to assign key concepts that, per our experience, would be used to find it and similar articles based on title and abstract.

Assigned MeSH were then displayed and analyzed to determine whether they adequately represented the key concepts of each record. We found that 47% of the records screened in our sample had issues in MeSH that would have affected their likelihood of retrieval.