Evaluation of the Unified Medical Language System as a medical knowledge source

Objective: The authors evaluated the use of the Unified Medical Language System (UMLS) as a medical knowledge source for the representation of medical procedures in the MAOUSSC system. Design: MAOUSSC, a multiaxial coding system, was used for the representation of 1500 procedures from 15 clinical specialties, using UMLS concepts (augmented by full sources for three new vocabularies being added to the UMLS) and relationships whenever possible. Evaluation criteria for the UMLS included (1) completeness of representation of concepts and of inter-concept relationships, (2) consistency in the categorization of both concepts and inter-concept relationships, and (3) usability, including adaptability of the UMLS to a foreign language (French), its suitability to a geographic region with different medical practices than the USA, and issues relative to the annual update changes in the test vocabularies. Results: During the MAOUSSC trial, the number of missing concepts or relationships identified in the augmented UMLS sources was deemed to be inconsequential relative to overall project goals. "Missing" UMLS inter-concept relationships were identified, although they were small. in number. Some inconsistencies in the UMLS were noted, especially in the area of hierarchic relationships. Conclusion: After UMLS was used for five years as a knowledge source for representing 1500 complex medical procedures in MAOUSSC, its value is considered significant. Future editions of the UMLS are expected to improve representation of inter-concept relationships and global consistency.