Skip to main content

Advertisement

Table 9 Metrics to evaluate and compare ontology and traditional data model approaches

From: Ontological specification of quality of chronic disease data in EHRs to support decision analytics: a realist review

Criteria Metrics for ontology evaluation References Metrics for data model evaluation References
Flexibility Easily adapted to multiple views in terms of parameters such as modularity, partitioning, context-boundedness Gangemi et al. 2006 Ability to deal with changes in business and/or regulatory rules/context? Moody and Shanks 2003
  Ability to accept input of new data from various research groups and disciplines Maiga and Williams 2008 Ability to add new data elements and relationships if project scope or regulatory rules (e.g. patient identification) change Kahn et al. 2012
  Easily re-define the extraction procedure logics and adapt it to user needs Pannarale et al. 2012 Flexibility of data models include "extensibility", "scalability", and "adaptability" as defined operationally below. Kahn et al. 2012
  Easily manage the changes of the database schema or the ontology Pannarale et al. 2012   
Reusability Ability to integrate data so that it is useful to different users and disciplines Maiga and Williams 2008   
  Ability to match user requirements across different disciplines Pinto 2004   
Scalability    Can data model be sized in smaller or larger data sets? Kahn et al. 2012
Completeness    Does the data model contain all user requirements? Moody and Shanks 2003
    Can the data model store and retrieve data to meet investigator needs? Kahn et al. 2012
Correctness    Does the data model conform to the rules of the data modelling techniques? Moody and Shanks 2003
    Does the model conform to good data modelling practices such as limited data storage redundancy? Kahn et al. 2012
Extensibility    Can the data model expand data elements, data types and include new data domains? Kahn et al. 2012
Adaptability    Can the data model represent a broad data domain? Kahn et al. 2012
Cohesiveness A measure of the separation of responsibilities and independence of components of ontologies Yao et al. 2005   
Precision A measure of the amount of knowledge correctly identified in the ontology w.r.t. the whole domain knowledge available Brewster et al. 2004   
Recall A measure of the amount of knowledge correctly identified with respect to all the knowledge that it should identify Brewster et al. 2004   
Fitness for purpose Can the ontology define and assess if routinely collected EHR data is fit for purpose? Wand and Wang, 1996; Can the data model store and retrieve data to meet investigator needs correctly? (Note: Kahn defined this as completeness of the data model) Kahn et al. 2012
   Liaw et al. 2011