Standard

A Comparative Study of Ontology Matching Systems via Inferential Statistics. / Mohammadi, Majid; Hofman, Wout; Tan, Yao Hua.

In: IEEE Transactions on Knowledge and Data Engineering, 29.05.2018, p. 1-14.

Research output: Contribution to journalArticleScientificpeer-review

Harvard

APA

Vancouver

Author

Mohammadi, Majid ; Hofman, Wout ; Tan, Yao Hua. / A Comparative Study of Ontology Matching Systems via Inferential Statistics. In: IEEE Transactions on Knowledge and Data Engineering. 2018 ; pp. 1-14.

BibTeX

@article{18b4db2a4c9c45f797cd1fadc5409bdc,
title = "A Comparative Study of Ontology Matching Systems via Inferential Statistics",
abstract = "Comparing ontology matching systems are typically performed by comparing their average performances over multiple datasets. However, this paper examines the alignment systems using statistical inference since averaging is statistically unsafe and inappropriate. The statistical tests for comparison of two or multiple alignment systems are theoretically and empirically reviewed. For comparison of two systems, the Wilcoxon signed-rank and McNemar's mid-p and asymptotic tests are recommended due to their robustness and statistical safety in different circumstances. The Friedman and Quade tests with their corresponding post-hoc procedures are studied for comparison of multiple systems, and their [dis]advantages are discussed. The statistical methods are then applied to benchmark and multifarm tracks from the ontology matching evaluation initiative (OAEI) 2015 and their results are reported and visualized by critical difference diagrams.",
keywords = "Benchmark testing, Bergmann, Friedman, Geoscience, Holm, McNemar, Nemenyi, Ontologies, Ontology alignment evaluation, paired t-test, post-hoc, Quade, Robustness, Shaffer, Statistical analysis, Task analysis, Wilcoxon signed-rank",
author = "Majid Mohammadi and Wout Hofman and Tan, {Yao Hua}",
year = "2018",
month = may,
day = "29",
doi = "10.1109/TKDE.2018.2842019",
language = "English",
pages = "1--14",
journal = "IEEE Transactions on Knowledge & Data Engineering",
issn = "1041-4347",
publisher = "IEEE",

}

RIS

TY - JOUR

T1 - A Comparative Study of Ontology Matching Systems via Inferential Statistics

AU - Mohammadi, Majid

AU - Hofman, Wout

AU - Tan, Yao Hua

PY - 2018/5/29

Y1 - 2018/5/29

N2 - Comparing ontology matching systems are typically performed by comparing their average performances over multiple datasets. However, this paper examines the alignment systems using statistical inference since averaging is statistically unsafe and inappropriate. The statistical tests for comparison of two or multiple alignment systems are theoretically and empirically reviewed. For comparison of two systems, the Wilcoxon signed-rank and McNemar's mid-p and asymptotic tests are recommended due to their robustness and statistical safety in different circumstances. The Friedman and Quade tests with their corresponding post-hoc procedures are studied for comparison of multiple systems, and their [dis]advantages are discussed. The statistical methods are then applied to benchmark and multifarm tracks from the ontology matching evaluation initiative (OAEI) 2015 and their results are reported and visualized by critical difference diagrams.

AB - Comparing ontology matching systems are typically performed by comparing their average performances over multiple datasets. However, this paper examines the alignment systems using statistical inference since averaging is statistically unsafe and inappropriate. The statistical tests for comparison of two or multiple alignment systems are theoretically and empirically reviewed. For comparison of two systems, the Wilcoxon signed-rank and McNemar's mid-p and asymptotic tests are recommended due to their robustness and statistical safety in different circumstances. The Friedman and Quade tests with their corresponding post-hoc procedures are studied for comparison of multiple systems, and their [dis]advantages are discussed. The statistical methods are then applied to benchmark and multifarm tracks from the ontology matching evaluation initiative (OAEI) 2015 and their results are reported and visualized by critical difference diagrams.

KW - Benchmark testing

KW - Bergmann

KW - Friedman

KW - Geoscience

KW - Holm

KW - McNemar

KW - Nemenyi

KW - Ontologies

KW - Ontology alignment evaluation

KW - paired t-test

KW - post-hoc

KW - Quade

KW - Robustness

KW - Shaffer

KW - Statistical analysis

KW - Task analysis

KW - Wilcoxon signed-rank

UR - http://www.scopus.com/inward/record.url?scp=85047821778&partnerID=8YFLogxK

U2 - 10.1109/TKDE.2018.2842019

DO - 10.1109/TKDE.2018.2842019

M3 - Article

AN - SCOPUS:85047821778

SP - 1

EP - 14

JO - IEEE Transactions on Knowledge & Data Engineering

JF - IEEE Transactions on Knowledge & Data Engineering

SN - 1041-4347

ER -

ID: 45459279