Combining accuracy and prior sensitivity for classifier design under prior uncertainty

TCW Landgrebe; RPW Duin

Combining accuracy and prior sensitivity for classifier design under prior uncertainty

TCW Landgrebe, RPW Duin

Multimedia Computing

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

2 Citations (Scopus)

Abstract

Consideringtheclassi¿cationprobleminwhichclasspriorsormisallocationcostsarenotknownprecisely,receiveroperatorcharacteristic(ROC)analysishasbecomeastandardtoolinpatternrecognitionforobtainingintegratedperformancemeasurestocopewiththeuncertainty.Similarly,insituationsinwhichpriorsmayvaryinapplication,theROCcanbeusedtoinspectperformanceovertheexpectedrangeofvariation.InthispaperwearguethateventhoughmeasuressuchastheareaundertheROC(AUC)areusefulinobtaininganintegratedperformancemeasureindependentofthepriors,itmayalsobeimportanttoincorporatethesensitivityacrosstheexpectedprior-range.Weshowthataclassi¿ermayresultinagoodAUCscore,butapoor(large)priorsensitivity,whichmaybeundesirable.Amethodologyisproposedthatcombinesbothaccuracyandsensitivity,providinganewmodelselectioncriterionthatisrelevanttocertainproblems.Experimentsshowthatincorporatingsensitivityisveryimportantinsomerealisticscenarios,leadingtobettermodelselectioninsomecases

Original language	Undefined/Unknown
Title of host publication	Structural, syntactic and statistical pattern recognition
Editors	DY Yeung, JT Kwok, A Fred, F Roli, D de Ridder
Place of Publication	Berlin-Heidelberg
Publisher	Springer
Pages	512-521
Number of pages	10
ISBN (Print)	3-540-37236-9
Publication status	Published - 2006
Event	Joint IAPR International Workshops SSPR 2006 and SPR 2006, Hong Kong, China - Heidelberg Duration: 17 Aug 2006 → 19 Aug 2006

Publication series

Name
Publisher	Springer

Name	Lecture Notes in Computer Science
Volume	4109
ISSN (Print)	0302-9743

Conference

Conference	Joint IAPR International Workshops SSPR 2006 and SPR 2006, Hong Kong, China
Period	17/08/06 → 19/08/06

Keywords

conference contrib. refereed
CWTS 0.75 <= JFIS < 2.00

Cite this

@inproceedings{b691487f30ce47689c44fc14d2ac0012,

title = "Combining accuracy and prior sensitivity for classifier design under prior uncertainty",

abstract = "Consideringtheclassi¿cationprobleminwhichclasspriorsormisallocationcostsarenotknownprecisely,receiveroperatorcharacteristic(ROC)analysishasbecomeastandardtoolinpatternrecognitionforobtainingintegratedperformancemeasurestocopewiththeuncertainty.Similarly,insituationsinwhichpriorsmayvaryinapplication,theROCcanbeusedtoinspectperformanceovertheexpectedrangeofvariation.InthispaperwearguethateventhoughmeasuressuchastheareaundertheROC(AUC)areusefulinobtaininganintegratedperformancemeasureindependentofthepriors,itmayalsobeimportanttoincorporatethesensitivityacrosstheexpectedprior-range.Weshowthataclassi¿ermayresultinagoodAUCscore,butapoor(large)priorsensitivity,whichmaybeundesirable.Amethodologyisproposedthatcombinesbothaccuracyandsensitivity,providinganewmodelselectioncriterionthatisrelevanttocertainproblems.Experimentsshowthatincorporatingsensitivityisveryimportantinsomerealisticscenarios,leadingtobettermodelselectioninsomecases",

keywords = "conference contrib. refereed, CWTS 0.75 <= JFIS < 2.00",

author = "TCW Landgrebe and RPW Duin",

year = "2006",

language = "Undefined/Unknown",

isbn = "3-540-37236-9",

publisher = "Springer",

pages = "512--521",

editor = "DY Yeung and JT Kwok and A Fred and F Roli and {de Ridder}, D",

booktitle = "Structural, syntactic and statistical pattern recognition",

note = "Joint IAPR International Workshops SSPR 2006 and SPR 2006, Hong Kong, China ; Conference date: 17-08-2006 Through 19-08-2006",

}

Landgrebe, TCW & Duin, RPW 2006, Combining accuracy and prior sensitivity for classifier design under prior uncertainty. in DY Yeung, JT Kwok, A Fred, F Roli & D de Ridder (eds), Structural, syntactic and statistical pattern recognition. Lecture Notes in Computer Science, vol. 4109, Springer, Berlin-Heidelberg, pp. 512-521, Joint IAPR International Workshops SSPR 2006 and SPR 2006, Hong Kong, China, 17/08/06.

Combining accuracy and prior sensitivity for classifier design under prior uncertainty. / Landgrebe, TCW; Duin, RPW.
Structural, syntactic and statistical pattern recognition. ed. / DY Yeung; JT Kwok; A Fred; F Roli; D de Ridder. Berlin-Heidelberg: Springer, 2006. p. 512-521 (Lecture Notes in Computer Science; Vol. 4109).

Research output: Chapter in Book/Conference proceedings/Edited volume › Conference contribution › Scientific › peer-review

TY - GEN

T1 - Combining accuracy and prior sensitivity for classifier design under prior uncertainty

AU - Landgrebe, TCW

AU - Duin, RPW

PY - 2006

Y1 - 2006

N2 - Consideringtheclassi¿cationprobleminwhichclasspriorsormisallocationcostsarenotknownprecisely,receiveroperatorcharacteristic(ROC)analysishasbecomeastandardtoolinpatternrecognitionforobtainingintegratedperformancemeasurestocopewiththeuncertainty.Similarly,insituationsinwhichpriorsmayvaryinapplication,theROCcanbeusedtoinspectperformanceovertheexpectedrangeofvariation.InthispaperwearguethateventhoughmeasuressuchastheareaundertheROC(AUC)areusefulinobtaininganintegratedperformancemeasureindependentofthepriors,itmayalsobeimportanttoincorporatethesensitivityacrosstheexpectedprior-range.Weshowthataclassi¿ermayresultinagoodAUCscore,butapoor(large)priorsensitivity,whichmaybeundesirable.Amethodologyisproposedthatcombinesbothaccuracyandsensitivity,providinganewmodelselectioncriterionthatisrelevanttocertainproblems.Experimentsshowthatincorporatingsensitivityisveryimportantinsomerealisticscenarios,leadingtobettermodelselectioninsomecases

AB - Consideringtheclassi¿cationprobleminwhichclasspriorsormisallocationcostsarenotknownprecisely,receiveroperatorcharacteristic(ROC)analysishasbecomeastandardtoolinpatternrecognitionforobtainingintegratedperformancemeasurestocopewiththeuncertainty.Similarly,insituationsinwhichpriorsmayvaryinapplication,theROCcanbeusedtoinspectperformanceovertheexpectedrangeofvariation.InthispaperwearguethateventhoughmeasuressuchastheareaundertheROC(AUC)areusefulinobtaininganintegratedperformancemeasureindependentofthepriors,itmayalsobeimportanttoincorporatethesensitivityacrosstheexpectedprior-range.Weshowthataclassi¿ermayresultinagoodAUCscore,butapoor(large)priorsensitivity,whichmaybeundesirable.Amethodologyisproposedthatcombinesbothaccuracyandsensitivity,providinganewmodelselectioncriterionthatisrelevanttocertainproblems.Experimentsshowthatincorporatingsensitivityisveryimportantinsomerealisticscenarios,leadingtobettermodelselectioninsomecases

KW - conference contrib. refereed

KW - CWTS 0.75 <= JFIS < 2.00

UR - http://www.springerlink.com/content/j7411g54780230j7/fulltext.pdf

M3 - Conference contribution

SN - 3-540-37236-9

SP - 512

EP - 521

BT - Structural, syntactic and statistical pattern recognition

A2 - Yeung, DY

A2 - Kwok, JT

A2 - Fred, A

A2 - Roli, F

A2 - de Ridder, D

PB - Springer

CY - Berlin-Heidelberg

T2 - Joint IAPR International Workshops SSPR 2006 and SPR 2006, Hong Kong, China

Y2 - 17 August 2006 through 19 August 2006

ER -

Combining accuracy and prior sensitivity for classifier design under prior uncertainty

Abstract

Publication series

Conference

Keywords

Other files and links

Cite this