The purpose of this Technical Report is to identify and characterize those aspects of performance testing that are unique to examiner assisted biometric applications.<\/p>\n
An examiner assisted biometric system has the following characteristics:<\/p>\n
reliant on the interaction and skill of a human examiner for one or more stages of the complete biometric process, be it data capture, enrolment, template generation, or final decision;<\/p>\n<\/li>\n
can incorporate identification functionality, verification functionality, or both;<\/p>\n<\/li>\n
will use a combination of the examiner\u2019s input and the functionality of the biometric algorithm to execute the complete biometric process;<\/p>\n<\/li>\n
will likely have inbuilt examination toolsets to assist the human examiner when enrolling biometric samples or when comparing the match results provided by the biometric algorithm.<\/p>\n<\/li>\n<\/ul>\n
Although there is a wide variation in the use of the term \u201cexaminer\u201d in the context of an \u201cexaminer assisted biometric system\u201d, as defined in this Technical Report, an \u201cexaminer\u201d typically has the following characteristics:<\/p>\n
field expert in the biometric modality being exploited;<\/p>\n<\/li>\n
trained to use the system to an advanced degree of proficiency;<\/p>\n<\/li>\n
authorized to override the biometric system\u2019s decisions in particular when accepting or rejecting a match decision based on their own examination of the biometric samples and the results returned.<\/p>\n<\/li>\n<\/ul>\n
Assessing an examiner\u2019s level of expertise is excluded from the scope of this Technical Report. However, the skill of the examiner does have a major bearing on system performance and vice versa. Measuring or assessing the ability of an examiner to employ their skills might be necessary to properly evaluate the performance of an examiner-assisted system.<\/p>\n
Other individuals, such as administrative users, or subjects whose biometrics are used within the system are not considered in this Technical Report. It is outside the scope of this Technical Report to consider non-expert examiners.<\/p>\n
| PDF Pages<\/th>\n | PDF Title<\/th>\n<\/tr>\n | ||||||
|---|---|---|---|---|---|---|---|
| 6<\/td>\n | Foreword <\/td>\n<\/tr>\n | ||||||
| 7<\/td>\n | Introduction <\/td>\n<\/tr>\n | ||||||
| 9<\/td>\n | 1\tScope 2\tTerms and definitions <\/td>\n<\/tr>\n | ||||||
| 10<\/td>\n | 3\tSymbols and abbreviated terms 4\tExample of an examiner assisted search process <\/td>\n<\/tr>\n | ||||||
| 12<\/td>\n | 5\tFactors to consider when evaluating examiner assisted biometric applications 5.1\tGeneral <\/td>\n<\/tr>\n | ||||||
| 13<\/td>\n | 5.2\tSystem-related factors to consider when evaluating examiner assisted biometric applications 5.2.1\tDependencies in the flow process\u00a0\u2014\u00a0Where does the examiner interact with the system? 5.2.2\tSystem and stage-level performance measurement <\/td>\n<\/tr>\n | ||||||
| 15<\/td>\n | 5.2.3\tMeasuring \u2018true\u2019 operational performance <\/td>\n<\/tr>\n | ||||||
| 16<\/td>\n | 5.2.4\tThe impact of prior probabilities on human performance 5.2.5\tConfidence Levels <\/td>\n<\/tr>\n | ||||||
| 17<\/td>\n | 5.2.6\tThe impact of automated systems on human performance 5.3\tExaminer-related factors to consider when evaluating examiner assisted biometric applications 5.3.1\tAn Examiner\u2019s perception of the system\u2019s accuracy <\/td>\n<\/tr>\n | ||||||
| 18<\/td>\n | 5.3.2\tUsability and examiner acceptance 5.3.3\tTraining and expertise <\/td>\n<\/tr>\n | ||||||
| 19<\/td>\n | 5.3.4\tWorkload 5.3.5\tBias in decision making <\/td>\n<\/tr>\n | ||||||
| 20<\/td>\n | 5.3.6\tIndividual differences between examiners 6\tPerformance evaluation of examiner assisted systems 6.1\tTypes of Evaluation <\/td>\n<\/tr>\n | ||||||
| 21<\/td>\n | 6.2\tPerformance measures for examiner assisted biometric systems 6.2.1\tIntroduction 6.2.2\tMeasures of accuracy <\/td>\n<\/tr>\n | ||||||
| 22<\/td>\n | 6.2.3\tExaminer-assisted performance considerations in watch list scenarios 6.2.4\tDiscrimination and bias 6.2.5\tExaminer Decision Confidence 6.2.6\tProcessing speed <\/td>\n<\/tr>\n | ||||||
| 23<\/td>\n | 6.3\tUsability assessment 6.3.1\tIntroduction 6.3.2\tQualitative observations 6.3.3\tQuestionnaires 6.3.4\tInterviews and focus groups 6.4\tReporting results <\/td>\n<\/tr>\n | ||||||
| 24<\/td>\n | 6.5\tApplying controls in evaluations 6.5.1\tIntroduction 6.5.2\tControls for examiner expertise 6.5.3\tControls for examiner decision bias <\/td>\n<\/tr>\n | ||||||
| 25<\/td>\n | 6.5.4\tControls for the test environment 6.5.5\tControls for variations in examiner input <\/td>\n<\/tr>\n | ||||||
| 26<\/td>\n | 6.6\tEvaluation challenges 6.6.1\tIntroduction 6.6.2\tChallenges with testing on a live operational system 6.6.3\tChallenges in repeatable operational test <\/td>\n<\/tr>\n | ||||||
| 27<\/td>\n | Bibliography <\/td>\n<\/tr>\n<\/table>\n","protected":false},"excerpt":{"rendered":" Information technology. Biometrics. Evaluation of examiner assisted biometric applications<\/b><\/p>\n |