BSI PD ISO/IEC TR 29189:2015
$142.49
Information technology. Biometrics. Evaluation of examiner assisted biometric applications
Published By | Publication Date | Number of Pages |
BSI | 2015 | 32 |
The purpose of this Technical Report is to identify and characterize those aspects of performance testing that are unique to examiner assisted biometric applications.
An examiner assisted biometric system has the following characteristics:
-
reliant on the interaction and skill of a human examiner for one or more stages of the complete biometric process, be it data capture, enrolment, template generation, or final decision;
-
can incorporate identification functionality, verification functionality, or both;
-
will use a combination of the examiner’s input and the functionality of the biometric algorithm to execute the complete biometric process;
-
will likely have inbuilt examination toolsets to assist the human examiner when enrolling biometric samples or when comparing the match results provided by the biometric algorithm.
Although there is a wide variation in the use of the term “examiner” in the context of an “examiner assisted biometric system”, as defined in this Technical Report, an “examiner” typically has the following characteristics:
-
field expert in the biometric modality being exploited;
-
trained to use the system to an advanced degree of proficiency;
-
authorized to override the biometric system’s decisions in particular when accepting or rejecting a match decision based on their own examination of the biometric samples and the results returned.
Assessing an examiner’s level of expertise is excluded from the scope of this Technical Report. However, the skill of the examiner does have a major bearing on system performance and vice versa. Measuring or assessing the ability of an examiner to employ their skills might be necessary to properly evaluate the performance of an examiner-assisted system.
Other individuals, such as administrative users, or subjects whose biometrics are used within the system are not considered in this Technical Report. It is outside the scope of this Technical Report to consider non-expert examiners.
PDF Catalog
PDF Pages | PDF Title |
---|---|
6 | Foreword |
7 | Introduction |
9 | 1 Scope 2 Terms and definitions |
10 | 3 Symbols and abbreviated terms 4 Example of an examiner assisted search process |
12 | 5 Factors to consider when evaluating examiner assisted biometric applications 5.1 General |
13 | 5.2 System-related factors to consider when evaluating examiner assisted biometric applications 5.2.1 Dependencies in the flow process — Where does the examiner interact with the system? 5.2.2 System and stage-level performance measurement |
15 | 5.2.3 Measuring ‘true’ operational performance |
16 | 5.2.4 The impact of prior probabilities on human performance 5.2.5 Confidence Levels |
17 | 5.2.6 The impact of automated systems on human performance 5.3 Examiner-related factors to consider when evaluating examiner assisted biometric applications 5.3.1 An Examiner’s perception of the system’s accuracy |
18 | 5.3.2 Usability and examiner acceptance 5.3.3 Training and expertise |
19 | 5.3.4 Workload 5.3.5 Bias in decision making |
20 | 5.3.6 Individual differences between examiners 6 Performance evaluation of examiner assisted systems 6.1 Types of Evaluation |
21 | 6.2 Performance measures for examiner assisted biometric systems 6.2.1 Introduction 6.2.2 Measures of accuracy |
22 | 6.2.3 Examiner-assisted performance considerations in watch list scenarios 6.2.4 Discrimination and bias 6.2.5 Examiner Decision Confidence 6.2.6 Processing speed |
23 | 6.3 Usability assessment 6.3.1 Introduction 6.3.2 Qualitative observations 6.3.3 Questionnaires 6.3.4 Interviews and focus groups 6.4 Reporting results |
24 | 6.5 Applying controls in evaluations 6.5.1 Introduction 6.5.2 Controls for examiner expertise 6.5.3 Controls for examiner decision bias |
25 | 6.5.4 Controls for the test environment 6.5.5 Controls for variations in examiner input |
26 | 6.6 Evaluation challenges 6.6.1 Introduction 6.6.2 Challenges with testing on a live operational system 6.6.3 Challenges in repeatable operational test |
27 | Bibliography |