Abstract
Abstract
This paper presents the identical twin test analysis of voice biometric authentication using the VoiceVantage™ VoiceCheck™ tool from THC Technologies Corp. Recorded voice samples from identical twins have been verified using the tool and the results analyzed statistically to determine the efficiency of the tool in detecting imposter attacks.
Samples were collected in a statistically controlled environment from the identical twins and verified them against each other’s to obtain their respective comparison scores. The scores were summarized and analyzed to derive the outcome of the test and relate the scores to the nature of the attempts on each phrase and its properties.
Research team
Tamil Muthu
Business and Information Technology
UOIT
Oshawa, Canada
tamilarasan.muthu@uoit.net
Sanjid Mahiba
Electrical and Computer Engineering
UOIT
Oshawa, Canada
sanjid.mahiba@uoit.net
THC Technologies Corp Identical Twin Testing Project – Bio/Acc Cont & Smart Card Tech ©2017 UOIT
I. Voice Biometrics
Biometric authentication provides identification and access control to any restricted environment via either its physical or network and application infrastructure. It works based on physical characteristics of the human body such as voice, fingerprint, retina, iris scan, face detection etc. Voice biometrics is considered the most cost effective and efficient biometric method which carries in addition to patterns, the mood, character and other emotional aspects of the speaker.
Voice
Fingerprint
Retina
Iris scan
Face detection
Voice carries, in addition to patterns, the mood, character and emotional aspects of the speaker.
II. About VoiceCheck™
THC Technologies Corporation’s VoiceVantage™ VoiceCheck™ is one of the leading voice verification biometric tool available in the biometrics market. This tool has been developed based on years of signal processing research and is currently deployed in telecom, finance, business and law enforcement verticals. It is very cost effective and works well with common recording and telephony devices.
Deployed in
Telecom
Finance
Business
Law enforcement
III. Voice Verification by VoiceCheck™
Voice verification also known as “Speaker Authentication” consists of several steps. It begins with enrollment, where the candidate provides multiple voice samples by pronouncing a phrase either given by the voice biometric system or chosen by the administrator. The next step is identity verification which verifies the provided live input against the prerecorded voice print to confirm identity. Verification is done by comparing the live voice sample with the enrolled voice print and calculating the comparison score based on the level of matching. A successful match will be the one with a comparison score equal to or above the specific threshold set by the administrator.
Comparing the live sample with the voice print consists of several steps; End Pointing, Analysis, Comparing and Scoring.
How VoiceCheck™ checks one phrase (illustration). Example scores: ‘favorite color’ phrase, Blue Yeti Pro averages from Fig. 7: twin A 76, twin B speaking as A 0.
End Pointing
End Pointing is identifying the exact beginning and ending of the phrase within the voice sample. Each phrase is recorded multiple times during enrollment and each of the voice print samples are compared against the live voice sample to determine the Score.
Analysis
The Analysis uses Linear Predictive Coefficient (LPC) Cepstral Analysis, a technique that slices the live voice sample print into 30ms frames and compares these slices against the earlier recorded voice print samples, also in multiple 30ms frames, to identify the Score.
Scoring
Scoring is the process of calculating the difference between the file being presented and the file in the database to create a similarity or confidence score.
Comparing and Scoring (DTW)
Both the Comparing and Scoring are achieved using a method called Dynamic Time Warping (DTW). The “live” speech is compared to each of the stored versions of that phrase and the best score is identified.
VoiceCheck™ uses a language-independent voice verification system in which a customer’s voice is enrolled into the system by creating and recording algorithmic examples of the customer’s speech (usually words or a text phrase) for use as a reference. When verifying an identity, the customer is prompted to speak the same words or phrase which is then compared to earlier stored versions. The two signals are compared based on signal timing, inflection and variance. If the new word or phrase is sufficiently similar to the previously stored versions, the claimed identity is accepted. If not, the person is assumed to be an imposter and the identity claim is rejected.
Good to know
Stored voiceprints are neither “wave files” nor audio samplings, but instead are distinct computations of algorithmic information.
IV. Significance of Identical Twin Tests
Identical twins are the most ideal candidates to validate any biometric system. Identical Twin testing is a superior reference and testing methodology as they are the closest natural imposters of each other. Non-identical twin tests have been used in the biometric industry to test and rate the efficiency of the biometric systems, however Identical Twin Testing is rare.
Characteristics measured
Nuance PerSay (in use at HSBC): 100 characteristics
VoiceCheck™: 600 total characteristics, a final 200 distinct characteristics voice print
600 total 200 distinct in the voice print 100 (Nuance PerSay)
Illustrative stock photo, not the twins who took part in the study.
Voice biometrics and twins in the news
2016 HSBC
In 2016 HSBC introduced voice biometrics to identify its customers to prevent banking fraud. The Nuance PerSay system in use at HSBC measures 100 different characteristics of the human voice to verify a user's identity (The VoiceCheck™ tool uses 600 total characteristics to create a final 200 distinct characteristics voice print). Banking customers input their account details and date of birth and then say: “My voice is my password.”
May 19, 2017 BBC Click
On May 19, 2017 BBC television reporter Dan Simmons on Click, the BBC’s flagship technology program [6] tested the Nuance PerSay product using his non-identical twin named Joe. Joe was able to access Dan’s account on an 8th attempt by impersonating Dan’s voice.
Several years earlier BBC Click
Several years earlier the same BBC program reporter Dan Simmons did a test of Agnitio, also a Nuance voice biometrics product, again using his non-identical twin brother Joe. [7] The Nuance Agnitio product failed to identify the brother imposter.
Both tests received significant media coverage particularly in financial and biometrics trade periodicals.
A. Voice Collection
The requirement was to create multiple voice prints of the identical twins participating in the test. The first task was to select the recording hardware and software platforms. Two different recording hardware products were used in voice sample collection to add an additional variant to the testing. The next task was to select the appropriate voice collecting environment in which to conduct the identical twins test.
The criteria for the enrollment process and project guidelines were provided by the vendor in detail. The identified candidates were thoroughly briefed on the enrollment process and efforts were made to ensure they were aware of the expectations of the voice collection process.
Candidates were asked to record their name, day/month and state/country of birth and favorite color as part of each enrollment process. Each candidate performed the enrollment twice, once as themselves and the other time as the imposter of their counterpart. The Voice Collection tool provided by the vendor is capable of evaluating the quality of the samples in real time and directs the speaker to repeat their phrase in the case of poor quality.
How each twin was recorded
Name
Day/month of birth
State/country of birth
Favorite color
Enrollment 1: as themselves
Twin A records A's phrases, twin B records B's phrases (the candidate scores A and B).
Enrollment 2: as the imposter
Each twin records the other twin's phrases: A'(B) is twin B speaking as A, B'(A) is twin A speaking as B.
Every phrase, every device
Four renditions of each phrase, on the Plantronics headset and on the Blue microphone.
B. Voice Verification
The VoiceCheck™ verification tool was provided by the vendor. This tool compares all the renditions of the candidate and of the imposter with each other and produces comparison scores.
We voice verified the samples collected using two different recording devices. Comparison Scores were recorded for further analysis.
Web version of Fig. 1 (the same output, as text)
As printed in the paper
Fig.1. VoiceCheck™ verification test results
C. Recording Hardware
We used two different vendors recording hardware to collect the samples. One was Plantronics Blackwire C320-M headset and the other one was Blue Yeti Pro. Below are their specifications for reference.
1) Plantronics Blackwire C320-M
Illustrative photo above. Specification as given in the paper:
| Connects to | PC via USB |
|---|---|
| Ideal for | Users transitioning to PC telephony for webinars, conference calls, music, and other multimedia applications |
| Audio performance | Wideband: Up to 6,800Hz, Hi-fi Stereo (binaural), echo cancellation, Dynamic EQ |
| Hearing protection | SoundGuard: Acoustic limiting for protection against sounds above 118 dBA |
| Limited warranty | 2 years |
| Models | C310 (monaural), C320 (binaural): UC Standard version built for UC applications and softphones from Avaya®, Cisco®, IBM®, Skype®, and more* C310-M (monaural), C320-M (binaural): Version optimized for Microsoft® Lync™ 2010, and Microsoft® OCS 2007 |
As printed in the paper
Fig.2. Plantronics device specification
2) Blue Yeti Pro
Illustrative photo above. Specification as given in the paper:
| Power Required/Consumption | 5V 500mA (USB)/48V DC (analog) |
|---|---|
| Sample Rate | 192 kHz |
| Bit Rate | 24-bit |
| Capsules | 3 Blue-proprietary 14mm condenser capsules |
| Polar Patterns | Cardioid, Bidirectional, Omnidirectional, Stereo |
| Frequency Response | 20Hz - 20kHz |
| Sensitivity | 4.5mV/Pa (1 kHz) |
| Max SPL | 120dB (THD: 0.5% 1kHz) |
As printed in the paper
Fig.3. Blue device specification
V. Score Analysis
All the recorded comparison scores from the verification process were statistically analyzed and displayed on a spreadsheet. The scores for each phrase were compared between the candidate’s score and imposter’s score per each specific recording device.
The lowest score of the Candidate with their own rendition is colored in green and the maximum imposter score is colored in red. An average score has been calculated for each phrase to summarize the findings.
How to read the tables
A and B: each twin speaking as themselves (the candidate). A'(B): twin B speaking as twin A. B'(A): twin A speaking as twin B (the imposter). In the raw output of Fig. 1, A is Alexis and B is Anthony.
Each block (R1–R4) compares one enrolled rendition with the four renditions in its rows.
Green lowest candidate scoreRed maximum imposter score
A. Phrase 1
Phrase 1 was the first and last name of the candidate. Four renditions of phrase 1 has been recorded from the candidate and the imposter using both of the recording devices.
For the twins, the last name is the same and the imposter has to imitate only the first name intentionally. The imposter scores are neither low nor high comparing to the other phrases.
Web version of Fig. 4. (same numbers as the paper)
| Rendition | Plantronics | Blue | ||||||
|---|---|---|---|---|---|---|---|---|
| A | A'(B) | B | B'(A) | A | A'(B) | B | B'(A) | |
| First and Last Name - R1 | ||||||||
| R1 | 100 | 5 | 100 | 24 | 100 | 40 | 100 | 0 |
| R2 | 37 | 23 | 58 | 0 | 61 | 39 | 31 | 0 |
| R3 | 42 | 14 | 65 | 0 | 59 | 51 | 0 | 0 |
| R4 | 34 | 16 | 58 | 17 | 24 | 53 | 0 | 22 |
| First and Last Name - R2 | ||||||||
| R1 | 37 | 0 | 58 | 30 | 61 | 42 | 31 | 28 |
| R2 | 100 | 13 | 100 | 17 | 100 | 41 | 100 | 27 |
| R3 | 51 | 0 | 59 | 15 | 30 | 32 | 24 | 40 |
| R4 | 51 | 3 | 59 | 29 | 5 | 43 | 55 | 0 |
| First and Last Name - R3 | ||||||||
| R1 | 42 | 0 | 65 | 31 | 59 | 0 | 0 | 24 |
| R2 | 51 | 25 | 59 | 9 | 30 | 7 | 24 | 19 |
| R3 | 100 | 24 | 100 | 3 | 100 | 0 | 100 | 0 |
| R4 | 46 | 21 | 62 | 31 | 55 | 4 | 73 | 0 |
| First and Last Name - R4 | ||||||||
| R1 | 37 | 0 | 58 | 35 | 24 | 0 | 0 | 47 |
| R2 | 100 | 13 | 59 | 28 | 5 | 3 | 55 | 44 |
| R3 | 51 | 0 | 62 | 15 | 55 | 0 | 73 | 6 |
| R4 | 51 | 3 | 100 | 30 | 100 | 0 | 100 | 0 |
| Average | 58 | 10 | 70 | 20 | 54 | 22 | 48 | 16 |
First and Last Name: the Average row
0
20
40
60
80
100
58
10
70
20
54
22
48
16
A
A'(B)
B
B'(A)
A
A'(B)
B
B'(A)
Plantronics
Blue
Candidate Imposter
As printed in the paper
Fig. 4. Phrase 1 - analysis
B. Phrase 2
Phrase 2 was the birth day and month of the candidate. Four renditions of phrase 2 were recorded from the candidate and from the imposter using both of the recording devices.
For the twins, phrase 2 was factually the same and the imposter didn’t need to imitate any part of it intentionally. The imposter scores are almost the same as phrase 3, at the same time relatively higher than phrase 1 and phrase 4.
Web version of Fig.5. (same numbers as the paper)
| Rendition | Plantronics | Blue | ||||||
|---|---|---|---|---|---|---|---|---|
| A | A'(B) | B | B'(A) | A | A'(B) | B | B'(A) | |
| B'day & Month - R1 | ||||||||
| R1 | 100 | 53 | 100 | 57 | 100 | 55 | 100 | 72 |
| R2 | 62 | 47 | 48 | 74 | 73 | 72 | 78 | 73 |
| R3 | 49 | 37 | 64 | 63 | 75 | 43 | 81 | 78 |
| R4 | 51 | 48 | 47 | 59 | 69 | 70 | 73 | 77 |
| B'day & Month - R2 | ||||||||
| R1 | 62 | 57 | 48 | 54 | 73 | 49 | 78 | 70 |
| R2 | 100 | 44 | 100 | 50 | 100 | 65 | 100 | 72 |
| R3 | 51 | 49 | 56 | 58 | 71 | 37 | 78 | 75 |
| R4 | 52 | 48 | 80 | 59 | 62 | 63 | 78 | 75 |
| B'day & Month - R3 | ||||||||
| R1 | 49 | 33 | 69 | 58 | 75 | 51 | 81 | 71 |
| R2 | 51 | 53 | 56 | 62 | 71 | 68 | 78 | 72 |
| R3 | 100 | 45 | 100 | 65 | 100 | 39 | 100 | 77 |
| R4 | 47 | 32 | 53 | 65 | 66 | 67 | 76 | 72 |
| B'day & Month - R4 | ||||||||
| R1 | 51 | 38 | 47 | 52 | 68 | 40 | 73 | 61 |
| R2 | 52 | 38 | 80 | 49 | 62 | 59 | 78 | 70 |
| R3 | 47 | 40 | 53 | 56 | 66 | 36 | 76 | 72 |
| R4 | 100 | 36 | 100 | 55 | 100 | 61 | 100 | 69 |
| Average | 64 | 44 | 69 | 59 | 77 | 55 | 83 | 72 |
B'day & Month: the Average row
0
20
40
60
80
100
64
44
69
59
77
55
83
72
A
A'(B)
B
B'(A)
A
A'(B)
B
B'(A)
Plantronics
Blue
Candidate Imposter
As printed in the paper
Fig.5. Phrase 2 – analysis
C. Phrase 3
Phrase 3 was the birth city and state of the candidate. Four renditions of phrase 3 were recorded from the candidate and the imposter using both of the recording devices.
For the twins, phrase 3 is factually the same and the imposter didn’t need to imitate any part of it intentionally. The imposter scores are almost the same as phrase 2, at the same time relatively higher than phrase 1 and phrase 4.
Web version of Fig.6. (same numbers as the paper)
| Rendition | Plantronics | Blue | ||||||
|---|---|---|---|---|---|---|---|---|
| A | A'(B) | B | B'(A) | A | A'(B) | B | B'(A) | |
| Birth City & State - R1 | ||||||||
| R1 | 100 | 47 | 100 | 49 | 100 | 68 | 100 | 77 |
| R2 | 59 | 52 | 56 | 45 | 71 | 60 | 73 | 72 |
| R3 | 45 | 36 | 59 | 36 | 71 | 53 | 74 | 74 |
| R4 | 48 | 39 | 53 | 43 | 67 | 52 | 72 | 73 |
| Birth City & State - R2 | ||||||||
| R1 | 59 | 36 | 56 | 55 | 71 | 62 | 73 | 74 |
| R2 | 100 | 40 | 100 | 57 | 100 | 62 | 100 | 66 |
| R3 | 61 | 33 | 76 | 62 | 68 | 51 | 72 | 68 |
| R4 | 55 | 33 | 65 | 53 | 73 | 49 | 69 | 73 |
| Birth City & State - R3 | ||||||||
| R1 | 45 | 39 | 59 | 56 | 71 | 69 | 74 | 73 |
| R2 | 61 | 41 | 76 | 55 | 68 | 63 | 72 | 63 |
| R3 | 100 | 37 | 100 | 60 | 100 | 57 | 100 | 73 |
| R4 | 62 | 42 | 65 | 57 | 76 | 62 | 76 | 72 |
| Birth City & State - R4 | ||||||||
| R1 | 48 | 39 | 53 | 47 | 67 | 70 | 72 | 76 |
| R2 | 55 | 52 | 65 | 66 | 73 | 65 | 69 | 71 |
| R3 | 62 | 52 | 65 | 54 | 76 | 56 | 76 | 73 |
| R4 | 100 | 50 | 100 | 51 | 100 | 55 | 100 | 75 |
| Average | 66 | 42 | 72 | 53 | 78 | 60 | 80 | 72 |
Birth City & State: the Average row
0
20
40
60
80
100
66
42
72
53
78
60
80
72
A
A'(B)
B
B'(A)
A
A'(B)
B
B'(A)
Plantronics
Blue
Candidate Imposter
As printed in the paper
Fig.6. Phrase 3 – analysis
D. Phrase 4
Phrase 4 was the favorite color of the candidate. Four renditions of phrase 4 were recorded from the candidate and the imposter using both the recording devices.
For the twins, phrase 4 is factually different between each other and the imposter needed to imitate the entire part of the phrase 4 intentionally. The imposter scores are remarkably low when compared to the other phrases.
Web version of Fig.7. (same numbers as the paper)
| Rendition | Plantronics | Blue | ||||||
|---|---|---|---|---|---|---|---|---|
| A | A'(B) | B | B'(A) | A | A'(B) | B | B'(A) | |
| Favorite Color - R1 | ||||||||
| R1 | 100 | 0 | 100 | 0 | 100 | 0 | 100 | 0 |
| R2 | 56 | 0 | 88 | 0 | 65 | 0 | 64 | 4 |
| R3 | 37 | 0 | 72 | 0 | 72 | 0 | 73 | 0 |
| R4 | 26 | 0 | 73 | 0 | 65 | 0 | 71 | 0 |
| Favorite Color - R2 | ||||||||
| R1 | 58 | 0 | 85 | 0 | 65 | 0 | 64 | 0 |
| R2 | 100 | 0 | 100 | 0 | 100 | 0 | 100 | 4 |
| R3 | 50 | 0 | 76 | 1 | 69 | 0 | 63 | 1 |
| R4 | 37 | 0 | 75 | 0 | 69 | 0 | 63 | 6 |
| Favorite Color - R3 | ||||||||
| R1 | 37 | 0 | 72 | 0 | 72 | 0 | 73 | 0 |
| R2 | 50 | 0 | 76 | 0 | 69 | 0 | 63 | 7 |
| R3 | 100 | 0 | 100 | 0 | 100 | 0 | 100 | 5 |
| R4 | 62 | 0 | 90 | 0 | 69 | 0 | 84 | 5 |
| Favorite Color - R4 | ||||||||
| R1 | 26 | 0 | 73 | 0 | 65 | 0 | 71 | 0 |
| R2 | 37 | 0 | 75 | 0 | 69 | 0 | 63 | 0 |
| R3 | 62 | 0 | 90 | 0 | 70 | 0 | 84 | 0 |
| R4 | 100 | 0 | 100 | 0 | 100 | 0 | 100 | 0 |
| Average | 59 | 0 | 84 | 0.1 | 76 | 0 | 77 | 2 |
Favorite Color: the Average row
0
20
40
60
80
100
59
0
84
0.1
76
0
77
2
A
A'(B)
B
B'(A)
A
A'(B)
B
B'(A)
Plantronics
Blue
Candidate Imposter
As printed in the paper
Fig.7. Phrase 4 - analysis
E. Summary
Plantronics – Phrases Vs Score
Average score of all renditions (from Fig. 4–7)
0
20
40
60
80
100
64.1
14.8
66.4
51.1
69.0
47.3
71.3
0.03
Name
Day/Month
City/State
Color
Candidate Imposter
Blue – Phrases Vs Score
Average score of all renditions (from Fig. 4–7)
0
20
40
60
80
100
51.1
19.1
80.0
63.5
78.9
65.8
76.7
1.0
Name
Day/Month
City/State
Color
Candidate Imposter
Comparing average scores of all renditions on each recording device shows that the probability of the identical twins successfully impersonates each other is high when the phrases are factually true for the imposter which in our opinion allows the imposter to record the phrase with normal emotions without any tension caused by impersonating the candidate.
As printed in the paper
Fig.8. Phrases Vs Score (Plantronics & Blue device)
VI. Conclusion
On analyzing the comparison scores generated by the VoiceCheck™ tool, it is clearly evident that the possibility of successfully spoofing between the identical twins is not possible provided the phrases are exclusively specific to the candidate. The comparison score of the imposter went above 60% but below 75% only twice among all the 32 attempts.
In both of these cases, the phrases are factually ‘true’ for the imposter. On average the comparison score is fairly high when the imposter has no need to lie or to make any effort to impersonate his counterpart. This clearly proves that voice is not just mere words; voice carries the emotional characteristics of the person at the moment of recording. Voice tone and difference in psychological intent, when speaking the truth versus attempting to impersonate, clearly caused a significant difference in the voice samples.
The scores for the phrase ‘favorite color’ which is factually different for each of the twins and the imposter score for this phrase were the absolute minimum when they attempted to spoof each other. This proves that choosing the phrase and its degree of exclusiveness to the individual is a key factor in effective and efficient authentication.
Sensitivity of the voice prints generated by VoiceCheck™’s algorithmic technology to all those above mentioned factors and properties of voice is very commendable and the results clearly prove that it is highly resistant to any such imposter attacks.
“
Voice is not just mere words; voice carries the emotional characteristics of the person at the moment of recording.
From the conclusion of the UOIT study
2 of 32
imposter scores went above 60% (and stayed below 75%); in both cases the phrase was factually ‘true’ for the imposter
Absolute minimum
imposter scores on ‘favorite color’, the phrase that is different for each twin
Phrase choice
its degree of exclusiveness to the individual is a key factor in effective and efficient authentication
Calibration
a recording device with ideal settings will reduce false acceptance rates
We attribute the difference between the two microphones to the hardware as the collection environment and input quality were exactly the same for both the devices. The 'Blue' recording device with relatively high scores for the imposter samples is a sophisticated device, more so than the 'Plantronics' one and has a number of configuration controls.
The test results show that using a recording device with ideal settings/configuration will make a difference in determining the 'real' threshold and will reduce false acceptance rates. Our results suggest that the 'Blue' device would benefit from careful calibration to enhance its accuracy. Additional samples from other identical twin candidates may generate additional insights into these results.
Acknowledgment
We would like to thank our twin candidates for their time and effort to participate in our test. Also, we would like to extend our sincere thanks to THC Technologies Corp and our professor Dr. Khalil El-Khatib for providing us with the opportunity to work on this project.
See VoiceCheck™ verify your own callers
Read how the same technology held up against AI voice clones in our independent deepfake tests, or ask us for a demo. The announcement of the twin tests is here.