Identical Twin Test Analysis using VoiceCheck™

date
 

Identical Twin Test Independent UOIT research

The closest natural imposters: identical twins put VoiceCheck™ to the test

Researchers at the University of Ontario Institute of Technology (UOIT) recorded identical twins on two microphones, had each twin speak as the other, and scored every attempt with VoiceCheck™. Every table and figure from the study is below.

See the resultsHow the test worked

VoiceVantageTHC Technologies Corp Identical Twin Testing Project – Bio/Acc Cont & Smart Card Tech ©2017 UOIT

 
 

Illustrative photo of identical twin brothers. A VoiceCheck panel shows twin A's own voice matching with 76 and twin B speaking as A rejected with 0 (favorite color phrase, Blue Yeti Pro averages from Fig. 7)

Illustrative stock photo, not the twins who took part in the study.

2

identical twins, each recorded as themselves and as the other twin

4 × 4

phrases × renditions per twin, on each device

2

recording devices: Plantronics Blackwire C320-M and Blue Yeti Pro

0

imposter average on the ‘favorite color’ phrase (A'(B), both devices)

Abstract

Abstract

This paper presents the identical twin test analysis of voice biometric authentication using the VoiceVantage™ VoiceCheck™ tool from THC Technologies Corp. Recorded voice samples from identical twins have been verified using the tool and the results analyzed statistically to determine the efficiency of the tool in detecting imposter attacks.

Samples were collected in a statistically controlled environment from the identical twins and verified them against each other’s to obtain their respective comparison scores. The scores were summarized and analyzed to derive the outcome of the test and relate the scores to the nature of the attempts on each phrase and its properties.

Research team

Tamil Muthu

Business and Information Technology
UOIT
Oshawa, Canada
tamilarasan.muthu@uoit.net

Sanjid Mahiba

Electrical and Computer Engineering
UOIT
Oshawa, Canada
sanjid.mahiba@uoit.net

THC Technologies Corp Identical Twin Testing Project – Bio/Acc Cont & Smart Card Tech ©2017 UOIT

I. Voice Biometrics

Biometric authentication provides identification and access control to any restricted environment via either its physical or network and application infrastructure. It works based on physical characteristics of the human body such as voice, fingerprint, retina, iris scan, face detection etc. Voice biometrics is considered the most cost effective and efficient biometric method which carries in addition to patterns, the mood, character and other emotional aspects of the speaker.

Voice

Fingerprint

Retina

Iris scan

Face detection

Voice carries, in addition to patterns, the mood, character and emotional aspects of the speaker.

II. About VoiceCheck™

THC Technologies Corporation’s VoiceVantage™ VoiceCheck™ is one of the leading voice verification biometric tool available in the biometrics market. This tool has been developed based on years of signal processing research and is currently deployed in telecom, finance, business and law enforcement verticals. It is very cost effective and works well with common recording and telephony devices.

Deployed in

Telecom

Finance

Business

Law enforcement

III. Voice Verification by VoiceCheck™

Voice verification also known as “Speaker Authentication” consists of several steps. It begins with enrollment, where the candidate provides multiple voice samples by pronouncing a phrase either given by the voice biometric system or chosen by the administrator. The next step is identity verification which verifies the provided live input against the prerecorded voice print to confirm identity. Verification is done by comparing the live voice sample with the enrolled voice print and calculating the comparison score based on the level of matching. A successful match will be the one with a comparison score equal to or above the specific threshold set by the administrator.

Comparing the live sample with the voice print consists of several steps; End Pointing, Analysis, Comparing and Scoring.

Animated illustration of the five steps: enrollment, end pointing, analysis in 30 ms frames, comparing with dynamic time warping, and scoring against the threshold

How VoiceCheck™ checks one phrase (illustration). Example scores: ‘favorite color’ phrase, Blue Yeti Pro averages from Fig. 7: twin A 76, twin B speaking as A 0.

End Pointing

End Pointing is identifying the exact beginning and ending of the phrase within the voice sample. Each phrase is recorded multiple times during enrollment and each of the voice print samples are compared against the live voice sample to determine the Score.

Analysis

The Analysis uses Linear Predictive Coefficient (LPC) Cepstral Analysis, a technique that slices the live voice sample print into 30ms frames and compares these slices against the earlier recorded voice print samples, also in multiple 30ms frames, to identify the Score.

Scoring

Scoring is the process of calculating the difference between the file being presented and the file in the database to create a similarity or confidence score.

Comparing and Scoring (DTW)

Both the Comparing and Scoring are achieved using a method called Dynamic Time Warping (DTW). The “live” speech is compared to each of the stored versions of that phrase and the best score is identified.

VoiceCheck™ uses a language-independent voice verification system in which a customer’s voice is enrolled into the system by creating and recording algorithmic examples of the customer’s speech (usually words or a text phrase) for use as a reference. When verifying an identity, the customer is prompted to speak the same words or phrase which is then compared to earlier stored versions. The two signals are compared based on signal timing, inflection and variance. If the new word or phrase is sufficiently similar to the previously stored versions, the claimed identity is accepted. If not, the person is assumed to be an imposter and the identity claim is rejected.

Good to know

Stored voiceprints are neither “wave files” nor audio samplings, but instead are distinct computations of algorithmic information.

IV. Significance of Identical Twin Tests

Identical twins are the most ideal candidates to validate any biometric system. Identical Twin testing is a superior reference and testing methodology as they are the closest natural imposters of each other. Non-identical twin tests have been used in the biometric industry to test and rate the efficiency of the biometric systems, however Identical Twin Testing is rare.

Characteristics measured

Nuance PerSay (in use at HSBC): 100 characteristics

 

VoiceCheck™: 600 total characteristics, a final 200 distinct characteristics voice print

 

 600 total 200 distinct in the voice print 100 (Nuance PerSay)

Identical twin sisters with dark hair in black tops on a warm light backdrop (illustrative stock photo)

Illustrative stock photo, not the twins who took part in the study.

Voice biometrics and twins in the news

2016 HSBC

In 2016 HSBC introduced voice biometrics to identify its customers to prevent banking fraud. The Nuance PerSay system in use at HSBC measures 100 different characteristics of the human voice to verify a user's identity (The VoiceCheck™ tool uses 600 total characteristics to create a final 200 distinct characteristics voice print). Banking customers input their account details and date of birth and then say: “My voice is my password.”

May 19, 2017 BBC Click

On May 19, 2017 BBC television reporter Dan Simmons on Click, the BBC’s flagship technology program [6] tested the Nuance PerSay product using his non-identical twin named Joe. Joe was able to access Dan’s account on an 8th attempt by impersonating Dan’s voice.

Several years earlier BBC Click

Several years earlier the same BBC program reporter Dan Simmons did a test of Agnitio, also a Nuance voice biometrics product, again using his non-identical twin brother Joe. [7] The Nuance Agnitio product failed to identify the brother imposter.

Both tests received significant media coverage particularly in financial and biometrics trade periodicals.

A. Voice Collection

The requirement was to create multiple voice prints of the identical twins participating in the test. The first task was to select the recording hardware and software platforms. Two different recording hardware products were used in voice sample collection to add an additional variant to the testing. The next task was to select the appropriate voice collecting environment in which to conduct the identical twins test.

The criteria for the enrollment process and project guidelines were provided by the vendor in detail. The identified candidates were thoroughly briefed on the enrollment process and efforts were made to ensure they were aware of the expectations of the voice collection process.

Candidates were asked to record their name, day/month and state/country of birth and favorite color as part of each enrollment process. Each candidate performed the enrollment twice, once as themselves and the other time as the imposter of their counterpart. The Voice Collection tool provided by the vendor is capable of evaluating the quality of the samples in real time and directs the speaker to repeat their phrase in the case of poor quality.

How each twin was recorded

Name

Day/month of birth

State/country of birth

Favorite color

Enrollment 1: as themselves

Twin A records A's phrases, twin B records B's phrases (the candidate scores A and B).

Enrollment 2: as the imposter

Each twin records the other twin's phrases: A'(B) is twin B speaking as A, B'(A) is twin A speaking as B.

Every phrase, every device

Four renditions of each phrase, on the Plantronics headset and on the Blue microphone.

B. Voice Verification

The VoiceCheck™ verification tool was provided by the vendor. This tool compares all the renditions of the candidate and of the imposter with each other and produces comparison scores.

We voice verified the samples collected using two different recording devices. Comparison Scores were recorded for further analysis.

Web version of Fig. 1 (the same output, as text)

   bash — TestTwins.sh

$ bash TestTwins.sh mitri Anthony Alexis

,mitri_Anthony_R00,mitri_Anthony_R01,mitri_Anthony_R02,mitri_Anthony_…
mitri_Anthony_E00_V00_R00,100,58,65,58,24,0,0,17,
mitri_Anthony_E00_V00_R01,58,100,59,59,30,17,15,29,
mitri_Anthony_E00_V00_R02,65,59,100,62,31,9,3,31,
mitri_Anthony_E00_V00_R03,58,59,62,100,35,28,15,30,
mitri_Anthony_E00_V01_R00,100,48,64,47,57,74,63,59,
mitri_Anthony_E00_V01_R01,48,100,56,80,54,50,58,59,
mitri_Anthony_E00_V01_R02,69,56,100,53,58,62,65,65,
mitri_Anthony_E00_V01_R03,47,80,53,100,52,49,56,55,
mitri_Anthony_E00_V02_R00,100,56,59,53,49,45,36,43,
mitri_Anthony_E00_V02_R01,56,100,76,65,55,57,62,53,
mitri_Anthony_E00_V02_R02,59,76,100,65,56,55,60,57,
mitri_Anthony_E00_V02_R03,53,65,65,100,47,66,54,51,
mitri_Anthony_E00_V03_R00,100,88,72,73,0,0,0,0,
mitri_Anthony_E00_V03_R01,85,100,76,75,0,0,1,0,
mitri_Anthony_E00_V03_R02,72,76,100,90,0,0,0,0,
mitri_Anthony_E00_V03_R03,73,75,90,100,0,0,0,0,
,mitri_Alexis_R00,mitri_Alexis_R01,mitri_Alexis_R02,mitri_Alexis_R03,m…
mitri_Alexis_E00_V00_R00,100,37,42,34,5,23,14,16,
mitri_Alexis_E00_V00_R01,37,100,51,51,0,13,0,3,
mitri_Alexis_E00_V00_R02,42,51,100,46,0,25,24,21,
mitri_Alexis_E00_V00_R03,34,51,46,100,0,1,0,5,
mitri_Alexis_E00_V01_R00,100,62,49,51,53,47,37,48,
mitri_Alexis_E00_V01_R01,62,100,51,52,57,44,49,48,
mitri_Alexis_E00_V01_R02,49,51,100,47,33,53,45,32,
mitri_Alexis_E00_V01_R03,51,52,47,100,38,38,40,36,
mitri_Alexis_E00_V02_R00,100,59,45,48,47,52,36,39,
mitri_Alexis_E00_V02_R01,59,100,61,55,36,40,33,33,
mitri_Alexis_E00_V02_R02,45,61,100,62,39,41,37,42,
mitri_Alexis_E00_V02_R03,48,55,62,100,39,52,52,50,
mitri_Alexis_E00_V03_R00,100,56,37,26,0,0,0,0,
mitri_Alexis_E00_V03_R01,58,100,50,37,0,0,0,0,
mitri_Alexis_E00_V03_R02,37,50,100,62,0,0,0,0,
mitri_Alexis_E00_V03_R03,26,37,62,100,0,0,0,0,

$ █

As printed in the paper

Fig.1. VoiceCheck™ verification test results (original figure from the UOIT paper)

Fig.1. VoiceCheck™ verification test results

C. Recording Hardware

We used two different vendors recording hardware to collect the samples. One was Plantronics Blackwire C320-M headset and the other one was Blue Yeti Pro. Below are their specifications for reference.

A woman wearing a USB headset at her desktop computer (illustrative photo)

1) Plantronics Blackwire C320-M

Illustrative photo above. Specification as given in the paper:

Connects toPC via USB
Ideal forUsers transitioning to PC telephony for webinars, conference calls, music, and other multimedia applications
Audio performanceWideband: Up to 6,800Hz, Hi-fi Stereo (binaural), echo cancellation, Dynamic EQ
Hearing protectionSoundGuard: Acoustic limiting for protection against sounds above 118 dBA
Limited warranty2 years
ModelsC310 (monaural), C320 (binaural): UC Standard version built for UC applications and softphones from Avaya®, Cisco®, IBM®, Skype®, and more*
C310-M (monaural), C320-M (binaural): Version optimized for Microsoft® Lync™ 2010, and Microsoft® OCS 2007

As printed in the paper

Fig.2. Plantronics device specification (original figure from the UOIT paper)

Fig.2. Plantronics device specification

A condenser microphone on a white desk (illustrative photo)

2) Blue Yeti Pro

Illustrative photo above. Specification as given in the paper:

Power Required/Consumption5V 500mA (USB)/48V DC (analog)
Sample Rate192 kHz
Bit Rate24-bit
Capsules3 Blue-proprietary 14mm condenser capsules
Polar PatternsCardioid, Bidirectional, Omnidirectional, Stereo
Frequency Response20Hz - 20kHz
Sensitivity4.5mV/Pa (1 kHz)
Max SPL120dB (THD: 0.5% 1kHz)

As printed in the paper

Fig.3. Blue device specification (original figure from the UOIT paper)

Fig.3. Blue device specification

V. Score Analysis

All the recorded comparison scores from the verification process were statistically analyzed and displayed on a spreadsheet. The scores for each phrase were compared between the candidate’s score and imposter’s score per each specific recording device.

The lowest score of the Candidate with their own rendition is colored in green and the maximum imposter score is colored in red. An average score has been calculated for each phrase to summarize the findings.

How to read the tables

A and B: each twin speaking as themselves (the candidate). A'(B): twin B speaking as twin A. B'(A): twin A speaking as twin B (the imposter). In the raw output of Fig. 1, A is Alexis and B is Anthony.

Each block (R1–R4) compares one enrolled rendition with the four renditions in its rows.

Green lowest candidate scoreRed maximum imposter score

A. Phrase 1

Phrase 1 was the first and last name of the candidate. Four renditions of phrase 1 has been recorded from the candidate and the imposter using both of the recording devices.

For the twins, the last name is the same and the imposter has to imitate only the first name intentionally. The imposter scores are neither low nor high comparing to the other phrases.

Web version of Fig. 4. (same numbers as the paper)

RenditionPlantronicsBlue
AA'(B)BB'(A)AA'(B)BB'(A)
First and Last Name - R1
R1100510024100401000
R237235806139310
R34214650595100
R4341658172453022
First and Last Name - R2
R1370583061423128
R210013100171004110027
R3510591530322440
R45135929543550
First and Last Name - R3
R14206531590024
R251255993072419
R310024100310001000
R446216231554730
First and Last Name - R4
R13705835240047
R2100135928535544
R35106215550736
R45131003010001000
Average5810702054224816

First and Last Name: the Average row

0

20

40

60

80

100

58

 

10

 

70

 

20

 

54

 

22

 

48

 

16

 

A

A'(B)

B

B'(A)

A

A'(B)

B

B'(A)

Plantronics

Blue

 Candidate Imposter

As printed in the paper

Fig. 4. Phrase 1 - analysis (original figure from the UOIT paper)

Fig. 4. Phrase 1 - analysis

B. Phrase 2

Phrase 2 was the birth day and month of the candidate. Four renditions of phrase 2 were recorded from the candidate and from the imposter using both of the recording devices.

For the twins, phrase 2 was factually the same and the imposter didn’t need to imitate any part of it intentionally. The imposter scores are almost the same as phrase 3, at the same time relatively higher than phrase 1 and phrase 4.

Web version of Fig.5. (same numbers as the paper)

RenditionPlantronicsBlue
AA'(B)BB'(A)AA'(B)BB'(A)
B'day & Month - R1
R110053100571005510072
R26247487473727873
R34937646375438178
R45148475969707377
B'day & Month - R2
R16257485473497870
R210044100501006510072
R35149565871377875
R45248805962637875
B'day & Month - R3
R14933695875518171
R25153566271687872
R310045100651003910077
R44732536566677672
B'day & Month - R4
R15138475268407361
R25238804962597870
R34740535666367672
R410036100551006110069
Average6444695977558372

B'day & Month: the Average row

0

20

40

60

80

100

64

 

44

 

69

 

59

 

77

 

55

 

83

 

72

 

A

A'(B)

B

B'(A)

A

A'(B)

B

B'(A)

Plantronics

Blue

 Candidate Imposter

As printed in the paper

Fig.5. Phrase 2 – analysis (original figure from the UOIT paper)

Fig.5. Phrase 2 – analysis

C. Phrase 3

Phrase 3 was the birth city and state of the candidate. Four renditions of phrase 3 were recorded from the candidate and the imposter using both of the recording devices.

For the twins, phrase 3 is factually the same and the imposter didn’t need to imitate any part of it intentionally. The imposter scores are almost the same as phrase 2, at the same time relatively higher than phrase 1 and phrase 4.

Web version of Fig.6. (same numbers as the paper)

RenditionPlantronicsBlue
AA'(B)BB'(A)AA'(B)BB'(A)
Birth City & State - R1
R110047100491006810077
R25952564571607372
R34536593671537474
R44839534367527273
Birth City & State - R2
R15936565571627374
R210040100571006210066
R36133766268517268
R45533655373496973
Birth City & State - R3
R14539595671697473
R26141765568637263
R310037100601005710073
R46242655776627672
Birth City & State - R4
R14839534767707276
R25552656673656971
R36252655476567673
R410050100511005510075
Average6642725378608072

Birth City & State: the Average row

0

20

40

60

80

100

66

 

42

 

72

 

53

 

78

 

60

 

80

 

72

 

A

A'(B)

B

B'(A)

A

A'(B)

B

B'(A)

Plantronics

Blue

 Candidate Imposter

As printed in the paper

Fig.6. Phrase 3 – analysis (original figure from the UOIT paper)

Fig.6. Phrase 3 – analysis

D. Phrase 4

Phrase 4 was the favorite color of the candidate. Four renditions of phrase 4 were recorded from the candidate and the imposter using both the recording devices.

For the twins, phrase 4 is factually different between each other and the imposter needed to imitate the entire part of the phrase 4 intentionally. The imposter scores are remarkably low when compared to the other phrases.

Web version of Fig.7. (same numbers as the paper)

RenditionPlantronicsBlue
AA'(B)BB'(A)AA'(B)BB'(A)
Favorite Color - R1
R11000100010001000
R2560880650644
R3370720720730
R4260730650710
Favorite Color - R2
R1580850650640
R21000100010001004
R3500761690631
R4370750690636
Favorite Color - R3
R1370720720730
R2500760690637
R31000100010001005
R4620900690845
Favorite Color - R4
R1260730650710
R2370750690630
R3620900700840
R41000100010001000
Average590840.1760772

Favorite Color: the Average row

0

20

40

60

80

100

59

 

0

 

84

 

0.1

 

76

 

0

 

77

 

2

 

A

A'(B)

B

B'(A)

A

A'(B)

B

B'(A)

Plantronics

Blue

 Candidate Imposter

As printed in the paper

Fig.7. Phrase 4 - analysis (original figure from the UOIT paper)

Fig.7. Phrase 4 - analysis

E. Summary

Plantronics – Phrases Vs Score

Average score of all renditions (from Fig. 4–7)

0

20

40

60

80

100

64.1

 

14.8

 

66.4

 

51.1

 

69.0

 

47.3

 

71.3

 

0.03

 

Name

Day/Month

City/State

Color

 Candidate Imposter

Blue – Phrases Vs Score

Average score of all renditions (from Fig. 4–7)

0

20

40

60

80

100

51.1

 

19.1

 

80.0

 

63.5

 

78.9

 

65.8

 

76.7

 

1.0

 

Name

Day/Month

City/State

Color

 Candidate Imposter

Comparing average scores of all renditions on each recording device shows that the probability of the identical twins successfully impersonates each other is high when the phrases are factually true for the imposter which in our opinion allows the imposter to record the phrase with normal emotions without any tension caused by impersonating the candidate.

As printed in the paper

Plantronics -Phrases Vs Score (original chart from the UOIT paper)

Blue- Phrases Vs Score (original chart from the UOIT paper)

Fig.8. Phrases Vs Score (Plantronics & Blue device)

VI. Conclusion

On analyzing the comparison scores generated by the VoiceCheck™ tool, it is clearly evident that the possibility of successfully spoofing between the identical twins is not possible provided the phrases are exclusively specific to the candidate. The comparison score of the imposter went above 60% but below 75% only twice among all the 32 attempts.

In both of these cases, the phrases are factually ‘true’ for the imposter. On average the comparison score is fairly high when the imposter has no need to lie or to make any effort to impersonate his counterpart. This clearly proves that voice is not just mere words; voice carries the emotional characteristics of the person at the moment of recording. Voice tone and difference in psychological intent, when speaking the truth versus attempting to impersonate, clearly caused a significant difference in the voice samples.

The scores for the phrase ‘favorite color’ which is factually different for each of the twins and the imposter score for this phrase were the absolute minimum when they attempted to spoof each other. This proves that choosing the phrase and its degree of exclusiveness to the individual is a key factor in effective and efficient authentication.

Sensitivity of the voice prints generated by VoiceCheck™’s algorithmic technology to all those above mentioned factors and properties of voice is very commendable and the results clearly prove that it is highly resistant to any such imposter attacks.

“

Voice is not just mere words; voice carries the emotional characteristics of the person at the moment of recording.

From the conclusion of the UOIT study

2 of 32

imposter scores went above 60% (and stayed below 75%); in both cases the phrase was factually ‘true’ for the imposter

Absolute minimum

imposter scores on ‘favorite color’, the phrase that is different for each twin

Phrase choice

its degree of exclusiveness to the individual is a key factor in effective and efficient authentication

Calibration

a recording device with ideal settings will reduce false acceptance rates

We attribute the difference between the two microphones to the hardware as the collection environment and input quality were exactly the same for both the devices. The 'Blue' recording device with relatively high scores for the imposter samples is a sophisticated device, more so than the 'Plantronics' one and has a number of configuration controls.

The test results show that using a recording device with ideal settings/configuration will make a difference in determining the 'real' threshold and will reduce false acceptance rates. Our results suggest that the 'Blue' device would benefit from careful calibration to enhance its accuracy. Additional samples from other identical twin candidates may generate additional insights into these results.

Acknowledgment

We would like to thank our twin candidates for their time and effort to participate in our test. Also, we would like to extend our sincere thanks to THC Technologies Corp and our professor Dr. Khalil El-Khatib for providing us with the opportunity to work on this project.

See VoiceCheck™ verify your own callers

Read how the same technology held up against AI voice clones in our independent deepfake tests, or ask us for a demo. The announcement of the twin tests is here.