The eye as a new side channel threat on smartphones

Ahmed Al-Haiqi, Mahamod Ismail, Rosdiadee Nordin

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

Eye tracking is not a new idea in human-computer interaction research. Since at least as early as 1990s, researchers have tried to utilize eye movement to drive or monitor user interaction with computers. The new idea is using eye movement tracking to breach the privacy of mobile users. In this paper, we study the feasibility of exploiting consumer-grade cameras built onto current smartphones to log eye gazes, and then estimating the keypad numbers being tapped by the user. Assuming Trojan applications with camera use permissions, this process could be implemented without the user contest or knowledge, imposing a potential new threat to the security and privacy of mobile users. Our approach does not involve machine learning methods. In these first preliminary proof-of-concept experiments, we mainly rely on a human attacker to manually analyze the collected images from the smartphone. Utilizing basic dimensionality and motion flow calculations, our results show a promising attack vector with more than 60% of taps inference accuracy.

Original languageEnglish
Title of host publicationProceeding - 2013 IEEE Student Conference on Research and Development, SCOReD 2013
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages475-479
Number of pages5
ISBN (Print)9781479926565
DOIs
Publication statusPublished - 6 Jan 2015
Event2013 11th IEEE Student Conference on Research and Development, SCOReD 2013 - Putrajaya
Duration: 16 Dec 201317 Dec 2013

Other

Other2013 11th IEEE Student Conference on Research and Development, SCOReD 2013
CityPutrajaya
Period16/12/1317/12/13

Fingerprint

Eye movements
Smartphones
Cameras
Computer keyboards
Human computer interaction
Learning systems
Experiments

Keywords

  • Android
  • eye-based keystrokes inference
  • eye-tracking
  • side-channel
  • smartphone security

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Biomedical Engineering
  • Electrical and Electronic Engineering

Cite this

Al-Haiqi, A., Ismail, M., & Nordin, R. (2015). The eye as a new side channel threat on smartphones. In Proceeding - 2013 IEEE Student Conference on Research and Development, SCOReD 2013 (pp. 475-479). [7002635] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/SCOReD.2013.7002635

The eye as a new side channel threat on smartphones. / Al-Haiqi, Ahmed; Ismail, Mahamod; Nordin, Rosdiadee.

Proceeding - 2013 IEEE Student Conference on Research and Development, SCOReD 2013. Institute of Electrical and Electronics Engineers Inc., 2015. p. 475-479 7002635.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Al-Haiqi, A, Ismail, M & Nordin, R 2015, The eye as a new side channel threat on smartphones. in Proceeding - 2013 IEEE Student Conference on Research and Development, SCOReD 2013., 7002635, Institute of Electrical and Electronics Engineers Inc., pp. 475-479, 2013 11th IEEE Student Conference on Research and Development, SCOReD 2013, Putrajaya, 16/12/13. https://doi.org/10.1109/SCOReD.2013.7002635
Al-Haiqi A, Ismail M, Nordin R. The eye as a new side channel threat on smartphones. In Proceeding - 2013 IEEE Student Conference on Research and Development, SCOReD 2013. Institute of Electrical and Electronics Engineers Inc. 2015. p. 475-479. 7002635 https://doi.org/10.1109/SCOReD.2013.7002635
Al-Haiqi, Ahmed ; Ismail, Mahamod ; Nordin, Rosdiadee. / The eye as a new side channel threat on smartphones. Proceeding - 2013 IEEE Student Conference on Research and Development, SCOReD 2013. Institute of Electrical and Electronics Engineers Inc., 2015. pp. 475-479
@inproceedings{d15c06c7aa204934a39c3e7614f4690a,
title = "The eye as a new side channel threat on smartphones",
abstract = "Eye tracking is not a new idea in human-computer interaction research. Since at least as early as 1990s, researchers have tried to utilize eye movement to drive or monitor user interaction with computers. The new idea is using eye movement tracking to breach the privacy of mobile users. In this paper, we study the feasibility of exploiting consumer-grade cameras built onto current smartphones to log eye gazes, and then estimating the keypad numbers being tapped by the user. Assuming Trojan applications with camera use permissions, this process could be implemented without the user contest or knowledge, imposing a potential new threat to the security and privacy of mobile users. Our approach does not involve machine learning methods. In these first preliminary proof-of-concept experiments, we mainly rely on a human attacker to manually analyze the collected images from the smartphone. Utilizing basic dimensionality and motion flow calculations, our results show a promising attack vector with more than 60{\%} of taps inference accuracy.",
keywords = "Android, eye-based keystrokes inference, eye-tracking, side-channel, smartphone security",
author = "Ahmed Al-Haiqi and Mahamod Ismail and Rosdiadee Nordin",
year = "2015",
month = "1",
day = "6",
doi = "10.1109/SCOReD.2013.7002635",
language = "English",
isbn = "9781479926565",
pages = "475--479",
booktitle = "Proceeding - 2013 IEEE Student Conference on Research and Development, SCOReD 2013",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - The eye as a new side channel threat on smartphones

AU - Al-Haiqi, Ahmed

AU - Ismail, Mahamod

AU - Nordin, Rosdiadee

PY - 2015/1/6

Y1 - 2015/1/6

N2 - Eye tracking is not a new idea in human-computer interaction research. Since at least as early as 1990s, researchers have tried to utilize eye movement to drive or monitor user interaction with computers. The new idea is using eye movement tracking to breach the privacy of mobile users. In this paper, we study the feasibility of exploiting consumer-grade cameras built onto current smartphones to log eye gazes, and then estimating the keypad numbers being tapped by the user. Assuming Trojan applications with camera use permissions, this process could be implemented without the user contest or knowledge, imposing a potential new threat to the security and privacy of mobile users. Our approach does not involve machine learning methods. In these first preliminary proof-of-concept experiments, we mainly rely on a human attacker to manually analyze the collected images from the smartphone. Utilizing basic dimensionality and motion flow calculations, our results show a promising attack vector with more than 60% of taps inference accuracy.

AB - Eye tracking is not a new idea in human-computer interaction research. Since at least as early as 1990s, researchers have tried to utilize eye movement to drive or monitor user interaction with computers. The new idea is using eye movement tracking to breach the privacy of mobile users. In this paper, we study the feasibility of exploiting consumer-grade cameras built onto current smartphones to log eye gazes, and then estimating the keypad numbers being tapped by the user. Assuming Trojan applications with camera use permissions, this process could be implemented without the user contest or knowledge, imposing a potential new threat to the security and privacy of mobile users. Our approach does not involve machine learning methods. In these first preliminary proof-of-concept experiments, we mainly rely on a human attacker to manually analyze the collected images from the smartphone. Utilizing basic dimensionality and motion flow calculations, our results show a promising attack vector with more than 60% of taps inference accuracy.

KW - Android

KW - eye-based keystrokes inference

KW - eye-tracking

KW - side-channel

KW - smartphone security

UR - http://www.scopus.com/inward/record.url?scp=84921724193&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84921724193&partnerID=8YFLogxK

U2 - 10.1109/SCOReD.2013.7002635

DO - 10.1109/SCOReD.2013.7002635

M3 - Conference contribution

AN - SCOPUS:84921724193

SN - 9781479926565

SP - 475

EP - 479

BT - Proceeding - 2013 IEEE Student Conference on Research and Development, SCOReD 2013

PB - Institute of Electrical and Electronics Engineers Inc.

ER -