A review of multimodal interaction technique in augmented reality environment

Siti Soleha Muhammad Nizam, Rimaniza Zainal Abidin, Nurhazarifah Che Hashim, Lam Meng Chun, Haslina Arshad, Nazatul Aini Abd Majid

Research output: Contribution to journalArticle

2 Citations (Scopus)

Abstract

Augmented Reality (AR) has proposed several types of interaction techniques such as 3D interactions, natural interactions, tangible interactions, spatial awareness interactions and multimodal interactions. Usually, interaction technique in AR involve unimodal interaction technique that only allows user to interact with AR content by using one modality such as gesture, speech, click, etc. Meanwhile, the combination of more than one modality is called multimodal. Multimodal can contribute to human and computer interaction more efficient and will enhance better user experience. This is because, there are a lot of issues have been found when user use unimodal interaction technique in AR environment such as fat fingers. Recent research has shown that multimodal interface (MMI) has been explored in AR environment and has been applied in various domain. This paper presents an empirical study of some of the key aspects and issues in multimodal interaction augmented reality, touching on the interaction technique and system framework. We reviewed the question of what are the interaction techniques that have been used to perform a multimodal interaction in AR environment and what are the integrated components applied in multimodal interaction AR frameworks. These two questions were used to be analysed in order to find the trends in multimodal field as a main contribution of this paper. We found that gesture, speech and touch are frequently used to manipulate virtual object. Most of the integrated component in MMI AR framework discussed only on the concept of the framework components or the information centred design between the components. Finally, we conclude this paper by providing ideas for future work involving this field.

Original languageEnglish
Pages (from-to)1460-1469
Number of pages10
JournalInternational Journal on Advanced Science, Engineering and Information Technology
Volume8
Issue number4-2
Publication statusPublished - 1 Jan 2018

Fingerprint

Augmented reality
Gestures
Touch
Fingers
methodology
Fats
touch (sensation)
Research
Oils and fats
lipids

Keywords

  • Augmented reality
  • Multimodal interaction technique
  • Review

ASJC Scopus subject areas

  • Computer Science(all)
  • Agricultural and Biological Sciences(all)
  • Engineering(all)

Cite this

A review of multimodal interaction technique in augmented reality environment. / Nizam, Siti Soleha Muhammad; Abidin, Rimaniza Zainal; Hashim, Nurhazarifah Che; Meng Chun, Lam; Arshad, Haslina; Abd Majid, Nazatul Aini.

In: International Journal on Advanced Science, Engineering and Information Technology, Vol. 8, No. 4-2, 01.01.2018, p. 1460-1469.

Research output: Contribution to journalArticle

@article{e9725c362fd4449ea1e25496a05dcf7b,
title = "A review of multimodal interaction technique in augmented reality environment",
abstract = "Augmented Reality (AR) has proposed several types of interaction techniques such as 3D interactions, natural interactions, tangible interactions, spatial awareness interactions and multimodal interactions. Usually, interaction technique in AR involve unimodal interaction technique that only allows user to interact with AR content by using one modality such as gesture, speech, click, etc. Meanwhile, the combination of more than one modality is called multimodal. Multimodal can contribute to human and computer interaction more efficient and will enhance better user experience. This is because, there are a lot of issues have been found when user use unimodal interaction technique in AR environment such as fat fingers. Recent research has shown that multimodal interface (MMI) has been explored in AR environment and has been applied in various domain. This paper presents an empirical study of some of the key aspects and issues in multimodal interaction augmented reality, touching on the interaction technique and system framework. We reviewed the question of what are the interaction techniques that have been used to perform a multimodal interaction in AR environment and what are the integrated components applied in multimodal interaction AR frameworks. These two questions were used to be analysed in order to find the trends in multimodal field as a main contribution of this paper. We found that gesture, speech and touch are frequently used to manipulate virtual object. Most of the integrated component in MMI AR framework discussed only on the concept of the framework components or the information centred design between the components. Finally, we conclude this paper by providing ideas for future work involving this field.",
keywords = "Augmented reality, Multimodal interaction technique, Review",
author = "Nizam, {Siti Soleha Muhammad} and Abidin, {Rimaniza Zainal} and Hashim, {Nurhazarifah Che} and {Meng Chun}, Lam and Haslina Arshad and {Abd Majid}, {Nazatul Aini}",
year = "2018",
month = "1",
day = "1",
language = "English",
volume = "8",
pages = "1460--1469",
journal = "International Journal on Advanced Science, Engineering and Information Technology",
issn = "2088-5334",
publisher = "INSIGHT - Indonesian Society for Knowledge and Human Development",
number = "4-2",

}

TY - JOUR

T1 - A review of multimodal interaction technique in augmented reality environment

AU - Nizam, Siti Soleha Muhammad

AU - Abidin, Rimaniza Zainal

AU - Hashim, Nurhazarifah Che

AU - Meng Chun, Lam

AU - Arshad, Haslina

AU - Abd Majid, Nazatul Aini

PY - 2018/1/1

Y1 - 2018/1/1

N2 - Augmented Reality (AR) has proposed several types of interaction techniques such as 3D interactions, natural interactions, tangible interactions, spatial awareness interactions and multimodal interactions. Usually, interaction technique in AR involve unimodal interaction technique that only allows user to interact with AR content by using one modality such as gesture, speech, click, etc. Meanwhile, the combination of more than one modality is called multimodal. Multimodal can contribute to human and computer interaction more efficient and will enhance better user experience. This is because, there are a lot of issues have been found when user use unimodal interaction technique in AR environment such as fat fingers. Recent research has shown that multimodal interface (MMI) has been explored in AR environment and has been applied in various domain. This paper presents an empirical study of some of the key aspects and issues in multimodal interaction augmented reality, touching on the interaction technique and system framework. We reviewed the question of what are the interaction techniques that have been used to perform a multimodal interaction in AR environment and what are the integrated components applied in multimodal interaction AR frameworks. These two questions were used to be analysed in order to find the trends in multimodal field as a main contribution of this paper. We found that gesture, speech and touch are frequently used to manipulate virtual object. Most of the integrated component in MMI AR framework discussed only on the concept of the framework components or the information centred design between the components. Finally, we conclude this paper by providing ideas for future work involving this field.

AB - Augmented Reality (AR) has proposed several types of interaction techniques such as 3D interactions, natural interactions, tangible interactions, spatial awareness interactions and multimodal interactions. Usually, interaction technique in AR involve unimodal interaction technique that only allows user to interact with AR content by using one modality such as gesture, speech, click, etc. Meanwhile, the combination of more than one modality is called multimodal. Multimodal can contribute to human and computer interaction more efficient and will enhance better user experience. This is because, there are a lot of issues have been found when user use unimodal interaction technique in AR environment such as fat fingers. Recent research has shown that multimodal interface (MMI) has been explored in AR environment and has been applied in various domain. This paper presents an empirical study of some of the key aspects and issues in multimodal interaction augmented reality, touching on the interaction technique and system framework. We reviewed the question of what are the interaction techniques that have been used to perform a multimodal interaction in AR environment and what are the integrated components applied in multimodal interaction AR frameworks. These two questions were used to be analysed in order to find the trends in multimodal field as a main contribution of this paper. We found that gesture, speech and touch are frequently used to manipulate virtual object. Most of the integrated component in MMI AR framework discussed only on the concept of the framework components or the information centred design between the components. Finally, we conclude this paper by providing ideas for future work involving this field.

KW - Augmented reality

KW - Multimodal interaction technique

KW - Review

UR - http://www.scopus.com/inward/record.url?scp=85055336304&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85055336304&partnerID=8YFLogxK

M3 - Article

AN - SCOPUS:85055336304

VL - 8

SP - 1460

EP - 1469

JO - International Journal on Advanced Science, Engineering and Information Technology

JF - International Journal on Advanced Science, Engineering and Information Technology

SN - 2088-5334

IS - 4-2

ER -