A combination of static and stroke gesture with speech for multimodal interaction in a virtual environment

Lam Meng Chun, Haslina Arshad, Thammathip Piumsomboon, Mark Billinghurst

Research output: Chapter in Book/Report/Conference proceedingConference contribution

2 Citations (Scopus)

Abstract

The computers nowadays are powerful enough to make the virtual reality technology more practical. Virtual reality is expected to have a bright future due to the rise of several virtual reality head mounted device such as low cost Google Cardboard, Samsung Gear VR, Oculus, HTC Vive and many others. Since virtual reality had been growing fast in this few years thus it is necessary to explore new approaches for user to interact with the virtual reality more naturally. It is an important research in HCI to make the interactions with computers to be as natural as the interaction between humans. One of the approaches is to allow user to interact with the virtual reality through gesture and speech input. This paper presents a virtual reality environment with multimodal interaction technique. It allows user to interact with the virtual reality system with the static and stroke hand gesture along with speech. This multimodal interaction technique is able to perform few functions such as select, move, scale, rotate, copy, mirror, delete, check the shape, check and change the colour of an object.

Original languageEnglish
Title of host publicationProceedings - 5th International Conference on Electrical Engineering and Informatics: Bridging the Knowledge between Academic, Industry, and Community, ICEEI 2015
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages59-64
Number of pages6
ISBN (Print)9781467373197
DOIs
Publication statusPublished - 10 Dec 2015
Event5th International Conference on Electrical Engineering and Informatics, ICEEI 2015 - Legian-Bali, Indonesia
Duration: 10 Aug 201511 Aug 2015

Other

Other5th International Conference on Electrical Engineering and Informatics, ICEEI 2015
CountryIndonesia
CityLegian-Bali
Period10/8/1511/8/15

Fingerprint

Virtual reality
Human computer interaction
Gears
Mirrors
Color
Costs

Keywords

  • Hand Gesture and Speech Recognition
  • Human Computer Interaction
  • Multimodal Interaction
  • Natural Interaction

ASJC Scopus subject areas

  • Computer Science Applications
  • Information Systems
  • Signal Processing
  • Electrical and Electronic Engineering

Cite this

Meng Chun, L., Arshad, H., Piumsomboon, T., & Billinghurst, M. (2015). A combination of static and stroke gesture with speech for multimodal interaction in a virtual environment. In Proceedings - 5th International Conference on Electrical Engineering and Informatics: Bridging the Knowledge between Academic, Industry, and Community, ICEEI 2015 (pp. 59-64). [7352470] Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/ICEEI.2015.7352470

A combination of static and stroke gesture with speech for multimodal interaction in a virtual environment. / Meng Chun, Lam; Arshad, Haslina; Piumsomboon, Thammathip; Billinghurst, Mark.

Proceedings - 5th International Conference on Electrical Engineering and Informatics: Bridging the Knowledge between Academic, Industry, and Community, ICEEI 2015. Institute of Electrical and Electronics Engineers Inc., 2015. p. 59-64 7352470.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Meng Chun, L, Arshad, H, Piumsomboon, T & Billinghurst, M 2015, A combination of static and stroke gesture with speech for multimodal interaction in a virtual environment. in Proceedings - 5th International Conference on Electrical Engineering and Informatics: Bridging the Knowledge between Academic, Industry, and Community, ICEEI 2015., 7352470, Institute of Electrical and Electronics Engineers Inc., pp. 59-64, 5th International Conference on Electrical Engineering and Informatics, ICEEI 2015, Legian-Bali, Indonesia, 10/8/15. https://doi.org/10.1109/ICEEI.2015.7352470
Meng Chun L, Arshad H, Piumsomboon T, Billinghurst M. A combination of static and stroke gesture with speech for multimodal interaction in a virtual environment. In Proceedings - 5th International Conference on Electrical Engineering and Informatics: Bridging the Knowledge between Academic, Industry, and Community, ICEEI 2015. Institute of Electrical and Electronics Engineers Inc. 2015. p. 59-64. 7352470 https://doi.org/10.1109/ICEEI.2015.7352470
Meng Chun, Lam ; Arshad, Haslina ; Piumsomboon, Thammathip ; Billinghurst, Mark. / A combination of static and stroke gesture with speech for multimodal interaction in a virtual environment. Proceedings - 5th International Conference on Electrical Engineering and Informatics: Bridging the Knowledge between Academic, Industry, and Community, ICEEI 2015. Institute of Electrical and Electronics Engineers Inc., 2015. pp. 59-64
@inproceedings{8f378c9a344c4d5db5eabc3f17682fbe,
title = "A combination of static and stroke gesture with speech for multimodal interaction in a virtual environment",
abstract = "The computers nowadays are powerful enough to make the virtual reality technology more practical. Virtual reality is expected to have a bright future due to the rise of several virtual reality head mounted device such as low cost Google Cardboard, Samsung Gear VR, Oculus, HTC Vive and many others. Since virtual reality had been growing fast in this few years thus it is necessary to explore new approaches for user to interact with the virtual reality more naturally. It is an important research in HCI to make the interactions with computers to be as natural as the interaction between humans. One of the approaches is to allow user to interact with the virtual reality through gesture and speech input. This paper presents a virtual reality environment with multimodal interaction technique. It allows user to interact with the virtual reality system with the static and stroke hand gesture along with speech. This multimodal interaction technique is able to perform few functions such as select, move, scale, rotate, copy, mirror, delete, check the shape, check and change the colour of an object.",
keywords = "Hand Gesture and Speech Recognition, Human Computer Interaction, Multimodal Interaction, Natural Interaction",
author = "{Meng Chun}, Lam and Haslina Arshad and Thammathip Piumsomboon and Mark Billinghurst",
year = "2015",
month = "12",
day = "10",
doi = "10.1109/ICEEI.2015.7352470",
language = "English",
isbn = "9781467373197",
pages = "59--64",
booktitle = "Proceedings - 5th International Conference on Electrical Engineering and Informatics: Bridging the Knowledge between Academic, Industry, and Community, ICEEI 2015",
publisher = "Institute of Electrical and Electronics Engineers Inc.",

}

TY - GEN

T1 - A combination of static and stroke gesture with speech for multimodal interaction in a virtual environment

AU - Meng Chun, Lam

AU - Arshad, Haslina

AU - Piumsomboon, Thammathip

AU - Billinghurst, Mark

PY - 2015/12/10

Y1 - 2015/12/10

N2 - The computers nowadays are powerful enough to make the virtual reality technology more practical. Virtual reality is expected to have a bright future due to the rise of several virtual reality head mounted device such as low cost Google Cardboard, Samsung Gear VR, Oculus, HTC Vive and many others. Since virtual reality had been growing fast in this few years thus it is necessary to explore new approaches for user to interact with the virtual reality more naturally. It is an important research in HCI to make the interactions with computers to be as natural as the interaction between humans. One of the approaches is to allow user to interact with the virtual reality through gesture and speech input. This paper presents a virtual reality environment with multimodal interaction technique. It allows user to interact with the virtual reality system with the static and stroke hand gesture along with speech. This multimodal interaction technique is able to perform few functions such as select, move, scale, rotate, copy, mirror, delete, check the shape, check and change the colour of an object.

AB - The computers nowadays are powerful enough to make the virtual reality technology more practical. Virtual reality is expected to have a bright future due to the rise of several virtual reality head mounted device such as low cost Google Cardboard, Samsung Gear VR, Oculus, HTC Vive and many others. Since virtual reality had been growing fast in this few years thus it is necessary to explore new approaches for user to interact with the virtual reality more naturally. It is an important research in HCI to make the interactions with computers to be as natural as the interaction between humans. One of the approaches is to allow user to interact with the virtual reality through gesture and speech input. This paper presents a virtual reality environment with multimodal interaction technique. It allows user to interact with the virtual reality system with the static and stroke hand gesture along with speech. This multimodal interaction technique is able to perform few functions such as select, move, scale, rotate, copy, mirror, delete, check the shape, check and change the colour of an object.

KW - Hand Gesture and Speech Recognition

KW - Human Computer Interaction

KW - Multimodal Interaction

KW - Natural Interaction

UR - http://www.scopus.com/inward/record.url?scp=84961710185&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84961710185&partnerID=8YFLogxK

U2 - 10.1109/ICEEI.2015.7352470

DO - 10.1109/ICEEI.2015.7352470

M3 - Conference contribution

AN - SCOPUS:84961710185

SN - 9781467373197

SP - 59

EP - 64

BT - Proceedings - 5th International Conference on Electrical Engineering and Informatics: Bridging the Knowledge between Academic, Industry, and Community, ICEEI 2015

PB - Institute of Electrical and Electronics Engineers Inc.

ER -