Semantic text-based image retrieval with multi-modality ontology and DBpedia

M. K. Yanti Idaya Aspura, Shahrul Azman Mohd Noah

Research output: Contribution to journalArticle

4 Citations (Scopus)

Abstract

Purpose-The purpose of this study is to reduce the semantic distance by proposing a model for integrating indexes of textual and visual features via a multi-modality ontology and the use of DBpedia to improve the comprehensiveness of the ontology to enhance semantic retrieval. Design/methodology/approach-A multi-modality ontology-based approach was developed to integrate high-level concepts and low-level features, as well as integrate the ontology base with DBpedia to enrich the knowledge resource. A complete ontology model was also developed to represent the domain of sport news, with image caption keywords and image features. Precision and recall were used as metrics to evaluate the effectiveness of the multi-modality approach, and the outputs were compared with those obtained using a single-modality approach (i.e. textual ontology and visual ontology). Findings-The results based on ten queries show a superior performance of the multi-modality ontologybased IMR system integrated with DBpedia in retrieving correct images in accordance with user queries. The system achieved 100 per cent precision for six of the queries and greater than 80 per cent precision for the other four queries. The text-based system only achieved 100 per cent precision for one query; all other queries yielded precision rates less than 0.500. Research limitations/implications-This study only focused on BBC Sport News collection in the year 2009. Practical implications-The paper includes implications for the development of ontology-based retrieval on image collection. Originality value-This study demonstrates the strength of using a multi-modality ontology integrated with DBpedia for image retrieval to overcome the deficiencies of text-based and ontology-based systems. The result validates semantic text-based with multi-modality ontology and DBpedia as a useful model to reduce the semantic distance.

Original languageEnglish
Pages (from-to)1191-1214
Number of pages24
JournalElectronic Library
Volume35
Issue number6
DOIs
Publication statusPublished - 1 Jan 2017

Fingerprint

multimodality
Image retrieval
ontology
Ontology
Semantics
semantics
sports news
Sports
BBC
integrated system

Keywords

  • DBpedia
  • Image retrieval
  • Multi-modality ontology
  • Ontology
  • Semantic indexing
  • Text-based retrieval

ASJC Scopus subject areas

  • Computer Science Applications
  • Library and Information Sciences

Cite this

Semantic text-based image retrieval with multi-modality ontology and DBpedia. / Yanti Idaya Aspura, M. K.; Mohd Noah, Shahrul Azman.

In: Electronic Library, Vol. 35, No. 6, 01.01.2017, p. 1191-1214.

Research output: Contribution to journalArticle

@article{1f3c649e994c4f058038a627e588b002,
title = "Semantic text-based image retrieval with multi-modality ontology and DBpedia",
abstract = "Purpose-The purpose of this study is to reduce the semantic distance by proposing a model for integrating indexes of textual and visual features via a multi-modality ontology and the use of DBpedia to improve the comprehensiveness of the ontology to enhance semantic retrieval. Design/methodology/approach-A multi-modality ontology-based approach was developed to integrate high-level concepts and low-level features, as well as integrate the ontology base with DBpedia to enrich the knowledge resource. A complete ontology model was also developed to represent the domain of sport news, with image caption keywords and image features. Precision and recall were used as metrics to evaluate the effectiveness of the multi-modality approach, and the outputs were compared with those obtained using a single-modality approach (i.e. textual ontology and visual ontology). Findings-The results based on ten queries show a superior performance of the multi-modality ontologybased IMR system integrated with DBpedia in retrieving correct images in accordance with user queries. The system achieved 100 per cent precision for six of the queries and greater than 80 per cent precision for the other four queries. The text-based system only achieved 100 per cent precision for one query; all other queries yielded precision rates less than 0.500. Research limitations/implications-This study only focused on BBC Sport News collection in the year 2009. Practical implications-The paper includes implications for the development of ontology-based retrieval on image collection. Originality value-This study demonstrates the strength of using a multi-modality ontology integrated with DBpedia for image retrieval to overcome the deficiencies of text-based and ontology-based systems. The result validates semantic text-based with multi-modality ontology and DBpedia as a useful model to reduce the semantic distance.",
keywords = "DBpedia, Image retrieval, Multi-modality ontology, Ontology, Semantic indexing, Text-based retrieval",
author = "{Yanti Idaya Aspura}, {M. K.} and {Mohd Noah}, {Shahrul Azman}",
year = "2017",
month = "1",
day = "1",
doi = "10.1108/EL-06-2016-0127",
language = "English",
volume = "35",
pages = "1191--1214",
journal = "Electronic Library",
issn = "0264-0473",
publisher = "Emerald Group Publishing Ltd.",
number = "6",

}

TY - JOUR

T1 - Semantic text-based image retrieval with multi-modality ontology and DBpedia

AU - Yanti Idaya Aspura, M. K.

AU - Mohd Noah, Shahrul Azman

PY - 2017/1/1

Y1 - 2017/1/1

N2 - Purpose-The purpose of this study is to reduce the semantic distance by proposing a model for integrating indexes of textual and visual features via a multi-modality ontology and the use of DBpedia to improve the comprehensiveness of the ontology to enhance semantic retrieval. Design/methodology/approach-A multi-modality ontology-based approach was developed to integrate high-level concepts and low-level features, as well as integrate the ontology base with DBpedia to enrich the knowledge resource. A complete ontology model was also developed to represent the domain of sport news, with image caption keywords and image features. Precision and recall were used as metrics to evaluate the effectiveness of the multi-modality approach, and the outputs were compared with those obtained using a single-modality approach (i.e. textual ontology and visual ontology). Findings-The results based on ten queries show a superior performance of the multi-modality ontologybased IMR system integrated with DBpedia in retrieving correct images in accordance with user queries. The system achieved 100 per cent precision for six of the queries and greater than 80 per cent precision for the other four queries. The text-based system only achieved 100 per cent precision for one query; all other queries yielded precision rates less than 0.500. Research limitations/implications-This study only focused on BBC Sport News collection in the year 2009. Practical implications-The paper includes implications for the development of ontology-based retrieval on image collection. Originality value-This study demonstrates the strength of using a multi-modality ontology integrated with DBpedia for image retrieval to overcome the deficiencies of text-based and ontology-based systems. The result validates semantic text-based with multi-modality ontology and DBpedia as a useful model to reduce the semantic distance.

AB - Purpose-The purpose of this study is to reduce the semantic distance by proposing a model for integrating indexes of textual and visual features via a multi-modality ontology and the use of DBpedia to improve the comprehensiveness of the ontology to enhance semantic retrieval. Design/methodology/approach-A multi-modality ontology-based approach was developed to integrate high-level concepts and low-level features, as well as integrate the ontology base with DBpedia to enrich the knowledge resource. A complete ontology model was also developed to represent the domain of sport news, with image caption keywords and image features. Precision and recall were used as metrics to evaluate the effectiveness of the multi-modality approach, and the outputs were compared with those obtained using a single-modality approach (i.e. textual ontology and visual ontology). Findings-The results based on ten queries show a superior performance of the multi-modality ontologybased IMR system integrated with DBpedia in retrieving correct images in accordance with user queries. The system achieved 100 per cent precision for six of the queries and greater than 80 per cent precision for the other four queries. The text-based system only achieved 100 per cent precision for one query; all other queries yielded precision rates less than 0.500. Research limitations/implications-This study only focused on BBC Sport News collection in the year 2009. Practical implications-The paper includes implications for the development of ontology-based retrieval on image collection. Originality value-This study demonstrates the strength of using a multi-modality ontology integrated with DBpedia for image retrieval to overcome the deficiencies of text-based and ontology-based systems. The result validates semantic text-based with multi-modality ontology and DBpedia as a useful model to reduce the semantic distance.

KW - DBpedia

KW - Image retrieval

KW - Multi-modality ontology

KW - Ontology

KW - Semantic indexing

KW - Text-based retrieval

UR - http://www.scopus.com/inward/record.url?scp=85035348674&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85035348674&partnerID=8YFLogxK

U2 - 10.1108/EL-06-2016-0127

DO - 10.1108/EL-06-2016-0127

M3 - Article

VL - 35

SP - 1191

EP - 1214

JO - Electronic Library

JF - Electronic Library

SN - 0264-0473

IS - 6

ER -