Image understanding and the web: a state-of-the-art review

Wan Fariza Paizi@Fauzi, Mohammed Belkhatir

Research output: Contribution to journalArticle

5 Citations (Scopus)

Abstract

The contextual information of Web images is investigated to address the issue of characterizing their content with semantic descriptors and therefore bridge the semantic gap, i.e. the gap between their automated low-level representation in terms of colors, textures, shapes…and their semantic interpretation. Such characterization allows for understanding the image content and is crucial in important Web-based tasks such as image indexing and retrieval. Although we are highly motivated by the availability of rich knowledge on the Web and the relative success achieved by commercial search engines in automatically characterizing the image content using contextual information in Web pages, we are aware that the unpredictable quality of the contextual information is a major limiting factor. Among the reasons explaining the difficulty to leverage on the image contextual information, some problems are related to the characterization and extraction of this information. Indeed, the first issue is the lack of large-scale studies to highlight what is considered the relevant contextual information of an image, where it is located in a Web page and whether it is consistent across Web pages of different types, content layouts and domains. Also, the matter related to the extraction of this contextual information is topical as state-of-the-art automated extraction tools are unable to handle the heterogeneous Web. As far as the processing of the contextual information is concerned, problems linked to the syntactic and semantic characterizations of the textual components are important to address in order to tackle the semantic gap. Furthermore, questions pertaining to the organization of these textual components into coherent structures that are usable in image indexing and retrieval frameworks shall arise. To address these issues, we lay down the anatomy of a generic context-based Web image understanding framework and propose its stage-based decomposition, covering topical issues from information indexing and retrieval, image description models, natural language processing, webpage segmentation and automated information extraction. For each of the identified stages, we review state-of-the-art solutions in the literature categorized and analyzed under the light of the techniques used.

Original languageEnglish
Pages (from-to)271-306
Number of pages36
JournalJournal of Intelligent Information Systems
Volume43
Issue number2
DOIs
Publication statusPublished - 1 Jan 2014
Externally publishedYes

Fingerprint

Image understanding
Semantics
Websites
Image retrieval
Syntactics
Search engines
Processing
World Wide Web
Textures
Availability
Color
Decomposition

Keywords

  • Automatic information extraction
  • Image retrieval
  • Image understanding/description
  • Natural language processing
  • Web contextual information
  • Webpage segmentation

ASJC Scopus subject areas

  • Software
  • Information Systems
  • Hardware and Architecture
  • Computer Networks and Communications
  • Artificial Intelligence

Cite this

Image understanding and the web : a state-of-the-art review. / Paizi@Fauzi, Wan Fariza; Belkhatir, Mohammed.

In: Journal of Intelligent Information Systems, Vol. 43, No. 2, 01.01.2014, p. 271-306.

Research output: Contribution to journalArticle

@article{35edfc299c974321bb5c385e0e981a2b,
title = "Image understanding and the web: a state-of-the-art review",
abstract = "The contextual information of Web images is investigated to address the issue of characterizing their content with semantic descriptors and therefore bridge the semantic gap, i.e. the gap between their automated low-level representation in terms of colors, textures, shapes…and their semantic interpretation. Such characterization allows for understanding the image content and is crucial in important Web-based tasks such as image indexing and retrieval. Although we are highly motivated by the availability of rich knowledge on the Web and the relative success achieved by commercial search engines in automatically characterizing the image content using contextual information in Web pages, we are aware that the unpredictable quality of the contextual information is a major limiting factor. Among the reasons explaining the difficulty to leverage on the image contextual information, some problems are related to the characterization and extraction of this information. Indeed, the first issue is the lack of large-scale studies to highlight what is considered the relevant contextual information of an image, where it is located in a Web page and whether it is consistent across Web pages of different types, content layouts and domains. Also, the matter related to the extraction of this contextual information is topical as state-of-the-art automated extraction tools are unable to handle the heterogeneous Web. As far as the processing of the contextual information is concerned, problems linked to the syntactic and semantic characterizations of the textual components are important to address in order to tackle the semantic gap. Furthermore, questions pertaining to the organization of these textual components into coherent structures that are usable in image indexing and retrieval frameworks shall arise. To address these issues, we lay down the anatomy of a generic context-based Web image understanding framework and propose its stage-based decomposition, covering topical issues from information indexing and retrieval, image description models, natural language processing, webpage segmentation and automated information extraction. For each of the identified stages, we review state-of-the-art solutions in the literature categorized and analyzed under the light of the techniques used.",
keywords = "Automatic information extraction, Image retrieval, Image understanding/description, Natural language processing, Web contextual information, Webpage segmentation",
author = "Paizi@Fauzi, {Wan Fariza} and Mohammed Belkhatir",
year = "2014",
month = "1",
day = "1",
doi = "10.1007/s10844-014-0323-6",
language = "English",
volume = "43",
pages = "271--306",
journal = "Journal of Intelligent Information Systems",
issn = "0925-9902",
publisher = "Springer Netherlands",
number = "2",

}

TY - JOUR

T1 - Image understanding and the web

T2 - a state-of-the-art review

AU - Paizi@Fauzi, Wan Fariza

AU - Belkhatir, Mohammed

PY - 2014/1/1

Y1 - 2014/1/1

N2 - The contextual information of Web images is investigated to address the issue of characterizing their content with semantic descriptors and therefore bridge the semantic gap, i.e. the gap between their automated low-level representation in terms of colors, textures, shapes…and their semantic interpretation. Such characterization allows for understanding the image content and is crucial in important Web-based tasks such as image indexing and retrieval. Although we are highly motivated by the availability of rich knowledge on the Web and the relative success achieved by commercial search engines in automatically characterizing the image content using contextual information in Web pages, we are aware that the unpredictable quality of the contextual information is a major limiting factor. Among the reasons explaining the difficulty to leverage on the image contextual information, some problems are related to the characterization and extraction of this information. Indeed, the first issue is the lack of large-scale studies to highlight what is considered the relevant contextual information of an image, where it is located in a Web page and whether it is consistent across Web pages of different types, content layouts and domains. Also, the matter related to the extraction of this contextual information is topical as state-of-the-art automated extraction tools are unable to handle the heterogeneous Web. As far as the processing of the contextual information is concerned, problems linked to the syntactic and semantic characterizations of the textual components are important to address in order to tackle the semantic gap. Furthermore, questions pertaining to the organization of these textual components into coherent structures that are usable in image indexing and retrieval frameworks shall arise. To address these issues, we lay down the anatomy of a generic context-based Web image understanding framework and propose its stage-based decomposition, covering topical issues from information indexing and retrieval, image description models, natural language processing, webpage segmentation and automated information extraction. For each of the identified stages, we review state-of-the-art solutions in the literature categorized and analyzed under the light of the techniques used.

AB - The contextual information of Web images is investigated to address the issue of characterizing their content with semantic descriptors and therefore bridge the semantic gap, i.e. the gap between their automated low-level representation in terms of colors, textures, shapes…and their semantic interpretation. Such characterization allows for understanding the image content and is crucial in important Web-based tasks such as image indexing and retrieval. Although we are highly motivated by the availability of rich knowledge on the Web and the relative success achieved by commercial search engines in automatically characterizing the image content using contextual information in Web pages, we are aware that the unpredictable quality of the contextual information is a major limiting factor. Among the reasons explaining the difficulty to leverage on the image contextual information, some problems are related to the characterization and extraction of this information. Indeed, the first issue is the lack of large-scale studies to highlight what is considered the relevant contextual information of an image, where it is located in a Web page and whether it is consistent across Web pages of different types, content layouts and domains. Also, the matter related to the extraction of this contextual information is topical as state-of-the-art automated extraction tools are unable to handle the heterogeneous Web. As far as the processing of the contextual information is concerned, problems linked to the syntactic and semantic characterizations of the textual components are important to address in order to tackle the semantic gap. Furthermore, questions pertaining to the organization of these textual components into coherent structures that are usable in image indexing and retrieval frameworks shall arise. To address these issues, we lay down the anatomy of a generic context-based Web image understanding framework and propose its stage-based decomposition, covering topical issues from information indexing and retrieval, image description models, natural language processing, webpage segmentation and automated information extraction. For each of the identified stages, we review state-of-the-art solutions in the literature categorized and analyzed under the light of the techniques used.

KW - Automatic information extraction

KW - Image retrieval

KW - Image understanding/description

KW - Natural language processing

KW - Web contextual information

KW - Webpage segmentation

UR - http://www.scopus.com/inward/record.url?scp=84919462981&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84919462981&partnerID=8YFLogxK

U2 - 10.1007/s10844-014-0323-6

DO - 10.1007/s10844-014-0323-6

M3 - Article

AN - SCOPUS:84919462981

VL - 43

SP - 271

EP - 306

JO - Journal of Intelligent Information Systems

JF - Journal of Intelligent Information Systems

SN - 0925-9902

IS - 2

ER -