Study of cost functions in three term backpropagation for classification problems

Siti Mariyam Shamsuddin, Razana Alwee, Puspadevi Kuppusamy, Maslina Darus

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

Three Term Backpropagation(BP) Network as proposed by Zweiri in 2003 has outperformed standard Two Term Backpropagation. However, further studies on Three Term Backpropagation in 2007 indicated that this network only surpassed standard BP for small scale datasets but not for medium and large scale datasets. It has also been observed that by using Mean Square Error (MSE) as a cost function in Three Term BP has some drawbacks, and these include incorrect saturation and tend to trap in local minima, resulting in slow convergence and poor performance. In this study, thorough experiments on implementing various cost functions are executed to probe the effectiveness of Three Term BP network. The cost functions under investigations include Mean Square Error (MSE), Bernoulli function, Modified cost function and Improved cost function. The results reveal that MSE is not an ideal cost function to be used for Three Term BP. Hence, alternative cost functions need to be considered when using BP network for classification problems.

Original languageEnglish
Title of host publication2009 World Congress on Nature and Biologically Inspired Computing, NABIC 2009 - Proceedings
Pages564-570
Number of pages7
DOIs
Publication statusPublished - 2009
Event2009 World Congress on Nature and Biologically Inspired Computing, NABIC 2009 - Coimbatore
Duration: 9 Dec 200911 Dec 2009

Other

Other2009 World Congress on Nature and Biologically Inspired Computing, NABIC 2009
CityCoimbatore
Period9/12/0911/12/09

Fingerprint

Backpropagation
Cost functions
Mean square error

Keywords

  • Classification
  • Cost function
  • Three term BP

ASJC Scopus subject areas

  • Artificial Intelligence
  • Computer Science Applications
  • Software

Cite this

Shamsuddin, S. M., Alwee, R., Kuppusamy, P., & Darus, M. (2009). Study of cost functions in three term backpropagation for classification problems. In 2009 World Congress on Nature and Biologically Inspired Computing, NABIC 2009 - Proceedings (pp. 564-570). [5393407] https://doi.org/10.1109/NABIC.2009.5393407

Study of cost functions in three term backpropagation for classification problems. / Shamsuddin, Siti Mariyam; Alwee, Razana; Kuppusamy, Puspadevi; Darus, Maslina.

2009 World Congress on Nature and Biologically Inspired Computing, NABIC 2009 - Proceedings. 2009. p. 564-570 5393407.

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Shamsuddin, SM, Alwee, R, Kuppusamy, P & Darus, M 2009, Study of cost functions in three term backpropagation for classification problems. in 2009 World Congress on Nature and Biologically Inspired Computing, NABIC 2009 - Proceedings., 5393407, pp. 564-570, 2009 World Congress on Nature and Biologically Inspired Computing, NABIC 2009, Coimbatore, 9/12/09. https://doi.org/10.1109/NABIC.2009.5393407
Shamsuddin SM, Alwee R, Kuppusamy P, Darus M. Study of cost functions in three term backpropagation for classification problems. In 2009 World Congress on Nature and Biologically Inspired Computing, NABIC 2009 - Proceedings. 2009. p. 564-570. 5393407 https://doi.org/10.1109/NABIC.2009.5393407
Shamsuddin, Siti Mariyam ; Alwee, Razana ; Kuppusamy, Puspadevi ; Darus, Maslina. / Study of cost functions in three term backpropagation for classification problems. 2009 World Congress on Nature and Biologically Inspired Computing, NABIC 2009 - Proceedings. 2009. pp. 564-570
@inproceedings{c2e4133eba0c4924bfb8b24ace3e9b7f,
title = "Study of cost functions in three term backpropagation for classification problems",
abstract = "Three Term Backpropagation(BP) Network as proposed by Zweiri in 2003 has outperformed standard Two Term Backpropagation. However, further studies on Three Term Backpropagation in 2007 indicated that this network only surpassed standard BP for small scale datasets but not for medium and large scale datasets. It has also been observed that by using Mean Square Error (MSE) as a cost function in Three Term BP has some drawbacks, and these include incorrect saturation and tend to trap in local minima, resulting in slow convergence and poor performance. In this study, thorough experiments on implementing various cost functions are executed to probe the effectiveness of Three Term BP network. The cost functions under investigations include Mean Square Error (MSE), Bernoulli function, Modified cost function and Improved cost function. The results reveal that MSE is not an ideal cost function to be used for Three Term BP. Hence, alternative cost functions need to be considered when using BP network for classification problems.",
keywords = "Classification, Cost function, Three term BP",
author = "Shamsuddin, {Siti Mariyam} and Razana Alwee and Puspadevi Kuppusamy and Maslina Darus",
year = "2009",
doi = "10.1109/NABIC.2009.5393407",
language = "English",
isbn = "9781424456123",
pages = "564--570",
booktitle = "2009 World Congress on Nature and Biologically Inspired Computing, NABIC 2009 - Proceedings",

}

TY - GEN

T1 - Study of cost functions in three term backpropagation for classification problems

AU - Shamsuddin, Siti Mariyam

AU - Alwee, Razana

AU - Kuppusamy, Puspadevi

AU - Darus, Maslina

PY - 2009

Y1 - 2009

N2 - Three Term Backpropagation(BP) Network as proposed by Zweiri in 2003 has outperformed standard Two Term Backpropagation. However, further studies on Three Term Backpropagation in 2007 indicated that this network only surpassed standard BP for small scale datasets but not for medium and large scale datasets. It has also been observed that by using Mean Square Error (MSE) as a cost function in Three Term BP has some drawbacks, and these include incorrect saturation and tend to trap in local minima, resulting in slow convergence and poor performance. In this study, thorough experiments on implementing various cost functions are executed to probe the effectiveness of Three Term BP network. The cost functions under investigations include Mean Square Error (MSE), Bernoulli function, Modified cost function and Improved cost function. The results reveal that MSE is not an ideal cost function to be used for Three Term BP. Hence, alternative cost functions need to be considered when using BP network for classification problems.

AB - Three Term Backpropagation(BP) Network as proposed by Zweiri in 2003 has outperformed standard Two Term Backpropagation. However, further studies on Three Term Backpropagation in 2007 indicated that this network only surpassed standard BP for small scale datasets but not for medium and large scale datasets. It has also been observed that by using Mean Square Error (MSE) as a cost function in Three Term BP has some drawbacks, and these include incorrect saturation and tend to trap in local minima, resulting in slow convergence and poor performance. In this study, thorough experiments on implementing various cost functions are executed to probe the effectiveness of Three Term BP network. The cost functions under investigations include Mean Square Error (MSE), Bernoulli function, Modified cost function and Improved cost function. The results reveal that MSE is not an ideal cost function to be used for Three Term BP. Hence, alternative cost functions need to be considered when using BP network for classification problems.

KW - Classification

KW - Cost function

KW - Three term BP

UR - http://www.scopus.com/inward/record.url?scp=77949757628&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=77949757628&partnerID=8YFLogxK

U2 - 10.1109/NABIC.2009.5393407

DO - 10.1109/NABIC.2009.5393407

M3 - Conference contribution

SN - 9781424456123

SP - 564

EP - 570

BT - 2009 World Congress on Nature and Biologically Inspired Computing, NABIC 2009 - Proceedings

ER -