Change search
ReferencesLink to record
Permanent link

Direct link
Bidirectional Hierarchical Neural Networks: Hebbian Learning Improves Generalization
Linköping University, Department of Computer and Information Science. Linköping University, The Institute of Technology.
Linköping University, Department of Computer and Information Science, MDALAB - Human Computer Interfaces. Linköping University, The Institute of Technology.ORCID iD: 0000-0003-2801-7050
Linköping University, Department of Computer and Information Science. Linköping University, The Institute of Technology.ORCID iD: 0000-0002-3997-1088
2010 (English)In: Proceedings of the Fifth International Conference on Computer Vision Theory and Applications,  Volume 1, 2010, 105-111 p.Conference paper (Other academic)
Abstract [en]

Visual pattern recognition is a complex problem, and it has proven difficult to achieve satisfactorily instandard three-layer feed-forward artificial neural networks. For this reason, an increasing number ofresearchers are using networks whose architecture resembles the human visual system. These biologicallybasednetworks are bidirectionally connected, use receptive fields, and have a hierarchical structure, withthe input layer being the largest layer, and consecutive layers getting increasingly smaller. These networksare large and complex, and therefore run a risk of getting overfitted during learning, especially if smalltraining sets are used, and if the input patterns are noisy. Many data sets, such as, for example, handwrittencharacters, are intrinsically noisy. The problem of overfitting is aggravated by the tendency of error-drivenlearning in large networks to treat all variations in the noisy input as significant. However, there is one wayto balance off this tendency to overfit, and that is to use a mixture of learning algorithms. In this study, weran systematic tests on handwritten character recognition, where we compared generalization performanceusing a mixture of Hebbian learning and error-driven learning with generalization performance using pureerror-driven learning. Our results indicate that injecting even a small amount of Hebbian learning, 0.01 %,significantly improves the generalization performance of the network.

Place, publisher, year, edition, pages
2010. 105-111 p.
Keyword [en]
generalization, image processing, bidirectional hierarchical neural networks, Hebbian learning, feature extraction, object recognition
National Category
Engineering and Technology
URN: urn:nbn:se:liu:diva-77026DOI: 10.5220/0002835501050111ISBN: 978-989-674-028-3OAI: diva2:524466
Fifth International Conference on Computer Vision Theory and Applications (VISAPP'10), May 17-21, 2010, Angers, France
Available from: 2012-08-28 Created: 2012-05-02 Last updated: 2016-01-13Bibliographically approved
In thesis
1. An Informed System Development Approach to Tropical Cyclone Track and Intensity Forecasting
Open this publication in new window or tab >>An Informed System Development Approach to Tropical Cyclone Track and Intensity Forecasting
2016 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Introduction: Tropical Cyclones (TCs) inflict considerable damage to life and property every year. A major problem is that residents often hesitate to follow evacuation orders when the early warning messages are perceived as inaccurate or uninformative. The root problem is that providing accurate early forecasts can be difficult, especially in countries with less economic and technical means.

Aim: The aim of the thesis is to investigate how cyclone early warning systems can be technically improved. This means, first, identifying problems associated with the current cyclone early warning systems, and second, investigating if biologically based Artificial Neural Networks (ANNs) are feasible to solve some of the identified problems.

Method: First, for evaluating the efficiency of cyclone early warning systems, Bangladesh was selected as study area, where a questionnaire survey and an in-depth interview were administered. Second, a review of currently operational TC track forecasting techniques was conducted to gain a better understanding of various techniques’ prediction performance, data requirements, and computational resource requirements. Third, a technique using biologically based ANNs was developed to produce TC track and intensity forecasts. Systematic testing was used to find optimal values for simulation parameters, such as feature-detector receptive field size, the mixture of unsupervised and supervised learning, and learning rate schedule. Five types of 2D data were used for training. The networks were tested on two types of novel data, to assess their generalization performance.

Results: A major problem that is identified in the thesis is that the meteorologists at the Bangladesh Meteorological Department are currently not capable of providing accurate TC forecasts. This is an important contributing factor to residents’ reluctance to evacuate. To address this issue, an ANN-based TC track and intensity forecasting technique was developed that can produce early and accurate forecasts, uses freely available satellite images, and does not require extensive computational resources to run. Bidirectional connections, combined supervised and unsupervised learning, and a deep hierarchical structure assists the parallel extraction of useful features from five types of 2D data. The trained networks were tested on two types of novel data: First, tests were performed with novel data covering the end of the lifecycle of trained cyclones; for these test data, the forecasts produced by the networks were correct in 91-100% of the cases. Second, the networks were tested with data of a novel TC; in this case, the networks performed with between 30% and 45% accuracy (for intensity forecasts).

Conclusions: The ANN technique developed in this thesis could, with further extensions and up-scaling, using additional types of input images of a greater number of TCs, improve the efficiency of cyclone early warning systems in countries with less economic and technical means. The thesis work also creates opportunities for further research, where biologically based ANNs can be employed for general-purpose weather forecasting, as well as for forecasting other severe weather phenomena, such as thunderstorms.

Place, publisher, year, edition, pages
Linköping, Sweden: Linköping University Electronic Press, 2016. 160 p.
Linköping Studies in Science and Technology. Dissertations, ISSN 0345-7524 ; 1734
Biologically based artificial neural networks; bi-directionally connected networks; temporal sequence learning; accurate tropical cyclone forecasting; informative warning message;
National Category
Other Computer and Information Science
urn:nbn:se:liu:diva-123198 (URN)10.3384/diss.diva-123198 (DOI)978-91-7685-854-7 (Print) (ISBN)
Public defence
2016-02-17, Alan Turing, Hus E, Campus Valla, Linköping, 13:15 (English)
Available from: 2016-01-13 Created: 2015-12-06 Last updated: 2016-01-28Bibliographically approved

Open Access in DiVA

fulltext(878 kB)394 downloads
File information
File name FULLTEXT01.pdfFile size 878 kBChecksum SHA-512
Type fulltextMimetype application/pdf

Other links

Publisher's full text

Search in DiVA

By author/editor
Saifullah, MohammadKovordanyi, RitaRoy, Chandan
By organisation
Department of Computer and Information ScienceThe Institute of TechnologyMDALAB - Human Computer Interfaces
Engineering and Technology

Search outside of DiVA

GoogleGoogle Scholar
Total: 394 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Altmetric score

Total: 579 hits
ReferencesLink to record
Permanent link

Direct link