Change search
ReferencesLink to record
Permanent link

Direct link
Learning Representations with a Dynamic Objective Sparse Autoencoder
Örebro University, School of Science and Technology, Örebro University, Sweden. (AASS)ORCID iD: 0000-0002-0579-7181
Örebro University, School of Science and Technology, Örebro University, Sweden. (AASS)ORCID iD: 0000-0002-3122-693X
2012 (English)Conference paper, Poster (Refereed)
Abstract [en]

The main objective of an auto-encoder is to reconstruct the input signals via a feature representation of latent variables. The number of latent variables defines the representational capacity limit of the model. For data sets where some or all signals contain noise there is an unnecessary amount of capacity spent on trying to reconstruct these signals. One solution is to increase the number of hidden units to increase the capacity so that there will be enough capacity to capture the valuable information. Another solution is to pre-process the signals or perform a manual signal selection. In this paper, we propose a method that will dynamically change the objective function depending on the current performance of the model. This is done by weighting the objective function individually for each input unit in order to guide the feature leaning and decrease the influence that problematic signals have on the learning of features. We evaluate our method on various multidimensional time-series data sets and handwritten digit recognition data sets and compare our results with a standard sparse auto-encoder.

Place, publisher, year, edition, pages
National Category
Computer Science
Research subject
Computer Science
URN: urn:nbn:se:oru:diva-34599OAI: diva2:710528
Neural Information Processing Systems
Available from: 2014-04-07 Created: 2014-04-07 Last updated: 2015-07-29Bibliographically approved

Open Access in DiVA

dosae(852 kB)181 downloads
File information
File name FULLTEXT01.pdfFile size 852 kBChecksum SHA-512
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Längkvist, MartinLoutfi, Amy
By organisation
School of Science and Technology, Örebro University, Sweden
Computer Science

Search outside of DiVA

GoogleGoogle Scholar
Total: 181 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Total: 401 hits
ReferencesLink to record
Permanent link

Direct link