Change search
ReferencesLink to record
Permanent link

Direct link
Information Theory for Analyzing Neural Networks
Norwegian University of Science and Technology, Faculty of Information Technology, Mathematics and Electrical Engineering, Department of Computer and Information Science.
2014 (English)MasteroppgaveStudent thesis
Abstract [en]

The goal of this thesis was to investigate how information theory could be used to analyze artificial neural networks. For this purpose, two problems, a classification problem and a controller problem were considered. The classification problem was solved with a feedforward neural network trained with backpropagation, the controller problem was solved with a continuous-time recurrent neural network optimized with evolution. Results from the classification problem shows that mutual information might indicate how much a particular neuron contributes to the classification. Tracking these neurons' mutual information during training might serve as an indicator of their progression, including neurons in the hidden layers. Results from the controller problem showed that time-delayed mutual information between a neuron and an environment variable might indicate what variable each neuron is estimating, and tracking this during evolution might tell us when this particular neuron started taking this role. Furthermore, unrolled transfer entropy appears to be a good measure for how neurons affect each other during simulation.

Place, publisher, year, edition, pages
Institutt for datateknikk og informasjonsvitenskap , 2014. , 76 p.
URN: urn:nbn:no:ntnu:diva-26773Local ID: ntnudaim:11773OAI: diva2:751083
Available from: 2014-09-30 Created: 2014-09-30 Last updated: 2014-09-30Bibliographically approved

Open Access in DiVA

fulltext(1269 kB)767 downloads
File information
File name FULLTEXT01.pdfFile size 1269 kBChecksum SHA-512
Type fulltextMimetype application/pdf
cover(184 kB)5 downloads
File information
File name COVER01.pdfFile size 184 kBChecksum SHA-512
Type coverMimetype application/pdf

By organisation
Department of Computer and Information Science

Search outside of DiVA

GoogleGoogle Scholar
Total: 767 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

Total: 200 hits
ReferencesLink to record
Permanent link

Direct link