Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
A Novel Transfer Function for Continuous Interpolation between Summation and Multiplication in Neural Networks
KTH, School of Computer Science and Communication (CSC).
2016 (English)Independent thesis Advanced level (professional degree), 20 credits / 30 HE creditsStudent thesisAlternative title
Kontinuerlig interpolering mellan addition och multiplikation med hjälp av en lämplig överföringsfunktion i artificiella neuronnät (Swedish)
Abstract [en]

In this work, we present the implementation and evaluation of a novel parameterizable transfer function for use in artificial neural networks. It allows the continuous change between summation and multiplication for the operation performed by a neuron. The transfer function is based on continuously differentiable fractional iterates of the exponential function and introduces an additional parameter per neuron and layer. This parameter can be determined along weights and biases during standard, gradient-based training. We evaluate the proposed transfer function within neural networks by comparing its performance to conventional transfer functions for various regression problems. Interpolation between summation and multiplication achieves comparable or even slightly better results, outperforming the latter on a task involving missing data and multiplicative interactions between inputs.

Abstract [sv]

I detta arbete presenterar vi implementationen och utvärderingen av en ny överföringsfunktion till användning i artificiella neuronnät. Den tillåter en kontinuerlig förändring mellan summering och multiplikation för operationen som utförs av en neuron. Överföringsfunktionen är baserad på kontinuerligt deriverbara bråkiterationer av exponentialfunktionen och introducerar ytterligare en parameter för varje neuron och lager. Denna parameter kan bestämmas längs vikter och avvikelser under vanlig lutningsbaserad träning. Vi utvärderar den föreslagna överföringsfunktionen inom neurala nätverk genom att jämföra dess prestanda med konventionella överföringsfunktioner för olika regressionsproblem. Interpolering mellan summering och multiplikation uppnår jämförbara eller något bättre resultat, till exempel för en uppgift som gäller saknade data och multiplikativ interaktion mellan indata.

Place, publisher, year, edition, pages
2016. , 60 p.
Keyword [en]
Machine Learning, Neural Networks, Transfer Functions
National Category
Computer Science
Identifiers
URN: urn:nbn:se:kth:diva-184087OAI: oai:DiVA.org:kth-184087DiVA: diva2:913911
External cooperation
Technische Universität München, Biomimetic Robotics and Machine Learning
Subject / course
Computer Science
Educational program
Master of Science in Engineering - Computer Science and Technology
Supervisors
Examiners
Available from: 2016-03-23 Created: 2016-03-23 Last updated: 2016-06-01Bibliographically approved

Open Access in DiVA

fulltext(27047 kB)138 downloads
File information
File name FULLTEXT01.pdfFile size 27047 kBChecksum SHA-512
bb114ddc895bcc23101d09fd867a3a54c373a1f9b24fd9cdf521d5374fab4e466360d7e281a11b4624546782209bae83608d84871f453e029842b5c8988b2999
Type fulltextMimetype application/pdf

Search in DiVA

By author/editor
Koepp, Wiebke
By organisation
School of Computer Science and Communication (CSC)
Computer Science

Search outside of DiVA

GoogleGoogle Scholar
Total: 138 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 528 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf