Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Stochastic Gradient Descent inom Maskininlärning
KTH, School of Engineering Sciences (SCI).
KTH, School of Engineering Sciences (SCI).
2019 (Swedish)Independent thesis Basic level (degree of Bachelor), 10 credits / 15 HE creditsStudent thesisAlternative title
Stochastic Gradient Descent in Machine Learning (English)
Abstract [sv]

Vissa problem som för människor är enkla att lösa, till exempel: att känna igen siffror och sagda ord, är svårt att implementera i datorprogram. Till exempel, den mänskliga intuitionen att känna igen siffran åtta ’’\textit{8}’’ är att notera två slingor ovanpå varandra, detta visar sig vara svårt att representera som en algoritm. Med maskininlärning är det möjligt att angripa problemet på ett nytt, enklare, sätt där datorprogrammet lärs att känna igen utformningar som datorprogrammet drar slutsatser från. I denna kandidatuppsats implementeras ett sifferigenkänningsprogram och parametrarna i ’’stochastic gradient descent’’ analyseras i deras påverkan av programmets beräkningshastighet och träffsäkerhet. Dessa parametrar är ’’learning rate’’ $\Delta t$ och ’’batch size’’ $N$. Det implementerade programmet för sifferigenkänning hade en träffsäkerhet på omkring 95 \% när det testades och tiden per iteration var konstant under träningen av programmet, samtidigt som den ökade linjärt med ökad batch size. Låga learning rates resulterade i låg men stadig konvergens medans större resulterade i snabbare men mer instabil konvergens. Större batch sizes förbättrade konvergensen men på bekostnad av längre beräkningstid.

Abstract [en]

Some tasks, like recognizing digits and spoken words, are simple for humans to complete yet hard to solve for computer programs. For instance the human intuition behind recognizing the number eight, ''\textit{8}'', is to identify two loops on top of each other and it turns out this is not easy to represent as an algorithm. With machine learning one can tackle the problem in a new, easier, way where the computer program learns to recognize patterns and make conclusions from them. In this bachelor thesis a digit recognizing program is implemented and the parameters of the stochastic gradient descent optimizing algorithm are analyzed based on how their effect on the computation speed and accuracy. These parameters being the learning rate $\Delta t$ and batch size $N$. The implemented digit recognizing program yielded an accuracy of around $95$ \% when tested and the time per iteration stayed constant during the training session and increased linearly with batch size. Low learning rates yielded a slower rate of convergence while larger ones yielded faster but more unstable convergence. Larger batch sizes also improved the convergence but at the cost of more computational power.

Place, publisher, year, edition, pages
2019.
Series
TRITA-SCI-GRU ; 2019:178
National Category
Engineering and Technology
Identifiers
URN: urn:nbn:se:kth:diva-254799OAI: oai:DiVA.org:kth-254799DiVA, id: diva2:1335380
Supervisors
Examiners
Available from: 2019-07-05 Created: 2019-07-05 Last updated: 2019-07-05Bibliographically approved

Open Access in DiVA

fulltext(2746 kB)16 downloads
File information
File name FULLTEXT01.pdfFile size 2746 kBChecksum SHA-512
99cf081027bc9b0cfb2d1bd97eaef8143c5f27276cc3df76f766723cd99e51f2c4735e57d57d1a22af6ba9377faae9eb1a5b3b2bbb8612deb325736585648d33
Type fulltextMimetype application/pdf

By organisation
School of Engineering Sciences (SCI)
Engineering and Technology

Search outside of DiVA

GoogleGoogle Scholar
Total: 16 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 37 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf