Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Sketch to 3D Model using Generative Query Networks
KTH, School of Electrical Engineering and Computer Science (EECS).
2019 (English)Independent thesis Advanced level (degree of Master (Two Years)), 20 credits / 30 HE creditsStudent thesis
Abstract [en]

For digital artists and animators, translating an idea from a rough sketch to a 3D model is a time consuming process requiring a plethora of different software. In this work, a Generative Model which can directly generate images of 3D models from arbitrary view points by observing sketched 2D images is presented. The model is based on Generative Query Networks and two different generative models were tested for generating new images, the first a Variational Auto Encoder and the second a Generative Adversarial Network. The model learns to produce new images from any queried view point allowing it to perform so called mental rotation of an object as if a 3D model had been generated. A paired dataset containing images of 3D models, the view point from where each image is captured and corresponding sketch versions was created in order to train the model. It was found that the Variational Auto Encoder could create plausible images from as little as a single sketch while the Generative Adversarial Network failed to correctly condition on the given sketches.

Abstract [sv]

För digitala artister och animatörer är processen att gå ifrån en idé i form av en sketch till en färdig 3D-modell tidskrävande och sträcker sig över en mängd olika mjukvaror. Detta arbete presenterar en Generativ Modell som direkt kan generera bilder av en 3D-modell ifrån sketchade bilder i 2D. Modellen är baserad på Generative Query Networks och två olika Generativa Modeller testades för att generera nya bilder, den första en Variational Auto Encoder och den andra en Generative Adversarial Network. Modellen lär sig att skapa nya bilder ifrån godtyckliga synvinklar vilket tillåter den att utföra så kallad mental rotation av ett objekt på samma sätt som om en 3D-modell hade genererats. För att kunna träna modellen skapades ett dataset där bilder sparades både i ursprungs- samt i sketchform tillsammans med synvinklarna där bilderna tagits ifrån. Modellen som använde sig av en Variational Auto Encoder visade sig kunna generera trovärdiga bilder efter att endast ha observerat en sketch medan modellen som använde ett Generative Adversarial Network misslyckades med att betinga de genererade bilderna på de sketcher den observerat.

Place, publisher, year, edition, pages
2019. , p. 63
Series
TRITA-EECS-EX ; 2019:89
National Category
Computer and Information Sciences
Identifiers
URN: urn:nbn:se:kth:diva-251507OAI: oai:DiVA.org:kth-251507DiVA, id: diva2:1315752
External cooperation
EA SEED
Supervisors
Examiners
Available from: 2019-05-20 Created: 2019-05-14 Last updated: 2019-05-20Bibliographically approved

Open Access in DiVA

fulltext(2529 kB)145 downloads
File information
File name FULLTEXT02.pdfFile size 2529 kBChecksum SHA-512
a66d0350012fdd9d3e0dd1fd9d90e55c86e8cb01d01986eb8e4391939f3aebb6bac5fe7dc5b884fbcb2ed43406d03b3e0a7de23f873e1701fed26ba4b1359166
Type fulltextMimetype application/pdf

By organisation
School of Electrical Engineering and Computer Science (EECS)
Computer and Information Sciences

Search outside of DiVA

GoogleGoogle Scholar
Total: 145 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 277 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf