Digitala Vetenskapliga Arkivet

Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Papers into Podcasts: Evaluating NotebookLM’s Audio Overviews of Scientific Articles
KTH, School of Electrical Engineering and Computer Science (EECS).
KTH, School of Electrical Engineering and Computer Science (EECS).
2025 (English)Independent thesis Basic level (degree of Bachelor), 10 credits / 15 HE creditsStudent thesis
Abstract [en]

As the volume of academic research continues to grow, tools that help make this information more accessible are becoming increasingly important. Google’s NotebookLM, described as a “virtual research assistant,” recently introduced an Audio Overview feature that generates podcast-style summaries from user-uploaded content. This study explores how well these AI-generated podcasts represent academic research based on feedback from the original authors of the papers. Thirteen researchers evaluated podcasts generated from their own articles, rating them in terms of Comprehensiveness, Factual Integrity, and Utility. The results suggest that the podcasts were generally well-received, with a median score of 4 across all three dimensions. The conversational format was praised for being engaging and lowering the barrier to understanding, in line with previous research. However, concerns were raised about the informal tone, occasional factual inaccuracies, and the loss of certain nuances. These findings highlight both the potential and current limitations of AI-generated podcasts as tools for conveying scientific knowledge.

Abstract [sv]

I takt med en ökad mängd vetenskaplig forskning blir verktyg som gör information mer tillgänglig allt viktigare. Googles NotebookLM, beskrivet som en ”virtuell forskningsassistent”, introducerade nyligen en funktion kallad Audio Overview som genererar podcast-liknande sammanfattningar från innehåll som användaren laddat upp. Denna studie undersöker hur väl dessa AI-genererade podcasts representerar akademisk forskning, baserat på återkoppling från de ursprungliga författarna till artiklarna. Tretton forskare utvärderade podcasts som genererats från sina egna artiklar och betygsatte dem utifrån omfång, faktuell integritet och användbarhet. Resultaten tyder på att podcasterna generellt mottogs väl, med en medianpoäng på fyra inom alla tre dimensioner. Det konversationella formatet hyllades för att vara engagerande och för att sänka tröskeln för förståelse, i enlighet med tidigare forskning. Däremot uttrycktes oro över den informella tonen, vissa faktuella felaktigheter och bristande nyansering. Dessa resultat pekar både på potentialen och de nuvarande begränsningarna hos AI-genererade podcaster som verktyg för att kommunicera vetenskaplig kunskap.

Place, publisher, year, edition, pages
2025. , p. 7
Series
TRITA-EECS-EX ; 2025:168
Keywords [en]
NotebookLM; Audio Overview; AI-generated Podcasts; Scientific Papers; Conversational format; Comprehension; Factual Integrity; Utility; Multimodal Language Models
National Category
Computer and Information Sciences
Identifiers
URN: urn:nbn:se:kth:diva-367570OAI: oai:DiVA.org:kth-367570DiVA, id: diva2:1985496
Supervisors
Examiners
Available from: 2025-07-29 Created: 2025-07-24 Last updated: 2025-07-29Bibliographically approved

Open Access in DiVA

fulltext(630 kB)3788 downloads
File information
File name FULLTEXT01.pdfFile size 630 kBChecksum SHA-512
32fd3670198f29de21424168f2f748a732f41f160f7f7406a6eca4ae9ff62cee75945322ccb64c15af18e0fdabe718bb399f4c32a1caa4d1b24ac8a16e8fc751
Type fulltextMimetype application/pdf

By organisation
School of Electrical Engineering and Computer Science (EECS)
Computer and Information Sciences

Search outside of DiVA

GoogleGoogle Scholar
Total: 3792 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 1472 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf