How to convert PDFs to audiobooks with machine learning

[@edwincaballero690]

Title says "How to convert PDFs to audiobooks with machine learning" it does not say "How to convert PDFs to audiobooks for the first time ever groundbreaking discovery with machine learning". Appreciate the work and explanation.

[@kayambilampulamasaka739]

A lot of people don't understand the sense of fulfilment and satisfaction that comes with building things yourself. Using existing apps is a good idea but building your own opens you up to a whole world of new possibilities. Thanks for sharing!

[@driziiD]

lol, i am literally working on this exact project, with the exact APIs, to solve the exact problem of listening to technical books

[@dalemarkowitz8021]

Lol! I'm curious to know how you handle the problem of deciding what to include in the audio!

[@DenITDao]

@@dalemarkowitz8021 that's autoML Tables task, I think

[@skaterope]

and i gonna build a startup from it hahahahaa

[@moustafarahal3396]

@@dalemarkowitz8021 guess someone needs to build a machine learning model/project to classify the parts of a page into different categories then we decide what part we read based on what "we know" it is

[@ahmadsum1]

@@moustafarahal3396 I am thinking like training a TensorFlow objection detection model with labeled pages to detect "important" portions and then feed the OCR with cropped portions... IDK if it would be a super noob approach... (since i am :3 )

[@snlagr]

Using that font size approach to remove garbage text was clever!

[@tusharmaurya1668]

But it will also remove the headings and sub-headings (b/c it is not in common font size)....

[@lvishal8556]

U can give the garbage text a different type font not size. Then u can just remove that type font. In that way no other issue will be seen I guess

[@ieornl]

@@tusharmaurya1668it would be looking at the actual font size of the garbage text, not just the font size of the body, that way, the headings won't be excluded.

[@archimidiz]

@@lvishal8556 isn't the problem how to identify the garbage text from the actual content? So, if we knew which one is the garbage text which we would have to know to give a different font as you said, we wouldn't have to go through all this trouble

[@lvishal8556]

@@archimidiz then what's ur approach ?

[@hereiamrakshith]

I thought there's an app ready to download here.

[@MridulSharma21]

No, she just converted the pdfs into mp3 files.

[@sreenivasulu7554]

Same here.😒

[@artthatsnice8817]

me too🤣

[@ihtesham_emon]

And my Moon+ reader can do this automatically without any machine learning 😄

[@RKEDITZ]

@@ihtesham_emon thanks

[@bryngerard4334]

I achieved this myself about 12 years ago but not using ML. I merely formatted the PDF files so that they were easily processed by the voice engine in Acrobat Reader. It worked fine although I didn't have the range of voices you have now. I got used to 'Marvin' (sounded a bit like Hawkings) and listened to many books. I did go so far as to create .wav files from them but the data was massive, although converting to MP3 would have been possible. Still a great deal of data though. I had all of the text in a Word file which I would print to a Postscript file and convert to PDF using Ghostscript. This approach would have been useful but I think using a voice engine is a lot more efficient.

[@9888622400]

I did same 5 years ago, using pyPDF and ESpeak!

[@asandax6]

Instead of MP3 convert to AAC

[@misutasa]

There's a lot of stuff I'm missing out just because I'm not resourceful enough huh

[@KS-wt6yg]

@@asandax6 how

[@UserUnknown07]

The art of Bodge ! I use @ Voice Aloud Reader. It's simple and has many voices too.

[@adiparzival4682]

Why not do it the easier way? 1. Convert pdf to epub file. 2. Upload the epub file to google playbooks and then download it on the same app. 3. Use the read aloud feature and enjoy.

[@alexjr977]

boi . Where is read aloud

[@adiparzival4682]

@@alexjr977 After steps1 and 2 . When u open the epub file and touch any page, u will find three vertical dots at the upper right corner. There u'll find the read aloud feature.

[@alexjr977]

@@adiparzival4682 bro i upload this on website not in app .But Thank you for sharing knowledge 😀🤗

[@adiparzival4682]

@@alexjr977 Also I was thinking why work so hard when things are easy? 😅 It's like u have a bike and u move around ur entire city to go to ur school which is just 15 minutes walk from ur home😂😂

[@alexjr977]

@@adiparzival4682 Lol

[@vidasale5633]

1 had no clue 2 watch the video 2nd time ive got 20% 3rd time ive got 40% and 4th time ive got 70% wow no more for today i need to clear my head and watch it tommorow .....thenk you very much ,,,

[@purpleghost3863]

This type of project puts a smile on my face. It’s fun, useful, and simple to understand and implement

[@nemila4904]

simple uh?

[@aravindkrishnasaravu2223]

Wow..This is legit the practical use of ML in real life which millions of people want to do..converting them into audiobooks is very helpful for college students and as a project to do it would be very helpful..Thank you, GCP...Now I know why I had subscribed to you guys..Thanks again :-)

[@dalemarkowitz8021]

Aw, glad to hear it! That's why we make 'em! :)

[@bhavyadhingra9463]

@@dalemarkowitz8021 is it thefree service

[@multiversodebolso]

I am completely in love with this Google tool! The videos on my channel will be made completely like this and even in two languages!

[@robmonkriedlinger]

This is so exciting - perfect for my Dyslexic daughter to use at school - thank you!!!

[@NathanSubramani]

You have shared the approach highlighting the ML functions & other GCP modules. Gives a clear pathway in to how problem was resolved. Good one.

[@bucketofbarnacles]

This is a fantastic idea for any PDF, not just research papers. I would take it two steps further. 1) Optionally converting it into a podcast format. As a podcast, I can listen to it in Airr, which allows you to bookmark audio clips and transcribe those clips. This is like taking notes while you listen, and is long-walk friendly. 2) Put this up on a web front-end for public use.

[@Beautiful1234]

How do you do this?

[@travv88]

that sounds like something I'd like. Often when I listen to things while walking I hear something I want to take note of and have thought of some solution like this.

[@sanapalalakshmipathi]

Don't do with Mathematics books.

[@sanapalalakshmipathi]

Hahahah

[@mannyatyourcorner4043]

Why ull read maths i thought u solve math

[@thatchipmunksings]

This is funny!

[@mellox5993]

Dead haha

[@АндрейПетров-э6в5я]

It's okay, if you use it for math books, it will also be incomprehensible like if you read it yourself. So the neural net does its job well.

[@adityashaw3198]

Add this tech in an e-book reader and launch something like kindle, Google

[@itsmerg5273]

yo HALO fan

[@sasimitra5871]

It's already been there for years Google Play Books and Kindle have a read aloud option. AND Microsoft Edge's pdf viewer has a read aloud option.

[@4TH4RV]

I am pretty sure Google books has that

[@johnjordan3552]

There are apps like that but they don't have advanced settings over the TTS, a shame really

[@flavioallemand5217]

Some PDFs are actually images so it’s necessary first to convert the file into intelligible text and afterwards audio files.

[@sadiqadeyanju]

I used balabolka to extract the text from the PDFs, used find and replace to remove the garbage and used Amazon Polly to save the audio to an S3. Downloaded 32 hours audio from academic papers for my MBA class.

[@hectorprx]

Hello, I was using Polly also but limited to a 100 k of characters which required batching the job. Did you figure out a work around ? Thanks

[@sadiqadeyanju]

@@hectorprx nope, let me know if you ever do

[@jofx4051]

Google: Upload video Also people: *There are many apps existing before you*

[@Mzulfreaky]

Well, GCP is for developers so the video uploaded is targeted for developers to build their own in Google's cloud

[@NoSTs123]

What a cool concept, Thanks for bringing it alive!

[@GeekMustHave]

You are a clever girl, said the Doctor. Love your delivery style and pace. The associated graphics are great for following the process. Keep broadcasting!!

[@carlitosviewsyou]

Wow before this video came out, I put together a rudimentary version of this with AWS Polly. Your font-size based approach to cutting out the junk is so good.

[@VilokReddy]

aws polly excellent tool

[@hectorprx]

Hello, I was using Polly also but limited to a 100 k of characters which required batching the job. Did you figure out a work around ? Thanks

[@VilokReddy]

@@hectorprx I used polly way before I didn't have much idea at now

[@carlitosviewsyou]

@@hectorprx My script it iterates through the entire text and at every word, it adds the word length to a characterCount variable. I use the characterCount variable to make sure it grabs 100k characters or less at a time and if a word pushes it past the limit, I drop the word and send the request, before starting the next chunk. There's no way to increase the limit as far as I can tell.

[@harshavardhannaik3325]

Most PDF readers already provide this feature of 'read out aloud'. I have been using this feature for more than 9 years. But, it reads the page numbers as well which isn't a big deal.

[@Blue-_-Jay]

I use this app called Read Aloud for PC and eReader Prestigo for Mobile. The voice sounds pretty natural to me - cutomizable. To add to that the text can be edited to your liking. As a lawyer, helps reading those lenghty judgements & books on the go. I am addicted to them now, can't imagine my life without them. No more scared of heavy texts and books - just need the pdf.

[@anispinner]

Finally... A channel without ads

[@RaviTrek]

This was what .i was working on few years ago but approach used in this was amazing..

[@javiermiranda-lozano9184]

Great Video! But I was wondering if you could do a step-by-step video because I am having trouble with the minute details?

[@rahmankhan566]

This is already available in adobe few months back.

[@tinselinkl]

I've been using text aloud for all my pdf. So far so good. It reads blogs too or whatever on the screen.

[@alanwheeler1]

The @Voice Aloud app for Android has been reading PDF research papers, articles, and entire books to my ears since 2015!

[@elvisjmartis]

Since I do not have so much expertise in coding, what I did was used ocr and aligned the content in a word file. Then used the microsoft immersive reader to read out the text for me.

[@heyrmi]

I am really amazed. I have a lot of books in pdf format. And have gone through the same phase.

[@TrendRain]

Yup. That's how it's done. Talk to someone who has done it, copy the code and tada you have built machine learning model that converts pdf to audio. I used to overthink about programing but this is a way to go.

[@anshulshrivastava7685]

I am trying with different approach, like removing the words (

[@ashfaqueahamedf2820]

Bring this soon i need to study with this for my exams😜

[@douglasharley2440]

smart shortcut using the font-style hack...waaaay better than several days of marking-up for training.

[@tanveersingh8290]

i made a similiar thing where i used my camera to detect text and it will speak it to me word by word while also highlighting the word. It can work in realtime where you use your camera and it can also work in normal mode.

[@parasdahiya1823]

Pocket app does this really well. I'm not sure if it handles PDFs but their own voice outputs are really good.

[@webdancer]

I've been using balabolka and the old ivona voices to achieve these seamlessly. Glad to see this on GCP. I'll definitely check this out. Thanks

[@marcus.the.younger]

yup the good old Balabolka

[@KanagawaMarcos]

The amount of views in this videos show how much good this problem is to teach as an example to google's cloud products.

[@deeliciousplum]

Wonderful and practical use of machine learning.

[@yogasounds1]

can you do a step by step walk through of this procedure please? thanks

[@rachman3339]

I did the exact same thing 40 years algo, but instead a computer I used a tape recorder and myself as text to speech. And as a player and headphones I used a walkman.

[@arpit.s]

How about building an AI that reads text and summaries it? What extra steps will be needed apart from the above mentioned ones?

[@kartikkalia01]

GPT3

[@madhu619]

Yeah exactly. GPT-3 is your answer.

[@drwriddhimanchattopadhyay2701]

There is an app. eReader Prestige. It reads pdfs like audiobooks. It has local languages too.

[@drwriddhimanchattopadhyay2701]

Removing garbage text was pretty clever though 😍

[@shashankjajoo]

Useful for reading anything which primarily requires your critical thinking, a good aid indeed.

[@DelfinoGarza77]

I like the "garbage text"....I gives me a clue that something is changing, or there is something I need to actually look at.

[@timeship]

Unfortunately, what concerns me is "New customers get $300 in free credits to spend on Text-to-Speech." So, what's the price after that? And why do we have to pay if Google Play offers us FREE audiobooks generated from clear-text ePub files that have no "garbage text" already? I don't understand this. Please explain.

[@manushitrivedi9182]

Hehheh I am doing this since last 6 months by just copy pasting the text to text to speech Google platform. Although this is the smart and easy way now ;) thanks for sharing...

[@iau]

_"So now that I'm in quarantine, I go out on walks without a mask"_

[@johnjordan3552]

The masks, especially when worn appropriately, greatly reduces the spread of water droplets that come out of our our mouths and nose. Which the germs that gets transmitted to one another reside in

[@JBB685]

Why would you walk outside with a mask on

[@jaydev7248]

Interesting & very useful concept for Google Android app making.

[@TIENTI0000]

that is just awesome! we are living in the future! finally

[@lovingputu5916]

You do know right that there is an app named Moonreader ?

[@zarnilynnkyaw]

Will there be an app for that, it would be really great to listen to paper while I'm on the treadmill.

[@sohamgurav7713]

The next thing we need to focus on is how to convert text to speech in natural voice, rather then that robotic

[@tomfreemanorourke1519]

I believe there is the voice 'sampler' method from the early 90's that has now come full circle and one's 'own' voice is convertible as are many celebrity voices.....Could you listen to yourself reading aloud to?....yourself?.......

[@williamseipp9691]

I was thinking about a related question: what are the audio qualities of the things we hear everyday? Specifically to your idea: what are the audio qualities of a given voice that make it sound robotic vs natural? If an AI model can Van-Goghify a photo with "style transfer" then surely another model can apply a realistic mask or filter of sorts to make a generic robotic voice sound human. Ugh. Actually that's a hard problem.

[@DhruvSingh-ru8nx]

Really interesting and informative. Looking forward to more videos based on projects on ML/AI

[@wisdomlounge4452]

In many cases you can't be sure which of the non-standard font size text is "junk text" without manually looking it over. I wouldn't simply and categorically think of all headings and subheadings as "junk text". These parts of the document weren't put there for nothing. Their purpose is to help organize the information being presented. Once you see or have the heading/subheading read, you'll know, especially if you got distracted, where in the document you're at. You can also know, through the headings what's about to be covered. I'm also not really bothered about page numbers read out loud. To me that's kind of like an "audible progress bar". I don't mind knowing how far along I am in the audiobook converted pdf, lol. If it's a matter of speed and efficiency, increase the read out loud speed to 1.5x - 2x or whatever you can handle.

[@bernard2735]

Perfect, just what I have been wanting! Thanks :-)

[@yash1152]

3:47 no comments on the "perfect" voice choice!!?

[@nathaliecedeno5251]

This is really insightful. Thanks for sharing!

[@victorboyi6383]

Wow!!! This is literally the starter that I needed for my project.

[@paulbernardo6626]

Pls launch such feature this year this is amazing

[@VishwaSeneviratne]

Hi Dale, I would like to see how to use GCP ML services to use time series data from things like metrics and predict future behavior patterns etc.

[@dalemarkowitz8021]

Hmm, good idea!

[@VishwaSeneviratne]

@@dalemarkowitz8021 I was clueless about how to do it. I'm interested on a using data set like InfluxDB or Prometheus. Would be grateful if you can initiate a video on that. By the way your videos are very good. Keep up the good work!!!

[@chrisford7351]

Good job and very interesting! Thanks

[@AdityaRajput-pj4yg]

I use an app named "eReader Prestigio" which uses google text to speech engine allows me to listen like an audiobook. it's works actually better than I expected. and also they have paid version which uses their own developed text to speech engine which works similarly as shown in the video.

[@pratyushpanda1985]

What is the total API and GCP cost for converting 1 pdf(10 pages) to an audiobook?

[@dalemarkowitz8021]

For me, it was free, because I stayed in the free tier! The cost varies depending on your usage. See the blog post for more details.

[@siliconslice]

@@dalemarkowitz8021 I used "IVONA Voices 2" with "Adobe Reader" 's 'Rread Out Loud' option but "IVONA" was for PC only and paid we need something same with Android handset but free. Can you please suggest us about that !?

[@siliconslice]

@@dalemarkowitz8021 And one friendly suggestion, you have 24 'subscribers' including me :D, you should upload.

[@Alkhinjari]

Please let the next project be: how to do a "literature review" using AI? this would be a revolutionary step in the research world.

[@daleonai7448]

This is an interesting idea!

[@zapy422]

I’m exploring this

[@shantanujain6376]

Copy other similar topic's literature review and change the words. No plagiarism, no time spent and ready made references lol

[@williamseipp9691]

literature review? you mean like sentiment analysis? Or summary? the latter is far more interesting, like an ELI5 app.

[@AdobadoFantastico]

@@zapy422 That's dope, best of luck!

[@Clire-h6r]

What about for those of us who can't buy the "bucket" on google? Thank you Dr Markowitz and chalom!

[@wanderinghooligan]

Necessity is the mother of inventions

[@ihtesham_emon]

It still sounds like machine voice. Instead try IVONA text to speech engine for TTS which has very robust natural voice in HD quality.

[@VaibhavShewale]

can we change the audio of the output?

[@Beautiful1234]

Thank you for the great video! how can we download the audio book after completing the process? appreciate if you can share

[@SGoswami41]

Have you ever heard about Narrator's Voice? It does the same thing, I'm using it since the last 3 years for making my college notes into audiobook, basically MP3 files.

[@johnjordan3552]

There are apps that let you use tts on PDFs, it's a whole lot of more practical than using an mp3 file except that I haven't found a software that gives you the option to make the tts ignore certain texts(like page numbers, repetitive sentences at the top-bottom of texts etc.). So other than converting the PDFs to mp3 it would be more beneficial and productive to add advanced settings such as making tts ignore "garbage texts". If you guys develop a PDF reader like that, I'd definitely use it instead of the current app I am using

[@Chloroplastism]

try moon readers built in tts - you can modify settings to ignore stuff.

[@1anre]

Been thinking of how to replicate Adobe Reader voice assistant in other documents which are open on other applications & now this GCP ML project popped up. Mega Cool

[@aspd00]

Fantastic Now give an option for procrastinators

[@neerajkotwal]

Convert pdf to epub and upload it to Google play books and select read it aloud. It works great.

[@sheikhakbar2067]

Wow, wonderful tips and tricks... I went through that ordeal!

[@prajnaj5450]

This is incredible piece of stuff!

[@olasideeq]

This is awesome! Thanks for sharing

[@KastanDay]

Really fantastic project!

[@GermannoTeles]

Can you show the cost off using cloud storage bucket + vision+ AutoML Tables and others

[@aravindv9291]

Should you be going for walks in the first place, if you are in quarantine?

[@Ramu294]

There are a lot of free apps which provide this function, what makes this different?

[@rishabgupta1772]

Do they have a website/app to do this on demand?

[@Jd-zd6bh]

Whoa!! Some part of the speech sounds like real human talking, specially the last part. . . . Great work. .

[@MerciBro]

You should try J on -4 pitch.

[@orhanavan7222]

Impressive work 👍

[@adamm2716]

neat project i might have to try to do it myself

[@user-or7ji5hv8y]

This is cool project. Is it expensive though? cool, I saw the answer now. Thanks!

[@yukiyuka731]

Need this because I’m too lazy to read my essays.

[@srisubhashpathuri3909]

Is there a web page, where they explain this in detail. I want to develop this on my own...

[@JL-sp8ss]

thanks for this i need to try this. It will help my dyslexia condition. V hard to read text. i am wondering how easy to set it up.

[@okhah]

I know we're all tech savvy and trying to create things, but the average guys out there just wants an app... which already exists as ebook/pdf readers with text2speach functions

[@1stSilence]

That is why "average guys" don't bring a society forward. It is the innovation that brings the most value.

[@FoxInFlame]

Those pdf text2speech functions search for actual boxes containing selectable text. What this video shows is a process to recognize the text from a completely static PDF (almost like an image), THEN apply the speech2text functions.

[@okhah]

@@1stSilence you're right💯

[@fallenangel8785]

I want subtitles on Google podcast.please

[@QuizmasterLaw]

What about books with 2 languages e.g. bilingual novels or language teaching books?

[@rishabhkumardjain]

@Voice app does this exact same thing. Not only does it handle PDFs but also a few other audiobook formats

[@stevenrldenault7451]

Hi Dale, I'm Steve (68yrs) living in Canada. I have a number of Classic Books that I would like to convert from PDF to Audiobooks. I'm not tech savvy and this approach seems daunting to me. Is there a simple way I can do this without having to learn each of the tools you mention? Thank you Steve

[@abhishtindulkar]

do they use chrome OS in the video or they just used apple? the three dots in the top left of the browser...

[@anupyadav1502]

How well does it do with the nuances of the text? Is it an ideal way to convert a story book pdf into MP3?

[@md0987]

Making this as a Google new service would be very easier and faster and even better for everyone

[@DumblyDorr]

This is very cool! Though I fear the usefulness of reading academic texts aloud is limited because of the many figures and formulas they often contain. I do like the approach of choosing the most-used font-size as a heuristic indicator - nice thinking :)

[@delfinenteddyson9865]

I suppose it highly depends in what academic field you are. For a lot of social sciences it could be quite nice.

[@Twilight_lounge]

*laughs in social sciences*

[@sunapurva]

Using the "Read Aloud" function in Microsoft Edge browser is simpler (at least for those who don't want to take the pains of building a custom program)!