Best tools for organizing PDFs in Obsidian

[@nicolevdh]

This was a long one, but I wanted to be thorough! Use the timestamps to jump around, and check the description for links to tools I mentioned. I also want to call out something about the Zotero deep link format. The Zotero link should be in one of the two formats to be clickable from within Obsidian: - zotero://open-pdf/library/items/4MEZZRVW?page=1 OR - zotero://open-pdf/0_4MEZZRVW/8 While experimenting, I found that only the first worked for me, but oddly, the second worked better for friends and we couldn't figure out why. Try both! Obsidian plugins used: Dataview, Templater THEME: AnuPpuccin: kzbin.info/www/bejne/bV62gKqupb-EhKs

[@seoyuhui3973]

Nicole, can I get your Obsidian course for 120? It's kind of pricey for me. I live in Hong Kong and the currency is weak compared to the US dollar.

[@nicolevdh]

​@@seoyuhui3973 Sorry, that would be very unfair for others who have paid the standard price. However, I wouldn't want you to feel like you NEED to get it. I've made a lot of videos for free here, and you can absolutely get a long way by just watching free videos.

[@SeanMassa]

This is one of the best comparison reviews on youtube. I'm sure this took a lot of work. Great job!

[@nicolevdh]

Thank you so much for noticing the work! It did take a lot of time and research. I appreciate you!

[@maks_st]

I had this question of how to process PDFs in Obsidian. I now realise that Readwise is the best option for my needs, so far at least. Thank you for the detailed overview, Nicole!

[@Trecoolerok]

This video was the reason i switched to zotero. I also have a huge collection of pdfs inside complex folder system, and adding them manually would be hell. Though I find more smooth way to export highlights to obsidian with obsidian-zotero plugin and biblex addon in zotero. It automatically add links, so you don’t have to manually edit links. There is one problem though. Pdf drawings on pdf inside zotero app on iPad doesn’t exports. Though I think there is the way to get these drawings outside exported pdf - because ipad software can manipulate with these drawings - so there must be the way to extract them, and export, let’s say, to excalidraw. So thanks again for this marvellous video!

[@davidgrubba]

Hi, thank you so much for the video. My only suggestion is to use the Zotero plugin integration in Obsidian; this would make the task much easier.

[@hydroross]

Regarding DEVONthink I’ve been very successful and satisfied with “indexing” my Obsidian vault folder. DT has good built-in Markdown-friendly annotation features with backlinks to the original PDF. After annotation/note-taking, I save the annotation markdown file into the Obsidian folder and archive the PDF separately. Works great. On the other hand you are spot on regarding slow and fiddly cloud syncing with the iOS app. Nicole, thanks for the excellent video. Your efforts are much appreciated.

[@KenoticMuse]

I think your take on Devonthink is very fair. I'm a fan of Devonthink, but I don't use it across multiple computers, so it's pretty much local only, and I have a use case for it.

[@MarkLevison]

Nicole - I agree that for you Zotero is a great choice. I think you did DEVONThink a disservice. I use DT on a daily basis. Once configured, the sync over Dropbox is rock solid. My archive is 1/3 the size of yours and it’s been working flawlessly for over a year. The killer feature of DT that you missed is search. Hands down it is the best I’ve ever used. So it wasn’t for you, but I don’t think you did it justice in your comments.

[@wilsonlamlyt7138]

May I ask how did you do it well? I’m struggling from DEVONthink and Zotero, and also figuring out which is best for me? I’m not quite sure how the syncing works between the cloud and how can they sync between the pdf and obsidian itself. I’ve tried to find tutorials about how to make DEVONthink work in my system but couldn’t find info about that in forums and so on. I’m still using windows now, maybe I’ll try DEVONthink later when I change to Mac. May I get some device how did you make it work in your workflow, I’ve heard some people indexed their obsidian vault into DEVONthink? And I’m not sure could obsidian then sync the vault to phone or other things like that.

[@andypolainepersonal7216]

I think there’s a misunderstanding here. DT works if you let it import the actual files into its database or have all the files downloaded. In mine and Nicole’s case, we have selective sync on, so we keep some PDFs in the cloud and some synced to the drive. DT can’t handle this well. While you can link to file references, it tries to download them all the time if you do any operations on them.

[@MarkLevison]

@@andypolainepersonal7216 and @Nicole I’m very confused. This is really not what DEVONThink is designed to do. I agree it doesn’t solve the selective sync problem well. My working assumption - anything that isn’t local to my main computer isn’t backed up. So I have all my PDFs on my computer. I have two DT databases - one that synced to my iPad and another that is local only. Also DT works well when it ingests all your files into its Database. Why would I want some files in the cloud only? I don’t trust either Dropbox or iCloud that much. Confused in Ottawa Mark

[@andypolainepersonal7216]

@@MarkLevison you’re right that this is not what DT is designed for. Nor is Zotero. Nothing only existing locally on your laptop is backed up. Sync services help. I use those plus Backblaze and Time Machine to an external drive to ensure redundancy. But this isn’t a backup issue, it’s a space issue for us. DT is designed mostly for academic papers and perhaps some other materials. RPG materials are extremely image heavy. I’m sure DT can handle a large database locally, though. And, of course, it’s main USP is the way it ingests and makes connections between papers and documents. While you can ingest the documents and then de-sync them and DT can work, they all have to be local for at least the first ingestion. This makes it unworkable if you have a library that is heading into the hundreds of GB. Personally, I consider this a flaw from the conception (I used DT a while ago and then upgraded again for our experiments), but really it’s a design decision on their behalf, so I can respect it. I don’t know what else would have to be retooled. I would prefer to have the option just to link files (so they can remain offline but referenced) on ingestion if I wanted, acknowledging I lose some of the power of DT’s “AI” connecting abilities until those files are online.

[@MarkLevison]

@@andypolainepersonal7216 My comment about backups was just that I can't imagine having files that are cloud only. At least in my business/family all files must reside on a computer. All computers must be backed up multiple ways. I have a much better understanding of your problem now. DT simply can't do what you ask of it. DT promises at any moment you can drag your files out of DT to a regular folder retaining all/most metadata. To do that metadata must be written to the file. Hence it pulls them down from the cloud. I see for game players with masses of PDFs w/images this is a problem. In which case, I want to bug @Nicole van der Hoeven - you advertised a video for "Best tools to organize PDFs for Obsidian" but that isn't the real problem you needed solved. Your deeper challenge is to: Organize files some of which are stored with Dropbox selective sync. In this case, yes DT fails. Not because of the Sync model. It fails because it's designed to work on local files. I feel the video is a bit commenting that a hammer did a bad job pushing a screw into a wall.

[@davidwgilmore]

Great review Nicole! I've also started using Zotero as my Read It Later library. I put everything in there - KZbin videos, podcasts, PDFs, articles from the web - and since I can tag it all I have all of my references in one place

[@Trecoolerok]

How do you save videos in there? I don’t know about any readwise-like plugin, which can transform youtube subtitles to text. Unfortunately, without it it mostly usefull (in my opinion of cause)

[@davidwgilmore]

@@Trecoolerok Don't have the transcripts so this may not be helpful for you. There is a Send To Zotero plugin for all browsers. And on Android there is a "Send To Zotero" app that I have installed. When I watch a video I may want to reference later I use the "Share" button in the KZbin app and the Zotero app is one of my choices

[@Trecoolerok]

@@davidwgilmore thanks for the answer!

[@ThomasTrutt-o1i]

thank you for your videos. I just started moving all my notes to Obsidian and have been binging your videos..

[@tejav1160]

You are awesome, Nicole. Keep up the great work. You helped me a lot in my Obsidian choices.

[@nicolevdh]

Great to hear that! :) <3

[@Anglosphere-ui9hv]

I use obsidian and pdf++, to classify i simple do an index note and drags books related to that note subject, not as much flexible as tagging ...but i have no need for that i dont handle too many gb of pdfs so i guess those index notes are more than enough for me, its like library reference cards after all. You could also to a note per pdf as a note registry and add tags to it, it would work for your data view and direct access to the book. And you do everthing inside obsidian of course if pdf++ annotation is enough

[@adriancabezas]

I really like Obsidian, but I tend to use Evernote because I haven't committed to personalizing my vault. For now, I'll continue to watch videos of people doing cool things with their vaults and live vicariously through KZbinrs. Keep up the good work! Your videos stand out above the other Obsidian videos I've seen.

[@uryaar7216]

Great video! I just started using Zotero for academics in order to use the tagging system as well, and still trying to figure out how to merge it better with Obsidian. This gives me some good ideas!

[@fabiankeller3047]

Thanks for the detailed comparison, much appreciated! A question on Zotero: What happens if you rename or move the PDF files in our Dropbox folder? Will the Zotero links be broken, or is there some sort of re-index so that it can re-discover the PDFs under their new path?

[@brunoresendesantos45]

Simply amazing. Been looking for something like this for ages! Great video.

[@visciuti1]

Adoro seus vídeos! Este é exatamente o que eu estava procurando para pdf de artigos científicos!

[@robson8833]

Também percebi o template em PT-BR

[@gideon9136]

Why not just use Obsidian Zotero plugin? It has most of the functionalities you mentioned

[@DannyHatcherTech]

Yay Zotero 😁

[@mageprometheus]

Yay Danny 😄

[@andresviera5059]

I´m using FOX IT PDF READER, and it´s pretty complete!!!

[@Grodnark]

Interesting! I've been on a similar hunt and had tried creating a TTRPG Vault with Obsidian by importing my big RPG directory into Obsidian (literally 1000s of files) and it takes Obsidian FOREVER to open that vault now (~10 minutes, may as well be forever). Once it got going it wasn't too bad with the aid of some plugins, but this sounds like it might be a better solution overall. The only other PDF manager that seems to get bought up is Calibre, but it kind of looks like trash and it has to duplicate all your files by bringing it into its own filing system.

[@nicolevdh]

Yay! It wasn't just me that had this problem! Yeah, I have way too many PDFs to even attempt to bring them all into Obsidian. I love that I can track everything within Obsidian, though. I feel like plaintext files win again-- no dependence on the tool. Interesting. I have used Calibre, but mainly to remove DRM from my purchased ebooks. I hadn't considered using it for organising PDFs, but just those things you mentioned make it sound not that appealing to try.

[@briancarnell]

Interesting. I have ~9000 PDF files stored in my vault and never experienced any issues with Obsidian either opening or indexing the PDFs (with Omnisearch).

[@nicolevdh]

I just checked and I have 40,000 files in my RPG directory. 😅 It could be a size thing, but in general I also thought that it would be a good idea to keep my notes separate from the actual files anyway.

[@Grodnark]

@@nicolevdh Yea, I definitely wanted it separate from my main vault, even separate from RPG Session logs. I went back in and turned off all the plugins and that seemed to work quite a bit better, only about a minute. I'll have to go through individually and figure out which one is causing the issue, though I remember it having lag even before I started adding plugins to it. Seems ok now.

[@americanbagel]

For me it was Obsidian Git causing lag. Years of regular git commits and commuting PDFs that are hundreds of megabytes large added up. I ended up pruning year old commits and using git-filter-repo to remove very large files from the history

[@Ikbenjanus]

Wow, your video was so extremely helpful. Thank you!

[@wilsonlamlyt7138]

Curious to ask from this video, which drive would you use and why is it the case? I’m deciding to do some research into how should I sync my obsidian and where I put my main files in? I’m thinking should I put files in Dropbox, Google drive, or something like iCloud, the obsidian syncing is also something that I struggle with, all kinds of syncing seems to have this or that problem and I would still like notes to be accessible on my phone. I’m thinking about this because I’m going to change from windows to macOS possibly. Which drive or syncing way would you recommend? Thanks for this great video again, I found these videos very helpful in making decisions, especially when you are in a same device or need as I am!

[@nicolevdh]

How to sync in Obsidian

[@BigHenFor]

Hi! I'm stuck using Obsidian on Android and I'm broke, so Google Drive was my only option. I found Drive Sync in the Play store extremely easy to use as a means to syncing my Obsidian vaults between my device and Google Drive. The only downside is that to sync more folders than in the free version you have to pay a one off fee, which I did as its functionality and ease of use justified it. It automatically backups changes in obsidian to the cloud.

[@timbushell8640]

And Zot can 'read' and highlight and note/comment on most of the other 'book' formats and most other files and basic web pages too... so 'could' be made your 'single source of collected stuff' if you wished.

[@allroni]

Great review, thank you!

[@ZakirHussainIbrahim]

Hi Nicole. Great video. Is there a work around for Mendeley instead of Zotero? Thank you in advance.

[@vishalsachdev6767]

Zotero's PDF annotations are just the best. I still keep PDFs in my vault too, I just use a markdown file to add tags, give it an id and make it searchable (just the metadata). I like to export annotations from Zotero to obsidian so I can use them within my own notes. I find zotero's note taking to be very primitive. Great video btw.

[@nicolevdh]

Yeah, I agree that Zotero's note-taking is a bit primitive. But luckily Obsidian is great at that! :) Glad you liked the video!

[@ciaspo]

Incredibly useful video, thank you! 👏🏻🙏🏻

[@nicolevdh]

Glad you liked it! :)

[@nhannguyen2190]

Could you share the syntax you have for Espanso at 20:15? Can't wait for your Espanso video ha ha. Thanks a lot!

[@nicolevdh]

Sure! It's zotero://open-pdf/library/items/4MEZZRVW?page=1 . I've also added it to the first (pinned) comment now. Thanks for the reminder!

[@giddycarousel]

What about Bookends instead of Zotero? I find more solid and Apple oriented.

[@marcobertalot-bay9342]

Thank you so much! I work with Evernote. Obsidian is difficult for my "hard" brain!🙃 Is there an instruction for linking Notes in Zotero PDF's with Evernote? I thank in advance ! Marco from Botucatu, Brazil

[@Hokageee]

Que legal! Obsidian é muito bom.

[@timbushell8640]

Also - with your comment of 'checked against another computer'... the setting up of Zotero isn't covered well in this vid - if you are a new user/database. I think many will need more help there - directly or with some links to Zotero sources - especially in building the 'linked' database, and not the local hard drive as so many instructors show... ... So your suggestions are?

[@fafilingdo]

I have a question, when you configure zotero to your online drive, do you put it in the same folder as your Obsidian vault or a different folder?

[@fabiano-co]

You can categorize a file using Obsidian's new feature bookmarks.

[@nicolevdh]

You definitely can, but since I have literally thousands of PDFs, that would get quickly out of hand! I prefer to think of it more as a database. But YMMV!

[@ankerschilling3308]

Have you figured out how to get the Links for Zotero PDF´s to work on your android phone as well?

[@pk-mind]

Hi Nicole, I saw once that there is a limit in the volume of documents on zotero, no? If yes this is quite important in your evaluation, no?

[@RBTTraining]

I believe that limit only applies if you store the documents in zotero, instead of using the method she is recommending.

[@ManmanLLC]

Hi Nicole, what is the Obsidian theme you use? It is so pretty!! And your videos are really clear and concise! Thank you so much!

[@chanzziew]

It looks like she is still using AnuPpuccin. She has a video on it called 'My New Favorite Obsidian Theme.

[@ManmanLLC]

Thank you!@@chanzziew

[@timbushell8640]

So - if Readwise 'fails' on the PDF front - in its tag/category and sharing - what about the other highlights and notes/comments of other 'documents'... or are you for PDF making an 'extra condition' over the other documents you work with? No RPGs for me - and whilst not fully academic - I do need the cite feature. Likewise, I like the 'collection' feature in Zot... it is just a sync (manual) to keep the collection name and the Obsidian project name tracking so more than just a link. [Along with the 'same' project names in say Research Rabbit and Elicit, etc., to hunt down other references.] And then there is Zot's link files - the missing possible con is... if you use Zot via copies - there is a limit (300MB IIRC) which is pretty low as you repeatedly pointed out you have a 'few' pdfs and some being graphical will be bigger than the average peer review file even... BIG issue if you forget to use the links. Above 300MB Zot wants/needs money spent on it - good value, but...

[@nicolevdh]

Hey Tim! Yeah, I still use Readwise for all other types of documents. PDFs are just a special case because I don't need the same things from it (detailed annotations) that I typically like to have with other notes. I haven't run into the issue with large PDFs being linked files yet... I'm not sure if I do have a PDF >300 MB yet-- I'll have to test that. Thanks for the heads up!

[@huenchenholli]

Thanks Nicoole for another great video! I was just curious why you did not use the "Zotero Integration" plugin for Obsidian to deal with annotations and links to them. For me this was the reason to switch to Zotero for my PDFs. And "ich wünsche Dir Glück!"

[@andypolainepersonal7216]

This only really works if you annotate does it not? The link is to the annotation, not the PDF page itself.

[@yemartin]

Thank you so much Nicole! I have been using Evernote for over 10 years, to scan all my paper documents, so I amassed quite a collection of PDFs. But I have been looking for an alternative, and Zotero might be it! Do you know if Zotero can (or has a plugin to) search the content of PDFs, and in particular of scanned PDFs?

[@grittang8531]

The add-on “Zotero OCR” working with tesseract is my choice. I think it will help you.

[@DrTrash-xq7kf]

I've only been using Zotero myself for a few weeks, but my experience with searching for terms within(!) PDFs with text layer is quite modest. I must say, however, that I have not yet done any research on this in the manual or on plug-ins for the search-tool. I use Zotero primarily on Windows and occasionally on iOS (iPads) pretty much out-of-the-box. Firstly, the PDFs have to be indexed by Zotero so that they are taken into account for the search results. For this purpose, there is a status for the individual files that shows whether this has already been done and, if necessary, you can trigger the indexing manually. It seems to me that indexing of individual PDFs getting sometimes lost and you have to trigger it again, which is tedious with large numbers of files and you also have no overview of what is indexed and what is not. Secondly, in the search field you must always make sure to change the scope to 'Everything', otherwise only the primary data fields are searched. Furthermore, the 'Advanced Search' does not support the search in the file contents - it only searches targeted metadata fields. Thirdly, the search results seem very inconsistent. If I enter single words completely, it works fine. It becomes problematic when I try to search with wildcards, which should not work, or when words in the PDF are written with an active caps lock. Here, terms are very quickly no longer found.

[@nicolevdh]

@@grittang8531 Nice of you to reply! I would not have known the answer, so I appreciate you jumping in. :)

[@nicolevdh]

Hey! It sounds like your use case is very different from mine-- I'm not really looking to be able to search text within PDFs; I need only very little metadata that I can look for within Obsidian. I'm aware this may be a niche case. Maybe something like DevonThink would work better for you. Its search capabilities are superior.

[@timbushell8640]

Really depends on the source method of making the pdf - if say created from word/latex as most peer-reviewed papers are then search is a piece of cake, if your own 'scanable-into-Evernote' no idea those older or more graphic pdfs - then Zot work well - but variable. Moving non-text from Evernote is going to be painful... but bulk export/import via Joplin works excellent on the text and pretty well on the graphic elements, and renders a very clean MD file for importing into Joplin or Obsidian - but you just have to realign your stacks and notebook structure in some way to suit Obsidian single or multiple vaults... But the famed Evernote feature of searching within the pics for texts and numbers often seems to get chopped off.

[@dannycobble2839]

Have you tried the zotero plug in for obsidian. It pulls your annotations in without having to do all the steps in zotero. You just us a hotkey

[@timbushell8640]

How to navigate the IPCC reports - which are typically 3,000 plus pages : )))) via your Arcdian example... off to play.

[@dd.oliver]

Obsidian should be more and better plugins integrations with Zotero. These tools are amazing and should be only one app

[@skylarkhighline]

What Obsidian theme are you using? Looks nice!

[@chanzziew]

It looks like she is still using AnuPpuccin. She has a video on it called 'My New Favorite Obsidian Theme.

[@Ben-Alaluf]

What is the first tool you use? When you show the PDF chart?

[@countryman6083]

Zotero is nice, but your pdf files are not in your vault, so I prefer the shim method while waiting for a pdf management feature in Obsidian.

[@nicolevdh]

There's nothing stopping you from having PDFs in your vault and using Zotero. Zotero doesn't care where your files are located as long as you point it to the right place.

[@ydherdn]

Cools .. Thank You

[@darkomen4343]

Thank you!

[@122hb]

marginote is a good choice, but only for MacOS

[@vencler]

Hey I know that youre doing whats comfortable for you, but this is just my personal opinion and its definitely subjective and I don't know if others agree: For me it's very hard to watch your content due to colors. The combination of frontal+back lighting is very hard on my eyes, and paired with very bright hair color, it gets really hard at times. I still watch because your content is great! But I can't say it's enjoyable viewing experience. I'm not saying this in a bad manner, just pointing out something that could maybe be improved.

[@giftcp82]

How can i refresh the pdfs folder after adding new files

[@randolphclark6427]

What is the name of this red theme? Thanks.

[@mechjack22]

Why does the reason Zotero gets a check for categorize not apply to Obsidian? You’d just create a Markdown pointer to the PDF either way. What am I missing?

[@nicolevdh]

Hey! My reasoning for that is that Zotero lets you natively tag PDFs AND create collections and sub-collections. Obsidian doesn't natively let you tag the PDFs themselves (you'd have to create a separate note to connect the two) and collections are different from folders in that you can't put something in more than one folder, but you can put something in more than one collection.

[@mechjack22]

@@nicolevdh That’s my point. You’re creating a separate note to connect to Zotero too unless I’m issuing something.

[@KeithDvorak]

also tons of metadata (built-in and custom) which might be useful-but i think maybe you don’t use tons of metadata?

[@nicolevdh]

Yep, you're right! I don't use much metadata. I don't really have the use case for it!

[@lpanebr]

Porque seu obsidian está em português?

[@bobbyv3]

A fellow digital information hoarder. Thought I was the only one sitting over here with 17GB of eBooks and 72GB of wallpapers. 😬

[@sdnnvs]

Obsidian's plugin Omnisearch is very useful....

[@nicolevdh]

Thanks! I have looked into Omnisearch, but I don't really need for it to search inside files, so I don't really have the use case for it. Could be helpful for others, though!

[@KrustalHersh]

I also "somehow" have ended up with many PDFs 😆

[@amallukose3763]

logseq is the best

[@gabrieloforti]

Você fala português?

[@nicolevdh]

Sim, um pouco, mas meu espanhol é melhor. :)

[@gabrieloforti]

@@nicolevdh Perguntei, pois o menu do obsidian (context menu) está em português. Legal, adoro seus vídeos.

[@harishkumar929]

Do really a person need somuch of notes (pdf) I guess you create your own internet 😅

[@nicolevdh]

Nobody NEEDS them, but I happen to have them! YMMV.

[@timbushell8640]

@@nicolevdh ha ha ha +30k number PDFs just on the lappy, archive and back up... not even going to guess.

[@monochr0m]

Do you know of a solution to link from PDF *to* Notes?

[@nicolevdh]

This was a long one, but I wanted to be thorough! Use the timestamps to jump around, and check the description for links to tools I mentioned. I also want to call out something about the Zotero deep link format. The Zotero link should be in one of the two formats to be clickable from within Obsidian: - zotero://open-pdf/library/items/4MEZZRVW?page=1 OR - zotero://open-pdf/0_4MEZZRVW/8 While experimenting, I found that only the first worked for me, but oddly, the second worked better for friends and we couldn't figure out why. Try both! Obsidian plugins used: Dataview, Templater THEME: AnuPpuccin: kzbin.info/www/bejne/bV62gKqupb-EhKs

[@ajpearce]

That espanso command seems pretty essential. I'm trying to figure it out now