How Do You Organize and Annotate PDFs?

I wanted to pass on this suggestion from @betajames on twitter: http://www.ironicsoftware.com/yep/index.html.

As a mathematician, I am dependent on BibTeX for producing bibliographies for my papers. Thus, BibTeX support is a key for me. I checked out Papers last summer and liked it. However, I would have had to export a new BibTeX file every time I added a new paper to Papers. I’ve stuck with BibDesk for the time being. It lets me attach the paper to the entry in the BibTeX database. I’m not much of an annotator, but Skim is my choice if I need to do it.

If you use Skim to comment and annotate, you can make sure your annotations are preserved if you print the annotated document as a new PDF (giving it a new filename) rather than simply saving it.

This is a topic I’ve been looking at for nearly a year. I’ve established a document system for the office and given a few talks (of varying technical levels) to a few departments on campus.The result seems to be the same: make it easy.With that in mind, the best way to work with PDFs is by far on a Mac. I’ve done Mac and PC tech support a while, and regardless of how PC-savvy you are, you’ll need a third party tool to be effective. On a mac, Preview is the fastest way to view, and do basic annotation.Zotero is probably the best for recording scholarly work, specifically from sites like JSTOR. However, for you scanned documents, or other PDFs, it’s not really the best tool.DevonThink and EverNote both seem to get a lot of play, and EverNote is PC compatible. Those are good options, but both have limits- EverNote especially unless you want to bleed money for more storage space.Ironic software’s Yep! does a good job of collecting PDFs, but version 2.0 has removed a great many features, and added a number of redundant ones, like the new bar on the side of your screen. Filing doesn’t prompt to tag, tagging doesn’t file, and archiving does neither. Who thought of that workflow?What I finally settled on was a combination of iDocument for mac (like iPhoto for your PDFs) and hazel.Hazel is an amazing app for automating the mundane without any coding. It allows you to build email-like rules for files in a given folder. So, for example, when I scan a PDF onto my computer, it lands in Documents. I have a Hazel rule set as follows: (note: I’m typing it out, but Hazel is a no code GUI, it looks almost exactly like Mail’s rules window)In folder Documents:If file kind is PDFRename with format [date] [keywords] [original name] .pdfRun Applescript:try tell application “Adobe Acrobat Pro” activate open theFile tell application “System Events” tell process “Acrobat” tell menu bar 1 tell menu “Document” tell menu item “OCR Text Recognition” tell menu 1 click menu item “Recognize Text Using OCR…” end tell end tell end tell end tell keystroke return end tell end tell save the front document close the front document end tell tell application “Adobe Acrobat Pro” to quitend tryMove PDF to subfolder “PDF Docs”From there I import into iDocument when I’m done working with the PDF.the above apple script makes Acrobat OCR the PDf so that it’s searchable. I also have the same code for PDFPen, another brilliant mac PDF app if anyone is interested.A note about PDFPen- PDFPen is a beautiful middle ground between preview and Acrobat. It’s small, lightweight, and feature rich. It will annotate, OCR, break apart, and put together PDF files in just about anyway you’d expect, and unlike Acrobat, it’s natively Applescriptable (unlike the above code which is Applescripting the GUI of Acrobat).I do everything I can in PDF, it’s simply a better way to work.For things other than documents, receipts and the like, I use NeatWorks for NeatCo. There are two scanner options, the NeatDesk scanner, and the smaller portable scanner. NeatWorks reads your receipts, OCRs them, and pulls the relevant data (store, items, total, tax, etc) into a database that is both searchable and exportable to most financial programs. Best of all, the PDF receipts are accepted by the IRS should you get audited.If anyone wants other code snippets, or more info, I’d be happy to share, email me at: aaron . perrell at gmail . com (hence the Peril username, it’s a phonetic thing ;)

Awesome post and follow ups–thank you–I’ve been wondering about this kind of application for a while. Question: Does iDocument reorganize your finder document folders (or, does it force you to), or does it leave the .pdfs where they are? If I don’t like it, I want to undo the damage. I can’t quite tell from their introductory videos.

Microsoft has done a horrible job marketing its tablet PCs. Real tablets with wacom pen digitizers.I have been annotating PDFs with a pen using Zotero and PDF Annotator for years in Windows. I keep my PDFs organized in Zotero and then annotate them using simple and cheap software. My handwriting is even searchable. Now of course, i could also annotate PDFs using a keyboard and mouse, but i find reading that way unnatural. This method has served me very well as it allows me to read and write on documents (via the tablet) and have my comments, underlines, and everything else saved and searchable. Amazing stuff!I travel internationally quite often and the tablet saves me from printing up anything to read on my field trips but allows me the ability to feel that I am reading in a fashion more natural to me (looking down with a pen in my hand rather than looking out at a computer screen). HP makes a very nice tablet you can buy for around $800. Lenovo makes the X201 tablet – the nicest and lightest – for more than $1000. This is fantastic technology that has made me extremely productive and organized. It’s been around for years. I would encourage you to give it a try!

Sente has come up in the twitter stream–anyone used it? http://www.thirdstreetsoftware.com/site/introduction.html.

I use Zotero with firefox under Linux for my bibliographic data, attaching PDF’s of the paper (when I have them) to the bibliographic record. This works fine for organizing my collection and preparing BibTex bibliographies for use in my papers. Unfortunately, I’m not aware of any Linux tools that can actually annotate directly on the .pdf.

Thanks – this is useful (the parts I understand). I feel good about my decisions so far to rely on Preview (Mac) and Zotero in general. As a historian I find the oddest kinds of materials. A cocktail napkin as a historical source? Oh yeah! I need a program with some flexibility. :-)

Sente users/enthusiasts have posted in the Chronicle’s Forums section. The search function is difficult to use, but most of the threads should be under “Tech Talk for Befuddled Academics.” I haven’t used it myself but the feature set looks very attractive and I’m planning to try it out this summer.

For Windows users: I’ve used Nitro PDF for several years and am very happy with it. I use it not only to edit, annotate, and combine PDF files, but also as a scanning utility. In addition to two different Nitro products (different feature sets/pricing) they now offer a variety of free tools, including an online PDF editor, various conversion tools, etc.

I would be very interested in more suggestions for annotation of PDFs on Windows based machines, please!

A distinction worth keeping in mind as you track through these suggestions: when a user or reviewer says s/he wants to “annotate PDF files” — this can either mean “add tags or notes to an attached PDF file” in an information manager type program (Zotero, Evernote) or it can mean “actually write on the PDF itself directly, using highlights or comment boxes” which requires a program that actually edits PDFs.

I use DEVONthink and really love it. If you can afford it, I would recommend getting DT Pro Office, which includes OCR for scanned documents. I was using the Pro Office Beta until the beta period expired, and got much use out of it. It’s especially powerful in its search and sorting features. It also captures and stores RSS feeds, so you can keep archives of your own blog writings or any other news feed that you think it so great that you need to capture every post. Finally, it includes a bookmarklet so that you can easily capture web pages.

Mendeley comes with both desktop and on-line components. The desktop program recently added a PDF viewer that allows annotation.

I’m right there with you, nsmith1017. My scattered approach to notes and PDFs is partly what led me to write this post. Peripatetic322 mentioned above that Sente is working on an iPad app. I’ll keep an eye on that as it develops.

I did some additional research on the Windows side and I would also suggest an option to make comments and notes on PDFs that you consider PDF-XChange Viewer. This version of the software is free, but there are also more robust versions as well.I will be sharing this too with my students as well to allow them to make notes directly on PDFs which I distribute to them in advance of class.

Katereit,One of my first articles here at ProfHacker was all about Scrivener: http://chronicle.com/blogPost/Scrivener-Scrivening/23026/. I also highly recommend it.–Ryan

I think it’s worth making a distinction between programs that (prefer to) store PDFs for you and those that do something to make the PDFs more organizable in their original locations. And I think it’s also worth distinguishing between those designed as some kind of reference management software (not really useful for PDFs not connected to research, but powerful features for that purpose) and those with a more general focus (wider range of uses but not very powerful for storing things like authors, titles, journals, etc.). And it can also be important to know whether the software can handle non-PDFs, too. Distinguishing some of the major OS X programs in these ways leads to something like this (I’m writing this off the top of my head so no doubt I’ll omit some important options; also, I only include applications that I actually own or have used in the past):1. StorageA: store in the program:Bookends, Sente, Papers, Mendeley Desktop, Yojimbo, DevonThinkB: track in the Finder:EagleFiler, Yep, Leap2. Reference ManagementA: Dedicated reference managers:Bookends, Sente, Papers, Mendeley DesktopB: General file organization:Yep, Leap, EagleFiler, Yojimbo, DevonThink3. File typesA: PDF only:Papers, Mendeley Desktop (?)B: PDFs and others:Bookends, Sente, Yojimbo, DevonThink, EagleFiler, Leap, YepAs I complete this list, I wonder how much sense the first criterion makes, as most of those programs in 1A also have options for indexing PDFs in their original location. But I think perhaps it’s still fair to say that in general that’s how those programs work optimally, and that you’d lose a lot of the work you’d put into a program of the 1A type if you decided to throw it away without any planning.A few other points:1. Papers is a strange beast. It’s the only “reference management” software that has an iPhone/iPod touch companion (and now iPad, too), and for me that alone makes it worth the price of admission. It’s also the only one in this category that doesn’t have the ability to make bibliographies in APA and MLA formats, and so on, instead focusing solely on finding the papers you need and organizing them.2. Bookends is the least slick looking of all the programs mentioned (though I guess DT competes pretty well). Yet it is the only one of the reference managers with tag clouds, often considered de rigueur in other software categories.3. Sente is the slickest (with the possible exception of Papers) of all the reference management type programs and has excellent notetaking capabilities. These include automatic page references, snapshots of non-textual info, and flexible export options.4. Mendeley Desktop has a companion website, and the combination of desktop software and website is wonderful when it comes to keeping track of information on multiple devices or in multiple locations. Mendeley’s method of extracting reference information automatically (scanning the first page of a PDF for strings that look like titles, authors, journal names, and so on) for now (and in my experience) generally works better than programs that favor online lookup.5. If anyone is interested in something like the Noguchi filing method, Yep may be the best choice, as it allows you to file new items in date-based folders that it creates automatically, and it allows you to browse at day, month, or year level seeing all items at different levels in the hierarchy in a single list or grid. In general, the number of options available on Mac OS X alone is too great for most people to find out about, let alone really experiment with. And it’s not just a matter of choosing “the best reference manager” or “the best snippet keeper” but of identifying the available categories of software, deciding which ones you need and how to use them together in a meaningful way, and then deciding which particular option fits your needs the best.To make the options more manageable, I would suggest at a minimum that you deal with reference management and other options in separate posts, assuming you have the space/time to follow up on this topic.

Wow, lots of ideas here! If I had a tablet, I’d follow the approach outlined in comment #12 above, using the tablet interface to take notes, highlight, etc., right on the PDFs. Since I don’t have a tablet, well, I take an unfortunately low-tech approach. I print out my PDF journal articles and annotate them with pen and highlighter! I’m embarrassed to be admitting this on ProfHacker…Actually, my approach is somewhat more high-tech than that. It takes some time, but I blog about articles I find interesting. This gives me the chance to organize my thoughts about the article, share them more publicly, and find them later through my blog’s categories and search tools. I’ll often search my blog for a particular idea, re-read my post about articles featuring that idea, then pull up the PDFs for those articles (now saved on DropBox, thanks to ProfHacker’s suggestion!) if I need to read the originals.Like I said, if I had a tablet (or at least a stylus input device), I’d use that. However, the pen-and-paper approach gives me something to do during take-offs and landings when I travel!

Behind the 8 ball, I know, but this article inspired me to *find* a solution. This is what I came up with.I use Hazel for a lot of stuff, so Aaron’s post (peril – May 11, 2010 at 07:56 pm) caught my eye. Hazel monitors a folder in my Documents folder called PDF. When a PDF appears in the folder it runs Aaron’s PDFpen AppleScript to OCR the document, renames the file using Spotlight comments as keywords (command-I) according to this formula [date created]_[comment]_[original filename].[extension], moves the file to a “Processed” folder, and sends it to Memeo Connect for document management.I really like Memeo Connect. It synchronizes with Google Docs so I can access my edited and annotated PDF’s from anywhere. It’s $39.00 per year, which includes a Google Premium subscription, but this is a small price to pay for accessibility in my opinion. Drop me a line at jgrafft at gmail dot com if you’d like to get my scripts.

Hi Ryan,Thanks for this roundup and giving us the opportunity to promote some of the other PDF and research management tools out there.I have been coming at the research management problem from a different perspective. I decided to build Qiqqa (pronounced “Quicker”) at the same time as working towards my PhD.Although I did try using existing reference management applications to locate my papers, I found them all quite lacking in the ability to mark the interesting parts of my papers (text, formulae and diagrams) so that I can quickly review them every few months, and especially when it comes time to writing up my thesis! Searching the annotation comments also works (tagging them is coming soon as we have had several people ask for that one). GoogleScholar is still my method of choice for locating documents, so I made it easy to query Google Scholar while reading a PDF. Of course, using it all day, I have added in tons of little nifty features that save me loads of time (e.g. right-click text in PDF and lookup word in dictionary.com). Also, I found the text searching and exporting support from most of the other tools to be quite unreliable for scanned PDFs, so I built in OCR to tackle that.At the moment I don’t support bibliography management – although that is coming fast (I have only been working on Qiqqa since the beginning of this year and I don’t want to add yet-another-half-baked-references-manager to the world). But first I am testing out the synchronizing and backing up onto the Amazon S3 cloud so that I can work from the lab, my laptop and from home.I am hoping that others will find Qiqqa useful, and will enjoy using it as much as I have enjoyed writing it. I hope to get a lot of feedback from other students and researchers to learn how they are approaching their PhDs or research, and where they are hitting bottlenecks.It’s available at http://www.qiqqa.com.Thanks,Jimme (jimme@qiqqa.com)