← All projects
Research tooling · Built by Sayan Mondal
PaperHarbor
Turn scattered references into an organized collection of available open-access PDFs.
Project overview
Less time collecting papers.
The problemFinding full texts, naming files, and tracking missing papers interrupts literature review. A bibliography may contain a mixture of publisher links, repository copies, and references with incomplete metadata, making manual collection repetitive.
The approachImport Zotero, BibTeX, CSL-JSON, or a DOI/URL. Search and select references locally, then download available copies. The same discovery pipeline supports interactive selection and repeatable command-line batches.
Supported inputs
Start with one paper or an existing library.
Choose the source that fits your research workflow
| Source | Input | Typical use |
| Single paper | DOI or article URL; optional fallback title | Retrieve a specific reference without a library. |
| Local Zotero | Auto-detected or explicit SQLite database path | Work through missing attachments in a collection. |
| Bibliography export | BibTeX or CSL-JSON file | Build an archive from an exported reading list. |
| Zotero Web API | API key and personal or group library ID | Read references from an online library. |
| Previous report | PaperHarbor CSV | Revisit unresolved references after an earlier run. |
In the browser interface, search references, filter collections, select papers, and follow progress and logs before reviewing the final report.
Architecture
From references to a personal archive.
- Read metadata
- Select references
- Discover candidates
- Validate downloads
- Organize PDFs
- Report outcomes
Discovery and fallbacksThe provider aggregator checks reviewed author copies and supported publisher links, then conditionally queries Unpaywall, OpenAlex, Europe PMC, arXiv, Semantic Scholar, and Crossref. If initial downloads fail, it queries providers skipped earlier, including search fallbacks, and inspects publisher pages.
Candidate URLs are deduplicated, direct PDFs receive priority over landing pages, and conflicting DOI candidates are rejected. Provider errors are recorded while other searches continue.
Shared orchestrationThe application coordinates readers, provider searches, landing-page resolution, file naming, downloads, and reporting. It checks existing attachments and destination files before searching. Repeated DOI outcomes within a batch are reused to avoid duplicate network requests.
References are processed sequentially with a default 0.5-second pause. An individual reference error is recorded and processing continues with the next paper.
Implementation: application coordinator ↗ and provider aggregator ↗.
Download reliability
Check the response before keeping the file.
File and identity checks
- Require a PDF signature within the first 1,024 bytes and a minimum download size of 1,024 bytes.
- Reject redirects that identify a conflicting DOI.
- For candidates requiring an identity check, verify the requested title against text from the PDF’s first page.
- Stream to a temporary file before moving a successful download into its destination.
Bounded retriesTransient network failures and selected HTTP responses receive up to two retries by default. Backoff uses the server’s Retry-After value when available; requested waits above 30 seconds are deferred for a later attempt.
Access-denied responses are not repeatedly retried. A PDF header check identifies the file format; it does not establish the scientific quality or completeness of a paper.
Implementation: PDF downloader ↗.
Reporting and recovery
Keep the outcome of every reference visible.
A timestamped CSV records bibliographic metadata, status, source provider, file path, size, PDF URL, and notes. Attempt counts and structured attempt logs preserve the discovery history; recovery actions and suggested filenames support manual follow-up.
Outcome categories
- Downloaded or already present in Zotero or the destination.
- No open copy found, download failed, or manual access required.
- Reference error, recognized web resource, or withdrawn paper.
Retry only unresolved workPass a previous report with --retry-report to revisit failed, manual-access, and error rows. Add --retry-not-found to include papers for which no copy was found. Verified alternate URLs can be supplied through --pdf-overrides.
Implementation: CSV reporter ↗.
Technology stack
A Python tool with a local interface.
PythonRequestsBeautiful SouppypdfSQLiteZotero Web API
Requests handles HTTP access, Beautiful Soup supports HTML parsing, and pypdf supports PDF inspection. The web server uses Python’s standard library.
Try the project
Use the browser or run a repeatable batch.
Clone the repository and install dependencies in a virtual environment. The following commands use Windows PowerShell:
git clone https://github.com/sayanmondal2098/PaperHarbor.git
cd PaperHarbor
python -m venv .venv
.\.venv\Scripts\python.exe -m pip install -r requirements.txt
.\.venv\Scripts\python.exe web.py
Choose a destination folder, upload a bibliography, select references, and start downloading. The default local address is http://127.0.0.1:8765/.
Import a reading list.\.venv\Scripts\python.exe main.py `
--export-file "references.bib" `
--dest-folder "D:\Research\Papers" `
--limit 10
Retry a previous report.\.venv\Scripts\python.exe main.py `
--retry-report "previous_report.csv" `
--dest-folder "D:\Research\Papers"
The limit counts references evaluated, including skipped entries. Use --collection for a Zotero collection filter and --template to configure year, author, and title in filenames. Review CSV outcomes after a batch completes.
Installation instructions ↗
Data handling
Local files, visible outcomes.
Library accessThe server binds to localhost. The SQLite reader locates common Zotero database paths or accepts a custom path. It queries a temporary copy and attempts to copy accompanying WAL and SHM files when present, then closes the connection and removes the snapshot.
Downloads stay in your chosen folder. PaperHarbor does not attach them to Zotero or update library records. Reference metadata is sent to discovery services during searches.
Practical boundariesAccess restrictions or missing metadata can prevent retrieval. Unresolved items retain follow-up links for manual review. Missing attachment detection relies on Zotero metadata, which may differ from files actually present on disk.
Sequential processing favors a traceable workflow over parallel throughput. An unsuccessful search does not establish that a paper is paywalled. Publisher access rules still apply.
Implementation: local Zotero reader ↗.
Project resources
Explore PaperHarbor.
Based on the project documentation, reviewed 31 October 2026.