goodreads-to-sqlite

Save data from Goodreads to a SQLite database. Can save all your public shelves and reviews, and also the public reviews and shelves of other people.

How to install

$ pip install goodreads-to-sqlite

Add the -U flag to update. Change notes can be found in the CHANGELOG file, next to this README.

Importing from CSV Export

The easiest way to use this tool is to import from a Goodreads CSV export. To get your CSV export:

1. Go to https://www.goodreads.com/review/import
2. Click "Export Library"
3. Wait for the email with your export file
4. Download the goodreads_library_export.csv file

Then import it:

$ goodreads-to-sqlite import-csv goodreads.db goodreads_library_export.csv

You can optionally specify a user ID to associate with the imported books:

$ goodreads-to-sqlite import-csv goodreads.db goodreads_library_export.csv --user-id your_user_id

This will create the same database structure as the API import, including:

• Books with titles, authors, ISBNs, publishers, publication dates, and series information
• Authors (including additional authors from the CSV)
• Reviews with ratings and dates
• Shelves (read, to-read, currently-reading, and any custom shelves)

Authentication (for API import - deprecated)

Note: The Goodreads API has been shut down, so the API-based import methods below no longer work. Use the CSV import method above instead.

Create a Goodreads developer token: https://www.goodreads.com/api/keys

Run this command and paste in your token and your profile URL:

$ goodreads-to-sqlite auth

This will create a file called auth.json in your current directory containing the required value. To save the file at a different path or filename, use the --auth=myauth.json option.

Retrieving books (API - deprecated)

The books command retrieves all of the books and reviews/ratings belonging to you:

$ goodreads-to-sqlite books goodreads.db

Note that your Goodreads profile must be public in order for this to work - if it is not already, you can enable this by visiting https://www.goodreads.com/user/edit?ref=nav_profile_settings and selecting "anyone (including search engines)" within the "Settings" tab.

You can also specify the user to target, to fetch books on public shelves of other users. Please provide either the user ID (the numerical part of a user's profile URL), or the name of their vanity URL.

$ goodreads-to-sqlite books goodreads.db rixx

Sometime in 2018 or 2017, Goodreads started leaving out some "read_at" timestamps in their API. If you want to include these datapoints regardless, you can add the --scrape parameter, and the dates will be scraped from the website. This will take a bit longer, by maybe a minute depending on the size of your library.

$ goodreads-to-sqlite books goodreads.db --scrape

The auth.json file is used by default for authentication. You can point to a different location of auth.json using -a:

$ goodreads-to-sqlite books goodreads.db rixx -a /path/to/auth.json

Limitations

• The Goodreads API has been shut down, so API-based imports no longer work. Use the CSV import instead.
• The order of books in shelves is not exposed in the CSV export, so we cannot determine the order of the to-read list.
• Book descriptions and cover image URLs are not included in the CSV export, so these fields will be null when importing from CSV.

Thanks

This package is heavily inspired by github-to-sqlite by Simon Willison.

The terminal recording above was made with ASCIInema.