shot-scraper

A CLI utility for taking screenshots of websites, recording video demos and scraping sites using JavaScript

For background on this project see shot-scraper: automated screenshots for documentation, built on Playwright.

Documentation

• Full documentation for shot-scraper
• Tutorial: Automating screenshots for the Datasette documentation using shot-scraper
• Release notes

Get started with GitHub Actions

To get started without installing any software, use the shot-scraper-template template to create your own GitHub repository which takes screenshots of a page using shot-scraper. See Instantly create a GitHub repository to take screenshots of a web page for details.

Quick installation

You can install the shot-scraper CLI tool using pip:

pip install shot-scraper
# Now install the browser it needs:
shot-scraper install

Taking your first screenshot

You can take a screenshot of a web page like this:

shot-scraper https://datasette.io/

This will create a screenshot in a file called datasette-io.png.

Many more options are available, see Taking a screenshot for details.

Examples

• The shot-scraper-demo repository uses this tool to capture recently spotted owls in El Granada, CA according to this page, and to generate an annotated screenshot illustrating a Datasette feature as described in my blog.
• The Datasette Documentation uses screenshots taken by shot-scraper running in the simonw/datasette-screenshots GitHub repository, described in detail in Automating screenshots for the Datasette documentation using shot-scraper.
• Ben Welsh built @newshomepages, a Twitter bot that uses shot-scraper and GitHub Actions to take screenshots of news website homepages and publish them to Twitter. The code for that lives in palewire/news-homepages.
• scrape-hacker-news-by-domain uses shot-scraper javascript to scrape a web page. See Scraping web pages from the command-line with shot-scraper for details of how this works.
• Reuters uses shot-scraper to generate regularly updating data dashboards for email newsletters.