scrapeadvisor

---

Table of Contents

• About
• Getting Started
• Run
• Usage
• Statistics
• Supported Languages
• Built Using
• Authors
• Acknowledgments
• Disclaimer

About

If you're reading, dear Tripadvisor, Inc., hire me!

Getting Started

Prerequisites

• Python installed
• R installed

Installing

Make sure you've all Python dependencies installed with:

scrape@advisor:~$ pip3 install -r requirements.txt

Also, the following R packages are needed:

• dplyr
• readr
• lubridate
• ggplot2
• tidytext
• tidyverse
• stringr
• tidyr
• scales
• broom
• purrr
• widyr
• igraph
• ggraph
• SnowballC
• wordcloud
• reshape2
• TeachingDemos

You can manually install missing ones with:

install.packages("library_name")

or run this script (credit to @stevenworthington) to install them all.

Note

For Italian language support, TextWiller library must be installed. To do that:

install.packages("devtools") 
install_github("livioivil/TextWiller")

Run

In order to launch scrapeadvisor GUI, run:

scrape@advisor:~$ python3 ScrapeAdvisor.py

Usage

Insert URL

1. Insert the main page URL of a TripAdvisor structure (pub/restaurant/hotel/whatever) in the URL bar and click Enter (or press Enter)
2. Wait until "Reviews Loaded" label appears (may take time, depending on number of reviews)

Show Reviews

After the download is finished, press "Show Reviews" to see all the downloaded reviews.

Sentiment Analysis

After the download is finished, press "Sentiment Analysis" button and wait: all the graphs related to the facility will appear follow after, so you can swipe between them.

Statistics

• Frequent couple/trio of consecutive words (bigrams/trigrams)
• Most positive/negative review
• Top positive/negative sentiments of users
• The trending/shrinking words
• Users' main cities

Screenshot

Supported Languages

• English
• Italian

Built Using

• Kivy - GUI
• Beautiful Soup - HTML scraping
• R - Sentiment Analysis

Acknowledgements

• @susanli2016 - Web Scraping TripAdvisor
• TextWiller - For providing Italian stop words and lexicon
• All the other packages - Thank you for being you

Disclaimer

Scrapeadvisor is provided under this License on an AS-IS basis, without warranty of any kind, either expressed, implied, or statutory, including, without limitation, warranties that the scrapeadvisor is free of defects, merchantable, fit for a particular purpose or non-infringing.

To the extent permitted under Law, scrapeadvisor is provided under an AS-IS basis. The scrapeadvisor Team shall never, and without any limit, be liable for any damage, cost, expense or any other payment incurred as a result of scrapeadvisor's actions, failure, bugs and/or any other interaction between scrapeadvisor and end-equipment, computers, other software or any 3rd party, end-equipment, computer or services.

We do not encourage running scrapeadvisor against Tripadvisor without prior mutual consent. The scrapeadvisor Team accept no liability and are not responsible for any misuse or damage caused by scrapeadvisor.