This project demonstrates how to perform asynchronous web scraping using FastAPI, aiohttp, and BeautifulSoup. The application scrapes quotes from two websites: Quotes to Scrape and Goodreads Quotes.
- Python 3.8+
- FastAPI
- aiohttp
- BeautifulSoup4
- Requests
-
Clone the repository:
git clone https://github.com/Petsamuel/Quotes.git
-
Create a virtual environment and activate it:
python -m venv venv source venv/bin/activate # On Windows use `venv\Scripts\activate`
-
Install the dependencies:
pip install -r requirements.txt
-
Start the FastAPI server:
uvicorn main:app --reload
-
Open your browser and navigate to
http://127.0.0.1:8000/
to see the scraped quotes.
- main.py: The main FastAPI application file that handles the asynchronous scraping.
- requirements.txt: Contains all the dependencies
The application scrapes quotes from the specified URLs and returns them as a JSON response. The scraping is done asynchronously to improve performance and handle multiple requests concurrently.
This project is licensed under the MIT License. See the LICENSE file for details.