Sorry for the long devlog it took me a bit to set up the flavortown project. So I built a custom facebook marketplace scraper for scraping cars.
How it works:
You have to do a bit of setup in settings.py for it to work, but all of it is well documented with both python comments and in the readme.
First, you run scraper.py. This opens chrome in a headless mode and goes to the marketplace url provided., scrolls down and every 3 scrolls runs runs a selector and saves all the cars that it finds to a database, along with the metadata mileage, url, title, year, location, and price.
To filter the results, you configure settings.py with the filters you want, then run search.py. This will filter through all the cars in the database, and can even send an email if it finds one it hasn’t found before. I set it up with windows task scheduler to run every day since I’m looking for a car.
The whole thing is really well documented (or so I think) and is relatively easy to use. I’m still working on making the email part of it easier to set up, and I need to add an example secrets.py.
The filters are pretty robust and hopefully I’ll add more categories of items along with more db tables to expand it beyond just cars.