Weekly Updated Menu Scraper
Employer not named by the sourceRemote
Frontier is not the employer and does not collect applications.
About this role
Python, Web Scraping, MySQL, PostgreSQL, Elasticsearch, Scrapy, BeautifulSoup, Database Management · I’m building an online directory similar to menuswithprice.com and need an automated pipeline that gathers data from roughly 6–10 external sites. The directory must hold three kinds of records at launch—restaurant menus, product prices, and service listings—and refresh the full data set on a weekly schedule.
Here’s the flow I have in mind: scraper scripts (Python with Scrapy, BeautifulSoup, or a comparable framework) pull structured information from each source, normalise it, and load it into a database that powers the public-facing site. A cron job or cloud function should trigger the weekly update, logging what changed so I can spot anomalies quickly. Duplicate detection, polite rate-limiting, and respect for robots.txt are essential.
Deliverables • Clean, well-commented scraping code for every target site • Normalisation routine that maps each source’s fields to a unified schema • Database (MySQL, Postgres, or NoSQL—recommend what fits best) seeded with the first full import • Automated weekly update job with simple reporting (e-mail or dashboard) • Brief setup guide so I can deploy or extend the scraper myself
Acceptance criteria The directory pages must load compl