Build Django Website Data Scraper
Employer not named by the sourceRemote
Frontier is not the employer and does not collect applications.
About this role
Python, Django, Software Architecture, NoSQL Couch & Mongo, PostgreSQL, SQLite, Data Scraping, BeautifulSoup · I need a Python developer to craft a self-contained scraper inside a lightweight Django app for our client’s own website. Phase 1 is all about reliability and clarity, not bells and whistles.
Scope of work • Create a fresh Django project with a single command (e.g., python manage.py scrape) that navigates the distributed pages of the site, gathers the required text-based content, and captures any embedded timestamps or IDs that help with later analysis. • Persist every record directly to the default SQLite database through Django models—no external queues, no Celery. • Make the scraper idempotent: re-running it should update or skip existing entries rather than duplicate them. • Keep the code modular and well-commented so we can swap in PostgreSQL and add scheduling in the next phase. • Deliver a concise README that covers environment setup, model definitions, and the command’s expected runtime.
Success looks like running one command and ending up with a clean, query-ready dataset our analysts can immediately explore.
If you have proven experience with Django, BeautifulSoup, Requests or similar libraries, and can meet these requirements quickly and cleanly, let’s talk ti