Python Flask Scraping Integration Fix and Improvements
Employer not named by the sourceRemote
Frontier is not the employer and does not collect applications.
About this role
JavaScript, Python, Web Scraping, Flask, Reverse Engineering, Software Engineering · Project Overview I have an existing Python / Flask application using Scrapfly for web scraping.
The application provides three main API flows:
Search destinations Search hotels for a selected destination. A separate working demo/proof of concept is available and successfully extracts the required data. Search hotels around geographic coordinates The current scraping integration stopped working due to upstream antibot and changes.
The main goal is to restore the existing integration and make targeted improvements to session handling, provider architecture, configuration, testing, and documentation.
This is an incremental fix and improvement project, not a full rewrite.
Main Scope The project includes:
Restore the three main scraping/API flows
Fix request setup, headers, referrer, cookies, and session state where needed
Improve session pooling, reuse, retries, expiration, cleanup, and persistence
Improve provider abstraction / Inversion of Control
Keep Scrapfly behind a clear provider interface
Add configurable language and currency
Default language: English Default currency: EUR Add meaningful unit and integration/end-to-end tests
Improve relevant logging and technical