University Program Data Scraper
Employer not named by the sourceRemote
Full StackEntry-level and new-grad engineering jobs
Frontier is not the employer and does not collect applications.
About this role
Python, Data Processing, Web Scraping, Software Architecture, Data Mining, Scrapy, Data Extraction, BeautifulSoup · I need a reliable scraper that can automatically collect up-to-date course details from a list of university program pages and export them in a clean, structured format (CSV or JSON is fine).
Scope • Target data: course description, course schedule, and course prerequisites only—no faculty info or admission requirements. • Update cycle: the dataset must refresh right before every new intake period (e.g., each semester or trimester), so the script should be easy for me to rerun or schedule. • Sources: I will supply the university URLs; you handle respectful crawling, pagination, and any anti-bot measures. • Output: a consolidated file plus the reusable scraping script (Python with BeautifulSoup/Scrapy or similar). Include brief setup instructions.
Acceptance • All requested fields populated for each course. • No duplicate rows; consistent field names. • Script runs on a standard desktop environment without paid APIs.
If you’ve tackled higher-ed sites before, mention it—otherwise just outline your approach, estimated turnaround, and any libraries you prefer.