linkedin_scraper
linkedin_scraper copied to clipboard
[FIX] Update LinkedIn scraper for current DOM structure, Selenium 4.x compatibility, Dev enhancements (UV, Ruff, pre-commit)
Hey, I am using this Repository for my LinkedIn MCP Server, which wraps the provided scraping functions of this repository with tools that LLMs can access. As users experience issues with the provided tools, here is a list of changes that make this project more usable again and simplify future contributions.
Changes
Issues
- Fixed deprecated Selenium methods (find_element_by_* → find_element(By.*)) #190
- Updated CSS selectors for the current LinkedIn DOM structure:
- job-card-container (jobs)
- artdeco-entity-lockup__subtitle (company names)
- job-card-container__link (job titles)
- Resolved stale element reference issues in job scraping
- Implemented error handling/fallback selectors
- Refactored recommended jobs logic to work with the current page structure
Examples Added
- Job search with keyword filtering
- Recommended jobs from the LinkedIn homepage
- Individual job details scraping
- Company profile and people extraction
- Person contacts and profile data
Testing
All scraping functionality from the added examples works.
Dev Enhancements
- Update the project to use UV package manager instead of pip
- added ruff (linting & formatting)
- added pre-commit hook