linkedin_scraper icon indicating copy to clipboard operation
linkedin_scraper copied to clipboard

[FIX] Update LinkedIn scraper for current DOM structure, Selenium 4.x compatibility, Dev enhancements (UV, Ruff, pre-commit)

Open stickerdaniel opened this issue 9 months ago • 0 comments

Hey, I am using this Repository for my LinkedIn MCP Server, which wraps the provided scraping functions of this repository with tools that LLMs can access. As users experience issues with the provided tools, here is a list of changes that make this project more usable again and simplify future contributions.

Changes

Issues

  • Fixed deprecated Selenium methods (find_element_by_* → find_element(By.*)) #190
  • Updated CSS selectors for the current LinkedIn DOM structure:
    • job-card-container (jobs)
    • artdeco-entity-lockup__subtitle (company names)
    • job-card-container__link (job titles)
  • Resolved stale element reference issues in job scraping
  • Implemented error handling/fallback selectors
  • Refactored recommended jobs logic to work with the current page structure

Examples Added

  • Job search with keyword filtering
  • Recommended jobs from the LinkedIn homepage
  • Individual job details scraping
  • Company profile and people extraction
  • Person contacts and profile data

Testing

All scraping functionality from the added examples works.

Dev Enhancements

  • Update the project to use UV package manager instead of pip
  • added ruff (linting & formatting)
  • added pre-commit hook

stickerdaniel avatar Jul 04 '25 19:07 stickerdaniel