twitter-scraper icon indicating copy to clipboard operation
twitter-scraper copied to clipboard

Unable to get a user replies

Open louisgrasset opened this issue 2 years ago • 5 comments

I'm trying to get the lastest 50 tweets (expecting tweets and replies) but it sounds like I'm only able to get:

  • tweets
  • retweets.

I'm leveraging getTweets('userHandle', 50) to do so, no auth required. Am I missing something? Is this a bug or an expected behavior?

From what I see, Twitter isn't allowing non-authenticated webapp to access replies, may this be the reason?

Twitter Webapp not returning replies

louisgrasset avatar Aug 12 '23 08:08 louisgrasset

I believe that's probably what's happening — I'll have to check if we get more data when authenticated via the API method that uses.

karashiiro avatar Aug 12 '23 19:08 karashiiro

I'd consider this a bug for now, but given that we're at the whims of ~~the company formerly known as~~ Twitter, this might have to become the new expected behavior, at least while unauthenticated.

karashiiro avatar Aug 12 '23 19:08 karashiiro

I'd consider this a bug for now, but given that we're at the whims of ~the company formerly known as~ Twitter, this might have to become the new expected behavior, at least while unauthenticated.

It seems like even while authenticated replies don't show when using getTweets

whynotmarc avatar Aug 12 '23 19:08 whynotmarc

You may be totally true, when I discovered that issue, I moved my code to use the following: searchTweets(`from:@${handle}`, 50, SearchMode.Latest)

louisgrasset avatar Aug 13 '23 08:08 louisgrasset

With authentication you can use fetchSearchTweets. Something along the lines of this:

  const tweets = []
  const maxTweets = 50
  const query = `(from:${userHandle}) filter:replies`
  
  let cursor
  let nTweets = 0
  while (nTweets < maxTweets) {
    const res = await scraper.fetchSearchTweets(
      query,
      maxTweets,
      SearchMode.Latest,
      cursor
    )

    if (res.tweets.length === 0) break

    nTweets += res.tweets.length
    cursor = res.next
    tweets.push(...res.tweets)
  }

whynotmarc avatar Aug 13 '23 08:08 whynotmarc