Some posts/messages are being skipped

[yourfavoriteweapon]

When I initially set this up, everything worked great. Now I've noticed it seems to be skipping content at times. There was a Paid message I unlocked, that was 2 images and a video. The downloader only picked up the first image and the video, not the 2nd one.

I went to another profile and noticed there were a few recent free posts that it didn't download. If i tried to force it to download by choosing single post and putting in that URL, the tool said it had already downloaded that post.

Logs dont have that much in them, is there a known issue going on?

[naodrej]

perhaps it's a dupe and the model reposted content.

[jallisan]

I have this exact same issue, there is a creator who has uploaded messages with many videos / photos in but when I try download them it only seems to get the first one that shows up in the browser. Attached my config.json but I've tried to change everything I could think of and still no cigar.

[tj961120]

Following, I'm experiencing the same issues. A lot of pics and vids from messages being skipped for some reason...

[DP9aq]

Following too. I get those messages, even after i start whit an empty installation (so there should be no downloaded posts in the database) is that the same Issue ?

[sim0n00ps]

I have this exact same issue, there is a creator who has uploaded messages with many videos / photos in but when I try download them it only seems to get the first one that shows up in the browser. Attached my config.json but I've tried to change everything I could think of and still no cigar.

Your Filename Formats are not unique enough, essentially using a format like that will mean every file for that post will try to be named the same so they will be overwritten so you're left with 1 file per post. Maybe include the media id which will make each filename unique.

[DP9aq]

Today i tried to scrap the whole media from one file - there was an error and the OF-DL didn't find any posts. Maybe the issue is related in the mechanism the identification of the single posts are made ?

[jallisan]

I have this exact same issue, there is a creator who has uploaded messages with many videos / photos in but when I try download them it only seems to get the first one that shows up in the browser. Attached my config.json but I've tried to change everything I could think of and still no cigar.

Your Filename Formats are not unique enough, essentially using a format like that will mean every file for that post will try to be named the same so they will be overwritten so you're left with 1 file per post. Maybe include the media id which will make each filename unique.

Thanks @sim0n00ps this was my issue, been scratching my head for ages at this, had't occurred to me that when I reset everything I used the same filenames again.

Thanks as well for your hard work on this app, its second to none.

[DP9aq]

Hey there, im using the following file formats "PaidPostFileNameFormat": "{postedAt}-{username}_{id}_{text}", "PostFileNameFormat": "{postedAt}-{username}_{id}_{text}",

That should be unique enough or ? Still getting those 403 Errors

[tj961120]

I left my file format blank though... Any idea what might be causing this issue?

@sim0n00ps cannot thank you enough for this

[DP9aq]

Just tried again with a blank file format, same Error Maybe the Log file helps?

`2024-05-20 17:21:04.630 +02:00 [ERR] Auth failed 2024-05-20 17:22:02.853 +02:00 [ERR] Exception caught: Response status code does not indicate success: 403 (Forbidden).

StackTrace: at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode() at OF_DL.Helpers.APIHelper.GetDRMMPDPSSH(String mpdUrl, String policy, String signature, String kvp, Auth auth) 2024-05-20 17:26:55.664 +02:00 [ERR] Exception caught: Response status code does not indicate success: 403 (Forbidden).

StackTrace: at System.Net.Http.HttpResponseMessage.EnsureSuccessStatusCode() at OF_DL.Helpers.APIHelper.GetDRMMPDPSSH(String mpdUrl, String policy, String signature, String kvp, Auth auth) `

Any Ideas @sim0n00ps ?

[melithine]

403s and file formats are two separate issues. I reco,mend including the mediaId in the filename. For the 403 issue, try with the most recent version and with/without a VPN.

[4DeepThought2]

Same issue. Skips posts that have multiple media in it. Only downloads a single video or image.

[melithine]

Same issue. Skips posts that have multiple media in it. Only downloads a single video or image.

Post your config.json.

[4DeepThought2]

Post your config.json.

{
  "DownloadAvatarHeaderPhoto": false,
  "DownloadPaidPosts": true,
  "DownloadPosts": true,
  "DownloadArchived": true,
  "DownloadStreams": true,
  "DownloadStories": true,
  "DownloadHighlights": true,
  "DownloadMessages": true,
  "DownloadPaidMessages": true,
  "DownloadImages": true,
  "DownloadVideos": true,
  "DownloadAudios": true,
  "IncludeExpiredSubscriptions": true,
  "IncludeRestrictedSubscriptions": false,
  "SkipAds": false,
  "DownloadPath": "",
  "RenameExistingFilesWhenCustomFormatIsSelected": false,
  "Timeout": null,
  "FolderPerPaidPost": false,
  "FolderPerPost": false,
  "FolderPerPaidMessage": false,
  "FolderPerMessage": false,
  "LimitDownloadRate": false,
  "DownloadLimitInMbPerSec": 4,
  "DownloadOnlySpecificDates": false,
  "DownloadDateSelection": "after",
  "CustomDate": null,
  "ShowScrapeSize": true,
  "DownloadPostsIncrementally": true,
  "NonInteractiveMode": false,
  "NonInteractiveModeSitesName": "",
  "NonInteractiveModePurchasedTab": false,
}

I've tried just the flat base config as well.

[melithine]

That doesn't look like the whole file, as that last comma should break the JSON validation. Beyond that, try doing a fresh scrape for one of the users and attach the log file so we can see if there are any errors getting logged.

[4DeepThought2]

That doesn't look like the whole file, as that last comma should break the JSON validation. Beyond that, try doing a fresh scrape for one of the users and attach the log file so we can see if there are any errors getting logged.

Ok. I copied the flat config.json back over. Something is certainly wrong. There are some accounts with 1000s of posts, and its downloading like 1.

{
  "DownloadAvatarHeaderPhoto": true,
  "DownloadPaidPosts": true,
  "DownloadPosts": true,
  "DownloadArchived": true,
  "DownloadStreams": true,
  "DownloadStories": true,
  "DownloadHighlights": true,
  "DownloadMessages": true,
  "DownloadPaidMessages": true,
  "DownloadImages": true,
  "DownloadVideos": true,
  "DownloadAudios": true,
  "IncludeExpiredSubscriptions": false,
  "IncludeRestrictedSubscriptions": false,
  "SkipAds": false,
  "DownloadPath": "",
  "PaidPostFileNameFormat": "",
  "PostFileNameFormat": "",
  "PaidMessageFileNameFormat": "",
  "MessageFileNameFormat": "",
  "RenameExistingFilesWhenCustomFormatIsSelected": false,
  "Timeout": null,
  "FolderPerPaidPost": false,
  "FolderPerPost": false,
  "FolderPerPaidMessage": false,
  "FolderPerMessage": false,
  "LimitDownloadRate": false,
  "DownloadLimitInMbPerSec": 4,
  "DownloadOnlySpecificDates": false,
  "DownloadDateSelection": "after",
  "CustomDate": "",
  "ShowScrapeSize": false,
  "DownloadPostsIncrementally": false,
  "FFmpegPath": "",
  "NonInteractiveMode": false,
  "NonInteractiveModeListName": "",
  "NonInteractiveModePurchasedTab": false,
}

[melithine]

Is this still happening on current release? If so, need a log with debug or verbose mode enabled.