iOS: Terminated due to memory issue

[Curvel]

After uploading one or multiple files the app crashes due to a memory issue. This happens at around 1:30 min. after the upload is started/finished (not sure).

The only log I get is: Message from debugger: Terminated due to memory issue

On my iPad it is reproducible 100% of the time.

Commit: dbdc47d33905cc05ae939ab2d5bec2e678923703

My three plist keys are: FUMaximumConnectionsPerHost: 1 FUMaximumUploadOperation: 1 FUTimeoutInSeconds: 3600

My call:

taskId = await FlutterUploader().enqueue(
        url: url,
        files: [FileItem(path: file.path)],
        method: UploadMethod.POST,
        data: data,
        headers: headers,
      );

Device Info:

• OS Version: iPhone OS 13.5 (Build 17G68)
• Hardware model: iPad7,5

[ened]

How big is the file, roughly?

[Curvel]

It's just a tiny .pages file with 100KB

[MICKEY88661]

I have the same issue and it's five 20Mb picture

[joshoconnor89]

this is a really bad issue theres some sort of memory leak going on

[ened]

@joshoconnor89 @Curvel please retry this with the latest beta version 2.0.0-beta.5. If you can still reproduce it, please provide a sample app.

[trueb2]

I still see what appears to be a memory leak on 3.0.0-beta.3. I have plenty of successful uploads for a minute or two, then start ramping up memory usage

Most RawUploads are between 100B and 4KB

URLSessionDidSendBodyData: chillisource.flutter_uploader.upload.background.0FF9B967-1FEA-4158-AC81-69A3D0342895, byteSent: 132, totalBytesSent: 132, totalBytesExpectedToSend: 132, progress:100.0
URLSessionDidReceiveData:
URLSessionDidCompleteWithError: chillisource.flutter_uploader.upload.background.0FF9B967-1FEA-4158-AC81-69A3D0342895 with response: <NSHTTPURLResponse: 0x28307f180> { URL: http://localhostnamegoeshere.private:8000/v1/stream/foo/bar } { Status Code: 200, Headers {
    "Content-Length" =     (
        1
    );
    "Content-Type" =     (
        "application/json"
    );
    Date =     (
        "Tue, 07 Dec 2021 23:02:27 GMT"
    );
} } and status: 200
URLSessionDidCompleteWithError: upload completed
URLSessionDidCompleteWithError: response: 6, task: completed
* thread #1, queue = 'com.apple.main-thread', stop reason = EXC_RESOURCE RESOURCE_TYPE_MEMORY (limit=1850 MB, unused=0x0)
    frame #0: 0x00000001bda8cd40 libobjc.A.dylib`AutoreleasePoolPage::AutoreleasePoolPage(AutoreleasePoolPage*) + 88
libobjc.A.dylib`AutoreleasePoolPage::AutoreleasePoolPage:
->  0x1bda8cd40 <+88>:  str    w12, [x19, #0xc]
    0x1bda8cd44 <+92>:  adrp   x12, 17
    0x1bda8cd48 <+96>:  add    x12, x12, #0xe85          ; =0xe85
    0x1bda8cd4c <+100>: str    w22, [x19]
Target 0: (Runner) stopped.
Lost connection to device.
Exited (sigterm)

From a profile build (and debug looks about the same), I can see memory holding steady without the enqueue calls. With the enqueue calls, the RSS memory builds up and periodically returns to baseline. Eventually, it builds up to some iOS cutoff and the app is terminated

[trueb2]

I could be wrong but it seems like there is no mechanism to revoke a completed task from the cached stream handler (used by the progress and result event streams). Since these methods may receive multiple calls for each upload task would they always grow in size? I would expect to generate hundreds of uploads per minute with my code that exhibits the problem.

Looking at https://github.com/fluttercommunity/flutter_uploader/blob/main/ios/Classes/CachingStreamHandler.swift#L10 The cache is a [String: [String: Any]], but I am not familiar with the code enough to know how that should grow. The SwiftFlutterUploaderPlugin seems like it would be updating this map whenever there is a state change for result or progress

There is an add and a clear method but no remove or dequeue method for this cache. The clearUploads hook isn't a realistic option to prevent memory pressure issues; however, if I call clearUploads prior to enqueue, then the memory usage is fine.

[trueb2]

Canceling every upload on the first result callback seems to work well so far. It doesn't require the clearUploads call either. I will post if I see some other behavior.

I am using FlutterUploader instance like this now:

// One-time listener installation to cancel any tasks and delete the uploaded file
uploader!.result.listen((result) {
  final path = pendingUploadTasks[result.taskId];
  if (path != null) {
    File(path).delete();
    pendingUploadTasks.remove(result.taskId);
    uploader!.cancel(taskId: result.taskId);
  }
});
....
// Enqueue a file and save the taskId and path key-value pair
final file = File('adsf');
await file.writeAsBytes(bufferedBytes, mode: FileMode.write, flush: true);
final taskId = await uploader!.enqueue(
  RawUpload(
    url: 'https://foo.bar/v1/upload',
    path: file.path,
  ),
pendingUploadTasks[taskId] = file.path;

[ened]

You're right that CachingStreamHandler may be the issue here. I'll investigate. Thx @trueb2