bitcoin
Memory leak via API listtransactions
Hello. I use a bitcoind with a large wallet containing about 150K transactions. Each time the method listtransactions is used the memory consumed by the process grows 100MB. As a result it can take up all possible memory and ends with a kill signal.
System information: Bitcoin Core 23.0 CentOS 8.4 CPU Cores: 12 RAM: 64GB SSD: 1TB
I tried to reproduce this (Ubuntu 20.04.4 LTS) with a large wallet but could not see a memory leak (also not with valgrind).
What I did see is that repeated calls to listtransactions would lead to a step up in the memory claimed by bitcoind (as seen by top or pmap) that would not be released by the OS (enhanced by the fact that the memory for the large RPC response was reserved twice, fixed in #25237)
But after ~5 calls to listtransactions the memory of bitcoind would not increase any further for me with more listtransactions calls.
My application is querying listtransactions "*" 100000000 about once every 1 minute to retrieve a list of all transactions. The request takes about 10 seconds. After each call there is an increase in the consumed memory. Sometimes memory clearing is observed but this is useless and after 2-3 hours bitcoind dies with a kill signal from the OS.
I also additionally checked after 5 or more calls the memory also grows by 100+ MB.
Bitcoin Core 23.0
@CWDT250 are you able to test the behaviour using a build of the master branch?
I could reproduce a memory increase in the system monitor, but not in massif.
I also tried to reproduce the memory leak (in master 57c192767b0e24e51f2f777668ba6f5173f8960d). I ran listtransactions a lot of times on a large wallet (54,000 txs, some coinbases, some regular transactions). The first 4-5 calls increased the memory usage of bitcoind but then it stabilizes and didn't increase RAM usage, I'm guessing that could be cache?
I have a large wallet with 515K transactions. When I'm calling listunspent the memory increases after a new call. After 10 times the bitcoin server crashes.
listunspent does not return the number of transactions, but the number of coins available for the wallet.
How many items does this listunspent response return?
@w0xlt the output has 5662758 lines, equivalent to ~515K transactions. My point is that the same behavior exists in listunspent and not only in listransactions.