[BUG] Could not get historical market data for bitcoin (COINGECKO) when the time range is more than one year

[swissbuechi]

Bug Description

Not able to gather historical data for bitcoin via coingecko. I have existing activities starting from 2021-10-17 all the way to 2024. When I delete all activities from 2021 till 2023, the historical market data gathering works fine.

To Reproduce

1. Create activity to buy bitcoin in 2021
2. Create activity to buy bitcoin in 2024
3. Click on "Gather Historical Data" (bitcoin) -> Error (See logs)
4. Delete buy bitcoin activity from 2021
5. Click on "Gather Historical Data" (bitcoin) -> Works

Expected behavior

The market data should also be captured if the range is more than one year. Maybe request the data in junks of max one year?

Logs

[Nest] 109  - 04/01/2024, 12:14:08 PM     LOG [DataGatheringProcessor (GATHER_HISTORICAL_MARKET_DATA)] Historical market data gathering has been started for bitcoin (COINGECKO) at 2021-10-17
[Nest] 109  - 04/01/2024, 12:14:08 PM   ERROR [DataProviderService] Error: Could not get historical market data for bitcoin (COINGECKO) from 2021-10-17 to 2024-04-01: [HTTPError] Response code 401 (Unauthorized)
[Nest] 109  - 04/01/2024, 12:14:08 PM     LOG [DataGatheringProcessor (GATHER_HISTORICAL_MARKET_DATA)] Historical market data gathering has been completed for bitcoin (COINGECKO) at 2024-04-01
[Nest] 109  - 04/01/2024, 12:14:12 PM     LOG [DataGatheringProcessor (GATHER_HISTORICAL_MARKET_DATA)] Historical market data gathering has been started for bitcoin (COINGECKO) at 2021-10-17
[Nest] 109  - 04/01/2024, 12:14:12 PM   ERROR [DataProviderService] Error: Could not get historical market data for bitcoin (COINGECKO) from 2021-10-17 to 2024-04-01: [HTTPError] Response code 401 (Unauthorized)
[Nest] 109  - 04/01/2024, 12:14:12 PM     LOG [DataGatheringProcessor (GATHER_HISTORICAL_MARKET_DATA)] Historical market data gathering has been completed for bitcoin (COINGECKO) at 2024-04-01
[Nest] 109  - 04/01/2024, 12:14:19 PM     LOG [DataGatheringProcessor (GATHER_HISTORICAL_MARKET_DATA)] Historical market data gathering has been started for bitcoin (COINGECKO) at 2021-10-17
[Nest] 109  - 04/01/2024, 12:14:19 PM   ERROR [DataProviderService] Error: Could not get historical market data for bitcoin (COINGECKO) from 2021-10-17 to 2024-04-01: [HTTPError] Response code 401 (Unauthorized)
[Nest] 109  - 04/01/2024, 12:14:19 PM     LOG [DataGatheringProcessor (GATHER_HISTORICAL_MARKET_DATA)] Historical market data gathering has been completed for bitcoin (COINGECKO) at 2024-04-01
[Nest] 109  - 04/01/2024, 12:14:23 PM     LOG [DataGatheringProcessor (GATHER_HISTORICAL_MARKET_DATA)] Historical market data gathering has been started for bitcoin (COINGECKO) at 2021-10-17
[Nest] 109  - 04/01/2024, 12:14:23 PM   ERROR [DataProviderService] Error: Could not get historical market data for bitcoin (COINGECKO) from 2021-10-17 to 2024-04-01: [HTTPError] Response code 401 (Unauthorized)
[Nest] 109  - 04/01/2024, 12:14:23 PM     LOG [DataGatheringProcessor (GATHER_HISTORICAL_MARKET_DATA)] Historical market data gathering has been completed for bitcoin (COINGECKO) at 2024-04-01
[Nest] 109  - 04/01/2024, 12:14:27 PM     LOG [DataGatheringProcessor (GATHER_HISTORICAL_MARKET_DATA)] Historical market data gathering has been started for bitcoin (COINGECKO) at 2021-10-17
[Nest] 109  - 04/01/2024, 12:15:27 PM   ERROR [DataProviderService] Error: Could not get historical market data for bitcoin (COINGECKO) from 2021-10-17 to 2024-04-01: [RequestError] The operation was aborted
[Nest] 109  - 04/01/2024, 12:15:27 PM     LOG [DataGatheringProcessor (GATHER_HISTORICAL_MARKET_DATA)] Historical market data gathering has been completed for bitcoin (COINGECKO) at 2024-04-01

Environment

• Ghostfolio Version 2.69.0
• Self-hosted: Azure Kubernetes Service
• Experimental Features: disabled
• Browser: Edge
• OS: macOS

[karter16]

Can confirm I'm having the same issue (self-hosted, 2.70.0).

[swissbuechi]

Improved the usability of the date range support by specific years (2023, 2022, 2021, etc.) in the assistant (experimental)

Maybe this new feature from release 2.70 could fix the issue.

I will try and report back.

[karter16]

The assistant is purely for filtering is my understanding, it shouldn't have any bearing on things like retrieving historical data.

[swissbuechi]

I ended up manually importing a .csv with the recent three years of market data, which I've downloaded from Yahoo Finance.

date,marketPrice
2021-10-27,58482.386719
2021-10-28,60622.136719
2021-10-29,62227.964844
...

[danipolo]

I'm having the same issue. I see the following log but no historical data pulled:

[Nest] 163  - 07/16/2024, 1:52:17 AM     LOG [DataGatheringProcessor (GATHER_HISTORICAL_MARKET_DATA)] Historical market data gathering has been started for bitcoin (COINGECKO) at 2024-07-16
[Nest] 163  - 07/16/2024, 1:52:17 AM     LOG [DataGatheringProcessor (GATHER_HISTORICAL_MARKET_DATA)] Historical market data gathering has been completed for bitcoin (COINGECKO) at 2024-07-16
[Nest] 163  - 07/16/2024, 1:52:21 AM     LOG [DataGatheringProcessor (GATHER_ASSET_PROFILE)] Asset profile data gathering has been started for bitcoin (COINGECKO)
[Nest] 163  - 07/16/2024, 1:52:21 AM     LOG [DataGatheringProcessor (GATHER_ASSET_PROFILE)] Asset profile data gathering has been completed for bitcoin (COINGECKO)

I configured the Coingecko API on the env but no success.

[alexandrescieux]

I think the 1 year limit is linked to the API limitations of Coingecko's Demo plan : https://www.coingecko.com/en/api/pricing

[swissbuechi]

I think the 1 year limit is linked to the API limitations of Coingecko's Demo plan :

https://www.coingecko.com/en/api/pricing

You're right!

Thank you.