lean-cli icon indicating copy to clipboard operation
lean-cli copied to clipboard
verified publisher icon QuantConnect

*** PermissionError: [Errno 13] Permission denied: '/LeanCLI/__pycache__'

Open shawnngtq opened this issue 1 year ago • 5 comments

Context

I'm testing lean-cli using starter code, during backtest encounter *** PermissionError: [Errno 13] Permission denied: '/LeanCLI/pycache'

Setup

# confirm login
x lean whoami
You are logged in as myUsername ([email protected])

# create project
x lean project-create test_qc

# directory
x ls
data  lean.json  storage  test_qc

Error during backtest

x lean backtest test_qc --verbose
Label 'strict_python_version' found in image 'quantconnect/lean', value 3.11.7
Label 'python_version' found in image 'quantconnect/lean', value 3.11
Label 'target_framework' found in image 'quantconnect/lean', value net6.0
Label 'lean_version' found in image 'quantconnect/lean', value 16712
Context information:
  Python version: 3.12.3 | packaged by conda-forge | (main, Apr 15 2024, 18:38:13) [GCC 12.3.0]
  OS: Linux-5.15.0-124-generic-x86_64-with-glibc2.35
  Lean CLI version: 1.0.205
  .NET version: Not installed
  VS Code version: 1.94.2-384ff7382de624fb94dbaf6da11977bba1ecd427-x64
  VS Code installed versions: [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
                              [email protected]
--> POST https://www.quantconnect.com/api/v2/organizations/list
Request response: 
{"organizations":[{"id":"myOrganizationId","name":"myUsername","type":"Free","ownerName":"You","members":1,"selected":false,"skylightSync":1,"preferred":true,"profileImage":"https:\/\/cdn.quantconnect.com\/i\/tu\/o
rganization-default-icon.png"}],"success":true}
--> POST https://www.quantconnect.com/api/v2/organizations/read with data:
{
    "organizationId": "myOrganizationId"
}
Request response: 
{"organization":{"id":"myOrganizationId","allowSkylightSync":1,"seats":1,"type":"free","billing":null,"credit":{"movements":[],"movementsCount":0,"balance":0,"automaticCreditUsage":0,"automaticCreditTopUp":0,"
creditTopUpMin":500,"creditTopUpAmount":2000},"subscriptions":[{"id":524828,"name":"B-MICRO","price":0,"start":"2024-10-14 10:02:49"},{"id":524829,"name":"R-MICRO","price":0,"start":"2024-10-14 
10:02:49"}],"products":[{"name":"Backtest Node","icon":"https:\/\/cdn.quantconnect.com\/i\/tu\/backtest-node-icon-v1.0.png","items":[{"productId":30,"name":"B-MICRO","quantity":1,"unitPrice":0,"total":0}]},{"name":"Research 
Node","icon":"https:\/\/cdn.quantconnect.com\/i\/tu\/research-node-icon-v1.0.png","items":[{"productId":29,"name":"R-MICRO","quantity":1,"unitPrice":0,"total":0}]}],"invoices":[],"invoicesCount":0,"about":"","name":"myUsername","s
upportSeats":{"bronze":{"seats":0,"unassigned":0},"silver":{"seats":0,"unassigned":0},"gold":{"seats":0,"unassigned":0}},"shortDescription":"Write a short 
description","statistics":[{"name":"ComputeHours","value":"-"},{"name":"Lines","value":"20"},{"name":"Projects","value":"1"},{"name":"Backtests","value":"1"},{"name":"Members","value":"1"}],"profileImage":"https:\/\/cdn.quant
connect.com\/i\/tu\/organization-default-icon.png","useResourcesListView":false,"useMembersListView":false,"isUsingPaidFeatures":false,"isAbleToDowngrade":true,"nodes":{"backtest":[{"speed":3,"price":{"monthly":0,"yearly":0},
"cpu":1,"ram":7.7999999999999998,"name":"Community B-MICRO","sku":"B-MICRO","description":"1 CPU @ 3.0GHz, 7.8GB 
Ram","usedBy":"-","userProfile":"","projectName":"","projectId":null,"busy":false,"id":"BN-67b249b967e2c078bd1a3d2b862e5c8e","assets":100,"host":null,"hasGpu":0}],"research":[{"speed":2.3999999999999999,"price":{"monthly":0,"
yearly":0},"cpu":1,"ram":2,"name":"Community R-MICRO","sku":"R-MICRO","description":"1 CPU @ 2.4GHz, 2GB 
Ram","usedBy":"-","userProfile":"","projectName":"","projectId":null,"busy":false,"id":"RN-aaa768b110fb79d5697138ba8a75d64f","assets":100,"host":null,"hasGpu":0}],"live":[]},"environment":{"files":0,"bytes":4096,"size":"4 
KB"},"data":{"signedTime":null,"current":false},"objectStore":[{"id":24,"name":"S-MICRO","price":0,"bytes":52428800,"files":1000,"storage":"50 
MB","active":true},{"id":25,"name":"S-10","price":12,"bytes":2147483648,"files":20000,"storage":"2 GB","active":false},{"id":26,"name":"S-20","price":24,"bytes":5368709120,"files":50000,"storage":"5 
GB","active":false},{"id":27,"name":"S-50","price":60,"bytes":10737418240,"files":100000,"storage":"10 GB","active":false},{"id":28,"name":"S-100","price":120,"bytes":53687091200,"files":500000,"storage":"50 
GB","active":false}],"objectStoreExportDerivative":false,"objectStoreHighestConsumers":[],"objectStorageUsed":4096,"objectStorageUsedHuman":"4 
KB","members":[{"id":313917,"name":"myUsername","isAdmin":true,"isRequester":true,"profile":"https:\/\/cdn.quantconnect.com\/web\/i\/users\/thumbnail\/100\/default.jpg","editable":false,"support":null,"linkedin":null,"lastLoginTim
estamp":"2024-10-27 00:47:00","email":"[email protected]","profileUrl":"\/u\/myUsername_13","publicId":"myUsername_13","online":true,"permissions":{"updateBilling":{"name":"Update 
Billing","value":true},"researchNodeStop":{"name":"Stop Research Nodes","value":true},"researchNodeCreate":{"name":"Create Research Nodes","value":true},"researchNodeDelete":{"name":"Delete Research 
Nodes","value":true},"liveNodeStop":{"name":"Stop Live Nodes","value":true},"liveNodeCreate":{"name":"Create Live Nodes","value":true},"liveNodeDelete":{"name":"Delete Live 
Nodes","value":true},"backtestNodeStop":{"name":"Stop Backtest Nodes","value":true},"backtestNodeCreate":{"name":"Create Backtest Nodes","value":true},"backtestNodeDelete":{"name":"Delete Backtest 
Nodes","value":true},"teamAdd":{"name":"Add Team Members to The Organization","value":true},"teamEdit":{"name":"Edit Team Members Permissions","value":false},"teamRemove":{"name":"Remove Team 
Members","value":true},"storageBilling":{"name":"Manage Storage Billing","value":true},"storageCreate":{"name":"Create Storage Files","value":true},"storageDelete":{"name":"Delete Storage 
Files","value":true}}}],"alpha":null,"courses":{"coursesCont":4,"enrolledMembers":0,"completedTasks":0,"courses":[{"id":1,"name":"Boot Camp 101 \/ US 
Equities","authorName":"QuantConnect","thumbnail":"https:\/\/cdn.quantconnect.com\/i\/tu\/bootcamp-101-thumbnail-image3.jpg","shortDescription":"Learn algorithmic trading with python for US Equities. Guided strategy 
development in easily digestible portions.","members":[]},{"id":2,"name":"Boot Camp 102 \/ 
FOREX","authorName":"QuantConnect","thumbnail":"https:\/\/cdn.quantconnect.com\/i\/tu\/bootcamp-102-thumbnail-image3.jpg","shortDescription":"Learn algorithmic trading with python for FX. Guided strategy development in easily
digestible portions.","members":[]},{"id":3,"name":"Boot Camp 103 \/ Futures","authorName":"QuantConnect","thumbnail":"https:\/\/cdn.quantconnect.com\/i\/tu\/bootcamp-103-thumbnail-image3.jpg","shortDescription":"Learn 
algorithmic trading with python for Futures. Guided strategy development in easily digestible portions.","members":[]},{"id":5,"name":"Algorithmic Trading A-Z | The Complete 
Course","authorName":"Louis","thumbnail":"https:\/\/cdn.quantconnect.com\/i\/tu\/python-for-algorithmic-trading-thumbnail-image.jpg","shortDescription":"In this algorithmic trading tutorial series you will learn everything 
you need to know to start writing your own trading bots using Python and the QuantConnect quantitative trading 
platform.","members":[]}]},"channel":"ORGac335894ff-myOrganizationId","minimumTransactionSizeLimit":0,"billingFrequency":"monthly","billingStatus":null,"billingPausedUntil":null,"hasCardOnFile":false,"joinOrga
nizationLink":"https:\/\/www.quantconnect.com\/invite?t=organization&o=myOrganizationId&k=a1b8343ada8ef23bbb2185de8d01cd191f81d9dd","outOfSampleDays":90,"trial":null,"contracts":{"institution":null}},"success"
:true}
logged in as myUsername ([email protected])
Label 'lean_version' found in image 'quantconnect/lean', value 16712
Found module 'local' from given name
Configuring Local
_update_settings(Local): Settings: {'id': 'Local', 'data-provider': 'QuantConnect.Lean.Engine.DataFeeds.DefaultDataProvider'}
LeanRunner().run_lean: lean_config: {'data-folder': 'data', 'log-handler': 'QuantConnect.Logging.CompositeLogHandler', 'messaging-handler': 'QuantConnect.Messaging.Messaging', 'job-queue-handler': 
'QuantConnect.Queues.JobQueue', 'api-handler': 'QuantConnect.Api.Api', 'map-file-provider': 'QuantConnect.Data.Auxiliary.LocalDiskMapFileProvider', 'factor-file-provider': 
'QuantConnect.Data.Auxiliary.LocalDiskFactorFileProvider', 'data-provider': 'QuantConnect.Lean.Engine.DataFeeds.DefaultDataProvider', 'data-channel-provider': 'DataChannelProvider', 'object-store': 
'QuantConnect.Lean.Engine.Storage.LocalObjectStore', 'data-aggregator': 'QuantConnect.Lean.Engine.DataFeeds.AggregationManager', 'symbol-minute-limit': 10000, 'symbol-second-limit': 10000, 'symbol-tick-limit': 10000, 
'show-missing-data-logs': False, 'maximum-warmup-history-days-look-back': 5, 'maximum-data-points-per-chart-series': 1000000, 'maximum-chart-series': 30, 'force-exchange-always-open': False, 'transaction-log': '', 
'reserved-words-prefix': '@', 'live-data-url': 'ws://www.quantconnect.com/api/v2/live/data/', 'live-data-port': 8020, 'live-cash-balance': '', 'live-holdings': '[]', 'ib-account': '', 'ib-user-name': '', 'ib-password': '', 
'ib-agent-description': 'Individual', 'ib-trading-mode': 'paper', 'ib-enable-delayed-streaming-data': False, 'ib-weekly-restart-utc-time': '22:00:00', 'tradier-environment': 'paper', 'tradier-account-id': '', 
'tradier-access-token': '', 'oanda-environment': 'Practice', 'oanda-access-token': '', 'oanda-account-id': '', 'fxcm-server': 'http://www.fxcorporate.com/Hosts.jsp', 'fxcm-terminal': 'Demo', 'fxcm-user-name': '', 
'fxcm-password': '', 'fxcm-account-id': '', 'iqfeed-iqconnect': 'C:/Program Files (x86)/DTN/IQFeed/iqconnect.exe', 'iqfeed-username': '', 'iqfeed-password': '', 'iqfeed-productName': '', 'iqfeed-version': '1.0', 
'coinbase-rest-api': 'https://api.coinbase.com', 'coinbase-url': 'wss://advanced-trade-ws.coinbase.com', 'coinbase-api-key': '', 'coinbase-api-secret': '', 'bitfinex-api-secret': '', 'bitfinex-api-key': '', 
'binance-api-secret': '', 'binance-api-key': '', 'binance-api-url': 'https://api.binance.com', 'binance-websocket-url': 'wss://stream.binance.com:9443/ws', 'binance-fapi-url': 'https://fapi.binance.com', 
'binance-fwebsocket-url': 'wss://fstream.binance.com/ws', 'binance-dapi-url': 'https://dapi.binance.com', 'binance-dwebsocket-url': 'wss://dstream.binance.com/ws', 'binanceus-api-secret': '', 'binanceus-api-key': '', 
'binanceus-api-url': 'https://api.binance.us', 'binanceus-websocket-url': 'wss://stream.binance.us:9443/ws', 'bybit-api-secret': '', 'bybit-api-key': '', 'bybit-api-url': 'https://api.bybit.com', 'bybit-websocket-url': 
'wss://stream.bybit.com', 'eze-api-address-url': '', 'eze-port': '', 'eze-domain': '', 'eze-locale': '', 'eze-password': '', 'eze-user-name': '', 'kraken-api-secret': '', 'kraken-api-key': '', 'kraken-verification-tier': 
'Starter', 'wolverine-host': '', 'wolverine-port': '', 'wolverine-account': '', 'wolverine-sender-comp-id': '', 'wolverine-target-comp-id': '', 'wolverine-on-behalf-of-comp-id': '', 'wolverine-log-fix-messages': False, 
'tdameritrade-account-number': '', 'tdameritrade-api-key': '', 'tdameritrade-access-token': '', 'rbi-host': '', 'rbi-port': '', 'rbi-account': '', 'rbi-sender-comp-id': '', 'rbi-target-comp-id': '', 
'rbi-on-behalf-of-comp-id': '', 'rbi-log-fix-messages': False, 'tt-user-name': '', 'tt-session-password': '', 'tt-account-name': '', 'tt-rest-app-key': '', 'tt-rest-app-secret': '', 'tt-rest-environment': '', 
'tt-market-data-sender-comp-id': '', 'tt-market-data-target-comp-id': '', 'tt-market-data-host': '', 'tt-market-data-port': '', 'tt-order-routing-sender-comp-id': '', 'tt-order-routing-target-comp-id': '', 
'tt-order-routing-host': '', 'tt-order-routing-port': '', 'tt-log-fix-messages': False, 'trade-station-client-id': '', 'trade-station-client-secret': '', 'trade-station-redirect-url': 'http://localhost', 
'trade-station-refresh-token': '', 'trade-station-api-url': 'https://sim-api.tradestation.com', 'trade-station-account-type': 'Cash|Margin|Futures', 'trade-station-account-id': '', 'alpaca-api-key': '', 'alpaca-api-secret': 
'', 'alpaca-access-token': '', 'alpaca-paper-trading': True, 'exante-client-id': '', 'exante-application-id': '', 'exante-shared-key': '', 'exante-account-id': '', 'exante-platform-type': '', 'nasdaq-auth-token': '', 
'tiingo-auth-token': '', 'us-energy-information-auth-token': '', 'iex-cloud-api-key': '', 'coinapi-api-key': '', 'coinapi-product': 'free', 'polygon-api-key': '', 'zerodha-access-token': '', 'zerodha-api-key': '', 
'zerodha-product-type': 'MIS', 'zerodha-trading-segment': 'EQUITY', 'zerodha-history-subscription': 'false', 'samco-client-id': '', 'samco-client-password': '', 'samco-year-of-birth': '', 'samco-product-type': 'MIS', 
'samco-trading-segment': 'EQUITY', 'ftx-account-tier': 'Tier1', 'ftx-api-secret': '', 'ftx-api-key': '', 'ftxus-account-tier': 'Tier1', 'ftxus-api-secret': '', 'ftxus-api-key': '', 'regression-test-languages': ['CSharp', 
'Python'], 'python-additional-paths': [], 'environments': {'backtesting': {'live-mode': False, 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BacktestingSetupHandler', 'result-handler': 
'QuantConnect.Lean.Engine.Results.BacktestingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.FileSystemDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.BacktestingRealTimeHandler', 
'history-provider': ['QuantConnect.Lean.Engine.HistoricalData.SubscriptionDataReaderHistoryProvider'], 'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BacktestingTransactionHandler', 'id': 'Local', 
'data-provider': 'QuantConnect.Lean.Engine.DataFeeds.DefaultDataProvider'}, 'live-paper': {'live-mode': True, 'live-mode-brokerage': 'PaperBrokerage', 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 
'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'data-queue-handler': 
['QuantConnect.Lean.Engine.DataFeeds.Queues.LiveDataQueue'], 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 
'QuantConnect.Lean.Engine.TransactionHandlers.BacktestingTransactionHandler'}, 'live-zerodha': {'live-mode': True, 'live-mode-brokerage': 'ZerodhaBrokerage', 'data-queue-handler': ['ZerodhaBrokerage'], 'setup-handler': 
'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 
'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': 
['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-samco': {'live-mode': True, 'live-mode-brokerage': 'SamcoBrokerage', 'data-queue-handler': ['SamcoBrokerage'], 'setup-handler': 
'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 
'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': 
['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-tradier': {'live-mode': True, 'tradier-save-tokens': True, 'live-mode-brokerage': 'TradierBrokerage', 'data-queue-handler': ['TradierBrokerage'], 
'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 
'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 
'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': ['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-interactive': {'live-mode': True, 
'live-mode-brokerage': 'InteractiveBrokersBrokerage', 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 
'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'data-queue-handler': ['InteractiveBrokersBrokerage'], 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 
'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': ['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-interactive-iqfeed': 
{'live-mode': True, 'live-mode-brokerage': 'InteractiveBrokersBrokerage', 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler',
'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'data-queue-handler': ['QuantConnect.IQFeed.IQFeedDataQueueHandler'], 'real-time-handler': 
'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': 
['QuantConnect.IQFeed.IQFeedDataQueueHandler', 'SubscriptionDataReaderHistoryProvider']}, 'live-fxcm': {'live-mode': True, 'live-mode-brokerage': 'FxcmBrokerage', 'data-queue-handler': ['FxcmBrokerage'], 'setup-handler': 
'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 
'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': 
['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-oanda': {'live-mode': True, 'live-mode-brokerage': 'OandaBrokerage', 'data-queue-handler': ['OandaBrokerage'], 'setup-handler': 
'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 
'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': 
['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-coinbase': {'live-mode': True, 'live-mode-brokerage': 'CoinbaseBrokerage', 'data-queue-handler': ['CoinbaseBrokerage'], 'setup-handler': 
'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 
'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': 
['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-bitfinex': {'live-mode': True, 'live-mode-brokerage': 'BitfinexBrokerage', 'data-queue-handler': ['BitfinexBrokerage'], 'setup-handler': 
'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 
'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': 
['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-binance': {'live-mode': True, 'live-mode-brokerage': 'BinanceBrokerage', 'data-queue-handler': ['BinanceBrokerage'], 'setup-handler': 
'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 
'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': 
['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-futures-binance': {'live-mode': True, 'live-mode-brokerage': 'BinanceFuturesBrokerage', 'data-queue-handler': ['BinanceFuturesBrokerage'], 
'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 
'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 
'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': ['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-coin-futures-binance': {'live-mode': True, 
'live-mode-brokerage': 'BinanceCoinFuturesBrokerage', 'data-queue-handler': ['BinanceCoinFuturesBrokerage'], 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 
'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 
'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': ['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-binanceus': {'live-mode': 
True, 'live-mode-brokerage': 'BinanceUSBrokerage', 'data-queue-handler': ['BinanceUSBrokerage'], 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 
'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 
'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': ['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-bybit': {'live-mode': True, 
'live-mode-brokerage': 'BybitBrokerage', 'data-queue-handler': ['BybitBrokerage'], 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 
'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 
'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': ['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-trade-station': 
{'live-mode': True, 'live-mode-brokerage': 'TradeStationBrokerage', 'data-queue-handler': ['TradeStationBrokerage'], 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 
'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 
'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': ['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-alpaca': {'live-mode': True,
'live-mode-brokerage': 'AlpacaBrokerage', 'data-queue-handler': ['AlpacaBrokerage'], 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 
'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 
'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': ['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-futures-bybit': 
{'live-mode': True, 'live-mode-brokerage': 'BybitFuturesBrokerage', 'data-queue-handler': ['BybitFuturesBrokerage'], 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 
'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 
'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': ['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-trading-technologies': 
{'live-mode': True, 'live-mode-brokerage': 'TradingTechnologiesBrokerage', 'data-queue-handler': ['TradingTechnologiesBrokerage'], 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 
'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 
'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler'}, 'live-kraken': {'live-mode': True, 'live-mode-brokerage': 'KrakenBrokerage', 'data-queue-handler': ['KrakenBrokerage'], 
'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 
'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 
'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': ['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-ftx': {'live-mode': True, 'live-mode-brokerage': 
'QuantConnect.FTXBrokerage.FTXBrokerage', 'data-queue-handler': ['QuantConnect.FTXBrokerage.FTXBrokerage'], 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 
'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 
'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': ['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-exante': {'live-mode': True,
'live-mode-brokerage': 'QuantConnect.ExanteBrokerage.ExanteBrokerage', 'data-queue-handler': ['QuantConnect.ExanteBrokerage.ExanteBrokerage'], 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 
'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 
'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': 'BrokerageHistoryProvider'}, 'live-ftxus':
{'live-mode': True, 'live-mode-brokerage': 'QuantConnect.FTXBrokerage.FTXUSBrokerage', 'data-queue-handler': ['QuantConnect.FTXBrokerage.FTXUSBrokerage'], 'setup-handler': 
'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 
'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': 
['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}, 'live-wolverine': {'live-mode': True, 'live-mode-brokerage': 'WolverineBrokerage', 'history-provider': ['SubscriptionDataReaderHistoryProvider'], 
'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 
'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 
'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler'}, 'live-tdameritrade': {'live-mode': True, 'live-mode-brokerage': 'TDAmeritradeBrokerage', 'history-provider': ['BrokerageHistoryProvider', 
'SubscriptionDataReaderHistoryProvider'], 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 
'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 
'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'data-queue-handler': ['TDAmeritradeBrokerage']}, 'live-rbi': {'live-mode': True, 'live-mode-brokerage': 'RBIBrokerage', 'data-queue-handler': 
['QuantConnect.Lean.Engine.DataFeeds.Queues.LiveDataQueue'], 'history-provider': ['SubscriptionDataReaderHistoryProvider'], 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 
'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 
'transaction-handler': 'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler'}, 'live-eze': {'live-mode': True, 'live-mode-brokerage': 'QuantConnect.EzeBrokerage.EzeBrokerage', 'data-queue-handler': 
['QuantConnect.EzeBrokerage.EzeBrokerage'], 'setup-handler': 'QuantConnect.Lean.Engine.Setup.BrokerageSetupHandler', 'result-handler': 'QuantConnect.Lean.Engine.Results.LiveTradingResultHandler', 'data-feed-handler': 
'QuantConnect.Lean.Engine.DataFeeds.LiveTradingDataFeed', 'real-time-handler': 'QuantConnect.Lean.Engine.RealTime.LiveTradingRealTimeHandler', 'transaction-handler': 
'QuantConnect.Lean.Engine.TransactionHandlers.BrokerageTransactionHandler', 'history-provider': ['BrokerageHistoryProvider', 'SubscriptionDataReaderHistoryProvider']}}, 'job-organization-id': 
'myOrganizationId', 'organization-id': 'myOrganizationId', 'file-database-last-update': '10/27/2024', 'id': 'Local', 'environment': 'backtesting', 'close-automatically': True, 
'composer-dll-directory': '.', 'debugging': False, 'debugging-method': 'LocalCmdline', 'job-user-id': '317654', 'api-access-token': '2e868a1e2afb0d1c75751f066bb0a1641203fb17567303489de402062d271a73', 'ib-host': '127.0.0.1', 
'ib-port': '4002', 'ib-tws-dir': '/root/Jts', 'ib-version': '985', 'iqfeed-host': 'host.docker.internal', 'algorithm-type-name': 'main', 'algorithm-language': 'Python', 'algorithm-location': '/LeanCLI/main.py', 'parameters': 
{}, 'storage-limit-mb': '9999999', 'storage-file-count': '9999999', 'algorithm-id': '1239616393'}
Running 'quantconnect/lean:latest' with the following configuration:
{
    'environment': {},
    'stop_signal': 'SIGINT',
    'mounts': [
        {'Target': '/Lean/Launcher/bin/Debug/config.json', 'Source': '/tmp/lean-cli-00e1qj2o/config.json', 'Type': 'bind', 'ReadOnly': True},
        {'Target': '/lean-cli-start.sh', 'Source': '/tmp/lean-cli-9c4gg8_v/lean-cli-start.sh', 'Type': 'bind', 'ReadOnly': True}
    ],
    'volumes': {
        '/myFilePath/test_qc': {'bind': '/LeanCLI', 'mode': 'rw'},
        '/myFilePath/data': {'bind': '/Lean/Data', 'mode': 'rw'},
        '/myFilePath/test_qc/backtests/2024-10-27_08-47-16': {'bind': '/Results', 'mode': 'rw'},
        '/myFilePath/storage': {'bind': '/Storage', 'mode': 'rw'}
    },
    'ports': {},
    'name': 'lean_cli_dd26966306e6407f96bb77f5f74707be',
    'entrypoint': ['bash', '/lean-cli-start.sh'],
    'detach': True,
    'hostname': 'thanos-Standard-PC-Q35-ICH9-2009',
    'tty': True,
    'stdin_open': True,
    'extra_hosts': {'host.docker.internal': '172.17.0.1'},
    'network': 'lean_cli'
}
+ '[' -d /LeanCLI ']'
+ python -m compileall /LeanCLI/main.py /LeanCLI/research.ipynb
Compiling '/LeanCLI/main.py'...
*** PermissionError: [Errno 13] Permission denied: '/LeanCLI/__pycache__'
Traceback (most recent call last):
  File "/home/thanos/miniconda3/envs/globalfintel/lib/python3.12/site-packages/lean/main.py", line 94, in main
    lean.main(standalone_mode=False)
  File "/home/thanos/miniconda3/envs/globalfintel/lib/python3.12/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/home/thanos/miniconda3/envs/globalfintel/lib/python3.12/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/thanos/miniconda3/envs/globalfintel/lib/python3.12/site-packages/lean/click.py", line 231, in invoke
    result = super().invoke(ctx)
             ^^^^^^^^^^^^^^^^^^^
  File "/home/thanos/miniconda3/envs/globalfintel/lib/python3.12/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/thanos/miniconda3/envs/globalfintel/lib/python3.12/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/thanos/miniconda3/envs/globalfintel/lib/python3.12/site-packages/lean/commands/backtest.py", line 400, in backtest
    lean_runner.run_lean(lean_config,
  File "/home/thanos/miniconda3/envs/globalfintel/lib/python3.12/site-packages/lean/components/docker/lean_runner.py", line 174, in run_lean
    raise RuntimeError(
RuntimeError: Something went wrong while running 'test_qc' in the 'backtesting' environment, the output is stored in 'test_qc/backtests/2024-10-27_08-47-16'
Error: Something went wrong while running 'test_qc' in the 'backtesting' environment, the output is stored in 'test_qc/backtests/2024-10-27_08-47-16'

shawnngtq avatar Oct 27 '24 01:10 shawnngtq

Hey, @shawnngtq!

Thanks for sharing the error. It seems that the issue might be because the __pycache__ folder was created with root user permissions, which the Docker container can’t access. Does your project already have a __pycache__ folder? If so, did you use different users while creating and running the backtest? Could you also check the folder's permissions with ls -l ./__pycache__/ ? Besides, you could also try deleting the folder and running it again.

Marinovsky avatar Oct 30 '24 17:10 Marinovsky

@Marinovsky

The test_qc project is the starter code, I don't make any changes to that. Read 1st post on how I generate that.

There is no *pycache* found in that directory.

PermissionError: [Errno 13] Permission denied: '/LeanCLI/__pycache__'

From my understanding /LeanCLI is the container's home directory, thus the issue is during the backtest. I believe the __pycache__ is generated via python -m compileall /LeanCLI/main.py /LeanCLI/research.ipynb?

x find . -iname '*pycache*'
x ls
data  lean.json  storage  test_qc
x tree test_qc 
test_qc
├── backtests
│   ├── 2024-10-26_15-36-38
│   │   ├── code
│   │   │   ├── main.py
│   │   │   └── research.ipynb
│   │   └── config
│   ├── 2024-10-27_08-40-53
│   │   ├── code
│   │   │   ├── main.py
│   │   │   └── research.ipynb
│   │   └── config
│   ├── 2024-10-27_08-47-16
│   │   ├── code
│   │   │   ├── main.py
│   │   │   └── research.ipynb
│   │   └── config
│   ├── 2024-10-27_21-37-54
│   │   ├── code
│   │   │   ├── main.py
│   │   │   └── research.ipynb
│   │   └── config
│   ├── 2024-10-27_21-39-30
│   │   ├── code
│   │   │   ├── main.py
│   │   │   └── research.ipynb
│   │   └── config
│   ├── 2024-10-27_21-41-25
│   │   ├── code
│   │   │   ├── main.py
│   │   │   └── research.ipynb
│   │   └── config
│   └── 2024-10-28_16-28-31
│       ├── code
│       │   ├── main.py
│       │   └── research.ipynb
│       └── config
├── config.json
├── main.py
└── research.ipynb

15 directories, 24 files

shawnngtq avatar Nov 01 '24 00:11 shawnngtq

Hey @shawnngtq! Sorry we can't seem to pin this issue, is there anything special about the lean cli folder on the host? believe we've seen some issue with mac network folder for example. Have u tried using root user or not? To determine whether that would solve it. Is research working for you or is it only backtesting? Can you try C# default project and backtest?

Martin-Molinero avatar Nov 15 '24 14:11 Martin-Molinero

@Martin-Molinero

Test case 1 - ensure no unexpected files

I deleted test_qc and created test_qc2, same error.

Test case 2 - ensure not permission issue

chmod 777 to the entire directory, don't work but different error

Test case 3 - ensure not user issue

execute as root, don't work but different error

Test case 4 - literally all you can do for permission issue

It works with chmod 777 to the entire directory + root user.

Possible cause

I guess the problem is with my setup (Linux OS with Linux network folder).

1st, using my standard user to everything don't work -> I suspect maybe root is required to work with the dotnet 6 installed for LEAN engine (https://www.quantconnect.com/docs/v2/lean-engine/getting-started#04-Installation-Instructions)?

2nd, 777 -> purely my problem caused by Linux network folder

shawnngtq avatar Nov 16 '24 01:11 shawnngtq

I guess the problem is with my setup (Linux OS with Linux network folder)

Yes I'd bet it's the linux network folder + how docker handles this. I'm glad to hear you found a work around for the issue, this will surely help others debug this. We will keep this open so we can document it

Martin-Molinero avatar Nov 18 '24 14:11 Martin-Molinero