bittensor icon indicating copy to clipboard operation
bittensor copied to clipboard

ANSI Escape Sequences do not work under PM2 in version of bittensor older than 6.9.4

Open vaqxai opened this issue 1 year ago • 3 comments

Describe the bug

When using ansi escape sequences with bittensor 6.9.4, the messages are correctly formatted, but not later (7.1.2 and 8.2.0 tested) versions. Even a simple print() before any bittensor functions are executed (other than the import itself) doesn't work with colorized messages in the later versions.

To Reproduce

print("Test Colored: \033[0;33mTest")
import bittensor as bt
print("Test Colored: \033[0;33mTest")

while using bittensor==6.9.4 yields correct colorization (while the python script is ran under PM2 process manager) then change bittensor to be 8.2.0 or 7.1.2 (i havent tested with other versions), and see that after import bittensor the colorization stops working. This behaviour will carry over to any bt.logging.info or ANY attempt to print to stdout.

Expected behavior

All messages accept and use ANSI Escape sequences correctly

Screenshots

image image image

Environment

Python3.12, PM2 5.4.2, Bittensor 6.9.4/7.1.2/8.2.0, Ubuntu 24.04, kitty terminal over openssh

Additional context

No response

vaqxai avatar Oct 18 '24 13:10 vaqxai

Confirming versions (on Python 3.12):

  • 6.9.4 works
  • 6.10.0 does not work

A change between those two versions broke this functionality.

vaqxai avatar Oct 21 '24 06:10 vaqxai

Did another test on Python version 3.11, and confirming it is also broken on any version after 6.9.4

vaqxai avatar Oct 21 '24 07:10 vaqxai

I have done analysis, it is precisely btlogging/format.py:34 (in 7.1.2) that breaks the ANSI Escape sequences. That line is colorama.init(autoreset=True).

From colorama's source code, doc-comment on AnsiToWin32:

Implements a 'write()' method which, on Windows, will strip ANSI character sequences from the text, and if outputting to a tty, will convert them into win32 function calls.

We're on linux, not windows, so further investigation:

  • have_tty = not self.stream.closed and self.stream.isatty() on line 104 of colorama/ansitowin32.py

means if there's not tty (likely a file output) we will need to strip ANSI characters (to have a clean logfile) Indeed. PM2 simply reads the logfile it creates, where the stdout of the application is redirected, and indeed reading that file with a text editor when Ansi isn't stripped, it shows the escape codes within the text editor.

However, we need these sequences in the log file so we can see the ANSI work within our PM2 output. also, bittensor manually sets up file logging within a different facility, and if need be could strip ANSI there. Therefore, I conclude, that ANSI stripping via colorama is not needed for bittensor's use case.

The fix is rather simple:

(bittensor/btlogging/format.py:34) init(autoreset=True) -> init(autoreset=True, strip=False)

vaqxai avatar Oct 21 '24 08:10 vaqxai