UTF-8 characters in `git status` output are not displayed correctly

[nekr0z]

While showing working dir, qgit shows git status output in Log. However, in case git is confugured to use non-ASCII locale (Russian in screenshots below) qgit fails to display this output correctly:

This doesn't happen when displaying actual commits log, UTF-8 is displayed correctly there.

[eraxillan]

Hi @nekr0z , it works ok on my setup (qgit built from master, latest git, macOS):

BTW, git output was in English, until i manually added LANG=ru_RU in the qgit code.

[nekr0z]

Yes, it's better on current master, but still no proper UTF support: output gets truncated. Linux, master, latest git.

[eraxillan]

@nekr0z Hmm, maybe this problem occurs only on special repositories.

1. Can you open qgit itself repo - will the output be corrupted too?
2. If no, can you upload an example of problem repository? with no your private data, of course. Thanks!

[nekr0z]

@eraxillan That's exactly the repo on previous screenshot. And it looks the same for every repo I've tried.

[eraxillan]

So, let's try to reproduce your issue. Please correct me if i understood some details wrong.

• qgit version is the same (master branch, manually built)
• git test repository is the same (qgit itself repo)
• operating system? (my: macOS)
• Qt version? (my: 5.12.3)
• git version? (my: 2.21.0, installed from HomeBrew)
• any custom qgit settings/environment variables?

However, i already mentioned that on my setup output get truncated too, just later:

Неотслеживаемые файлы:
  (испо

English output is still complete.

[nekr0z]

• qgit version is the same (master branch, manually built)

Correct.

• git test repository is the same (qgit itself repo)

Correct.

• operating system? (my: macOS)

Debian 9.9 (current stable version), Linux kernel version 4.9

• Qt version? (my: 5.12.3)

5.7.1

• git version? (my: 2.21.0, installed from HomeBrew)

2.11.0

• any custom qgit settings/environment variables?

Not that I can see.

[eraxillan]

The most possible source of your issue is Qt, i think first time. So i've tried to compile and run qgit with Qt 5.7.1, but still unable to reproduce your issue. git output was truncated too, but in another way than yours.

Looks like i need to setup virtual machine with Debian 9.9 to reproduce issue, it will take much time.

Also, how appropriate git status output in terminal looks like - it is correct and full, isn't? Can you post env command output here? Maybe Debian have custom environment variables setup

[nekr0z]

Also, how appropriate git status output in terminal looks like - it is correct and full, isn't?

Yes.

With dirty status I get qgit picture close to yours (but still truncated earlier):

With clean status I reliably get truncation on 5th character of 3rd line, as shown on screenshot a couple comments above.

Here comes env:

GS_LIB=/home/evgeny/.fonts
KDE_FULL_SESSION=true
LS_COLORS=rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=00:su=37;41:sg=30;43:ca=30;41:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.Z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=00;36:*.au=00;36:*.flac=00;36:*.m4a=00;36:*.mid=00;36:*.midi=00;36:*.mka=00;36:*.mp3=00;36:*.mpc=00;36:*.ogg=00;36:*.ra=00;36:*.wav=00;36:*.oga=00;36:*.opus=00;36:*.spx=00;36:*.xspf=00;36:                                                                                                                                                                                                
BYOBU_ULIMIT=ulimit                                                                                                                                                                                                                          
LANG=ru_RU.UTF-8                                                                                                                                                                                                                             
DISPLAY=:0                                                                                                                                                                                                                                   
PROFILEHOME=                                                                                                                                                                                                                                 
BYOBU_CONFIG_DIR=/home/evgeny/.byobu                                                                                                                                                                                                         
SHELL_SESSION_ID=82a1a2bead904a8c86d90af0ec4e907f                                                                                                                                                                                            
QSG_RENDER_LOOP=                                                                                                                                                                                                                             
BYOBU_DARK=\#333333                                                                                                                                                                                                                          
GPG_TTY=не телетайп                                                                                                                                                                                                                          
BYOBU_LIGHT=\#EEEEEE                                                                                                                                                                                                                         
COLORTERM=truecolor                                                                                                                                                                                                                          
XDG_VTNR=7                                                                                                                                                                                                                                   
PAM_KWALLET5_LOGIN=/tmp/kwallet5_evgeny.socket                                                                                                                                                                                               
SSH_AUTH_SOCK=/tmp/ssh-RJyNRpuwhcHx/agent.1673                                                                                                                                                                                               
BYOBU_PREFIX=/usr                                                                                                                                                                                                                            
XDG_SESSION_ID=3                                                                                                                                                                                                                             
USER=evgeny                                                                                                                                                                                                                                  
DESKTOP_SESSION=/usr/share/xsessions/plasma
BYOBU_HIGHLIGHT=\#DD4814
BYOBU_DISTRO=Debian GNU/Linux
PWD=/home/evgeny/tmp/qgit
HOME=/home/evgeny
BYOBU_PAGER=sensible-pager
SSH_AGENT_PID=1747
XCURSOR_SIZE=0
QT_ACCESSIBILITY=1
XDG_SESSION_TYPE=x11
XDG_DATA_DIRS=/usr/share:/usr/share:/usr/local/share
BYOBU_DATE=%Y-%m-%d 
KONSOLE_DBUS_SESSION=/Sessions/1
BYOBU_PYTHON=python2
XDG_SESSION_DESKTOP=KDE
BYOBU_READLINK=readlink
KONSOLE_DBUS_WINDOW=/Windows/1
BYOBU_WINDOW_NAME=-
SHELL=/bin/bash
TERM=xterm-256color
KONSOLE_DBUS_SERVICE=:1.913
XDG_SESSION_CLASS=user
XDG_SEAT_PATH=/org/freedesktop/DisplayManager/Seat0
BYOBU_SED=sed
XCURSOR_THEME=breeze_cursors
CHROME_REMOTE_DESKTOP_DEFAULT_DESKTOP_SIZES=1024×768
XDG_CURRENT_DESKTOP=KDE
QMLSCENE_DEVICE=
GPG_AGENT_INFO=/run/user/1000/gnupg/S.gpg-agent:0:1
KONSOLE_PROFILE_NAME=Profile #1
QT_LINUX_ACCESSIBILITY_ALWAYS_ON=1
BYOBU_BACKEND=tmux
XDG_SEAT=seat0
COLORFGBG=15;0
SHLVL=1
LANGUAGE=
BYOBU_ACCENT=\#75507B
BYOBU_TIME=%H:%M:%S
WINDOWID=75497478
BYOBU_RUN_DIR=/dev/shm/byobu-evgeny-7rCmVX4m
LOGNAME=evgeny
DBUS_SESSION_BUS_ADDRESS=unix:path=/run/user/1000/bus
XDG_RUNTIME_DIR=/run/user/1000
XAUTHORITY=/home/evgeny/.Xauthority
XDG_SESSION_PATH=/org/freedesktop/DisplayManager/Session1
QT_AUTO_SCREEN_SCALE_FACTOR=0
PATH=/home/evgeny/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games
KDE_SESSION_UID=1000
KDE_SESSION_VERSION=5
XDG_SESSION_COOKIE=b067e72916bb325d7150c2605125208c-1558653435.347257-629551393
SESSION_MANAGER=local/boa:@/tmp/.ICE-unix/1889,unix/boa:/tmp/.ICE-unix/1889
OLDPWD=/home/evgeny
_=/usr/bin/env

[nekr0z]

Also, I tried it on my laptop running Debian Testing (with Qt 5.11.3). The same results, truncation in exactly the same places.

[eraxillan]

@nekr0z Finally i've managed to reproduce the issue :) Looks like it is Debian-specific: even with new vanilla Qt 5.12 i've got truncated text, no difference with Debian one 5.7.
I try to debug qgit code on virtual machine, and probably found where text got corrupted: method Rev::longLog returns truncated text due to invalid (too small) lLogLen variable value.
This variable set up in method Rev::indexData, where ad hoc parser of git log command implemented. When (if) i got some free time, i'll try to fix that issue. I hope it's just a question of time - and patience :)

P.S. also git status output moves thru conversion QString <--> QByteArray <--> QString to be showed in GUI, this is potential source of encoding issues too

[nekr0z]

@eraxillan Awesome! Glad you're on the way to sorting it out, great work, thank you!

[eraxillan]

Hi @nekr0z, i've implemented the workaround for you issue. Just clone my fork from https://github.com/eraxillan/qgit.git, switch to branch upstream_issue_67 and build it from source (qmake qgit.pro, make).
I've refactored code using C++11, so probably it will not compile on your Debian, but you can try.

[nekr0z]

I've refactored code using C++11, so probably it will not compile on your Debian, but you can try.

It actually does compile (with a couple of warnings) and runs perfectly. Awesome job, thank you!

[eraxillan]

You are welcome :)