cannot pack inode in ql_syscall_fstat when rootfs on NTFS

[alfink]

ql_syscall_fstat relies on os.fstat. On a WSL2 host, it returns a 64 bit inode if the rootfs is located on a NTFS partition. Qiling will crash when packing it into 32 bit.

Traceback (most recent call last):
  File "tendaac_httpd.py", line 76, in <module>
    my_sandbox(["rootfs/bin/httpd"], "rootfs")
  File "tendaac_httpd.py", line 70, in my_sandbox
    ql.run()
  File "/home/alfink/.local/lib/python3.8/site-packages/qiling-1.1.post1-py3.8.egg/qiling/core.py", line 198, in run
    self.os.run()
  File "/home/alfink/.local/lib/python3.8/site-packages/qiling-1.1.post1-py3.8.egg/qiling/os/linux/linux.py", line 130, in run
    raise self.ql.internal_exception
  File "/home/alfink/.local/lib/python3.8/site-packages/qiling-1.1.post1-py3.8.egg/qiling/utils.py", line 19, in wrapper
    return func(*args, **kw)
  File "/home/alfink/.local/lib/python3.8/site-packages/qiling-1.1.post1-py3.8.egg/qiling/core_hooks.py", line 121, in _hook_intr_cb
    ret = h.call(ql, intno)
  File "/home/alfink/.local/lib/python3.8/site-packages/qiling-1.1.post1-py3.8.egg/qiling/core_hooks.py", line 36, in call
    return self.callback(ql, *args)
  File "/home/alfink/.local/lib/python3.8/site-packages/qiling-1.1.post1-py3.8.egg/qiling/os/linux/linux.py", line 65, in hook_syscall
    return self.load_syscall(intno)
  File "/home/alfink/.local/lib/python3.8/site-packages/qiling-1.1.post1-py3.8.egg/qiling/os/posix/posix.py", line 167, in load_syscall
    self.syscall_map(self.ql, self.get_func_arg()[0], self.get_func_arg()[1], self.get_func_arg()[2], self.get_func_arg()[3], self.get_func_arg()[4], self.get_func_arg()[5])
  File "/home/alfink/.local/lib/python3.8/site-packages/qiling-1.1.post1-py3.8.egg/qiling/os/posix/syscall/stat.py", line 270, in ql_syscall_fstat
    fstat_buf += ql.pack(fstat_info.st_ino)
  File "/home/alfink/.local/lib/python3.8/site-packages/qiling-1.1.post1-py3.8.egg/qiling/core_struct.py", line 102, in pack
    return self.pack32(data)
  File "/home/alfink/.local/lib/python3.8/site-packages/qiling-1.1.post1-py3.8.egg/qiling/core_struct.py", line 63, in pack32
    return struct.pack('I', x)
struct.error: 'I' format requires 0 <= number <= 4294967295

[xwings]

not too sure is a NTFS causing python to break ?

[alfink]

its not python what breaks, maybe it was confusing that i said it crashes. it just terminates with an exception.

The problem is that ql_syscall_fstat packs the inode as a 32bit int (depending on the emulated arch). On NTFS, inodes are 64bit, so the inode that os.fstat() returns can be too large to fit into a 32bit int.

[ucgJhe]

maybe try set_syscall in the script from fstat to fstat64, and see what will happen ?

[xwings]

we sort of fix this issue by wrapping fstat. will you be able to try it.

as usual. pull the latest dev branch.

[alfink]

I "fixed" everything by moving the rootfs to another partition that is not NTFS, so it's not much of a problem for me anymore. However, pulling the latest dev branch broke my previously working scripts. I'm currently looking for the bugs, so expect some issues or PRs soon.

To give you some feedback: I tried it again on NTFS again and it seems not to work, it seems there is still the same problem as before in fstat and stat (struct.error: 'I' format requires 0 <= number <= 4294967295). I also tried the set_syscall thing, but the binary didn't really like using stat64 instead of stat.

[xwings]

there are some more missing stat related syscall. i guess we cover abit more this time and not too sure did i miss out any. again, there are similar issue in macos too.

[alfink]

Maybe its a solution to create fake inodes for the guest and map them to the host-inodes, similar to what is already done with the file descriptors

[TheZ3ro]

I've experienced this same problem when using an ARM ELF (32 bit) binary stored a NTFS partition with the latest Qiling 1.3-dev.

Like @alfink mentioned, NTFS partitions always use 64 bit inode numbers. Currently ql_syscall_fstat64 unpacks the stat struct based on the value of ql.archtype (ARM 32 in my case) but being on a NTFS partition the resulting fstat64_info.st_ino value is 64 bit and doing fstat64_buf += ql.pack32(fstat64_info.st_ino) will crash.

The current workaround solution is to avoid runing 32 bit executables on NTFS partitions.

---

$ file int_overflow
int_overflow: ELF 32-bit LSB executable, ARM, EABI5 version 1 (SYSV), statically linked, for GNU/Linux 3.2.0,
$ stat -c %i int_overflow
6473924465859801

Full Qiling traceback

  [x] [posix.py:181]
Traceback (most recent call last):
  File "/home/thezero/.local/lib/python3.6/site-packages/qiling/os/posix/posix.py", line 171, in load_syscall
    ret = self.syscall_map(self.ql, self.get_func_arg()[0], self.get_func_arg()[1], self.get_func_arg()[2], self.get_func_arg()[3], self.get_func_arg()[4], self.get_func_arg()[5])
  File "/home/thezero/.local/lib/python3.6/site-packages/qiling/os/posix/syscall/stat.py", line 167, in ql_syscall_fstat64
    fstat64_buf += ql.pack32(fstat64_info.st_ino)
  File "/home/thezero/.local/lib/python3.6/site-packages/qiling/core_struct.py", line 60, in pack32
    return struct.pack(self._fmt32, x)
struct.error: 'I' format requires 0 
    my_sandbox(["./int_overflow"], ".")
  File "example.py", line 7, in my_sandbox
    ql.run()
  File "/home/thezero/.local/lib/python3.6/site-packages/qiling/core.py", line 765, in run
    self.os.run()
  File "/home/thezero/.local/lib/python3.6/site-packages/qiling/os/linux/linux.py", line 149, in run
    self.ql.emu_start(self.ql.loader.elf_entry, self.exit_point, self.ql.timeout, self.ql.count)
  File "/home/thezero/.local/lib/python3.6/site-packages/qiling/core.py", line 957, in emu_start
    raise self._internal_exception
  File "/home/thezero/.local/lib/python3.6/site-packages/qiling/utils.py", line 157, in wrapper
    return func(*args, **kw)
  File "/home/thezero/.local/lib/python3.6/site-packages/qiling/core_hooks.py", line 119, in _hook_intr_cb
    ret = h.call(ql, intno)
  File "/home/thezero/.local/lib/python3.6/site-packages/qiling/core_hooks.py", line 35, in call
    return self.callback(ql, *args)
  File "/home/thezero/.local/lib/python3.6/site-packages/qiling/os/linux/linux.py", line 71, in hook_syscall
    return self.load_syscall(intno)
  File "/home/thezero/.local/lib/python3.6/site-packages/qiling/os/posix/posix.py", line 183, in load_syscall
    raise e
  File "/home/thezero/.local/lib/python3.6/site-packages/qiling/os/posix/posix.py", line 171, in load_syscall
    ret = self.syscall_map(self.ql, self.get_func_arg()[0], self.get_func_arg()[1], self.get_func_arg()[2], self.get_func_arg()[3], self.get_func_arg()[4], self.get_func_arg()[5])
  File "/home/thezero/.local/lib/python3.6/site-packages/qiling/os/posix/syscall/stat.py", line 167, in ql_syscall_fstat64
    fstat64_buf += ql.pack32(fstat64_info.st_ino)
  File "/home/thezero/.local/lib/python3.6/site-packages/qiling/core_struct.py", line 60, in pack32
    return struct.pack(self._fmt32, x)
struct.error: 'I' format requires 0 

[xwings]

I guess this is a "unsolveable" issue. We strongly suggest WSL is a proper way to go.