Running filebench in gVisor

[Anjali05]

Description

I am trying to run some filesystem metadata stress tests using Filebench(https://github.com/filebench/filebench) on gVisor. For the experiments, I need to turn off address randomization using echo 0 > /proc/sys/kernel/randomize_va_space. I understand that I cannot access or modify this from gVisor. This has been an issue in Filebench for some time (https://github.com/filebench/filebench/issues/163 and https://github.com/filebench/filebench/issues/112) and I am not sure if they are going to fix it. I was wondering if is there any workaround within gVisor that I can do for this to work?

Is this feature related to a specific bug?

No response

Do you have a specific solution in mind?

No response

[milantracy]

I don't see randomize_va_space at https://github.com/google/gvisor/blob/115723cc9fefe7b8d3324bca18f636789843d251/pkg/sentry/fsimpl/proc/tasks_sys.go#L48-L79

would you implement that if you are interested

for the workaround, you can simply implement a fake entry to skip the step

[Anjali05]

I don't see randomize_va_space at

https://github.com/google/gvisor/blob/115723cc9fefe7b8d3324bca18f636789843d251/pkg/sentry/fsimpl/proc/tasks_sys.go#L48-L79

would you implement that if you are interested

for the workaround, you can simply implement a fake entry to skip the step

Yes, it's not currently there. I think adding it as a static file should work?

[milantracy]

since it is writing to the file, implementing the interface at https://github.com/google/gvisor/blob/12fb7f25d2e70f1ce1312d556407c7653b1f1cb4/pkg/sentry/vfs/file_description_impl_util.go#L259 would be better.

[EtiennePerot]

I'm not sure I understand the request.

Does Filebench absolutely require running on a host that has ASLR turned off, and you are looking to see how that could be enabled from within gVisor?
Or is Filebench just trying to write to /proc/sys/kernel/randomize_va_space as part of its initialization process, and that code is failing in gVisor because the file doesn't exist, and you're looking to stub out this file just to let Filebench start up?

[Anjali05]

I'm not sure I understand the request.

Does Filebench absolutely require running on a host that has ASLR turned off, and you are looking to see how that could be enabled from within gVisor? Or is Filebench just trying to write to /proc/sys/kernel/randomize_va_space as part of its initialization process, and that code is failing in gVisor because the file doesn't exist, and you're looking to stub out this file just to let Filebench startup?

So far I have not been able to successfully run Filebench unless /proc/sys/kernel/randomize_va_space contains 0. If the file is not present or contains something other than 0. Filebench starts but does not finish execution and abruptly terminates. I am not sure internally what it's doing.

[EtiennePerot]

Can you simply mount a file containing the string "0" there?

$ echo 0 > /tmp/just_zero.txt
$ docker run --rm --runtime=runsc \
    -v /tmp/just_zero.txt:/proc/sys/kernel/randomize_va_space \
    ubuntu cat /proc/sys/kernel/randomize_va_space
0

[Anjali05]

@EtiennePerot Hmm, that might work. Let me try it

[Anjali05]

@EtiennePerot It did not work, then there might be some other issue, not sure

[nixprime]

filebench shares a filebench_shm_t object, the pointee of ipc.h:filebench_shm, between multiple worker processes using a file that is mapped into all processes using MAP_SHARED. *filebench_shm contains pointers into itself; for this to work between processes, its address (filebench_shm) must be consistent between them. Thus, as currently written, filebench needs to disable ASLR to work; tricking it into thinking ASLR is disabled is insufficient.

[Anjali05]

@nixprime Is it possible to disable ASLR in gvisor currently?

[ayushr2]

IIUC ASLR in gVisor is implemented here: https://github.com/google/gvisor/blob/b050c045d16d222a977f982be8de2c1a1052228a/pkg/sentry/arch/arch_amd64.go#L232-L256

We don't have a way to disable that as of right now.

[EtiennePerot]

FYI, gVisor already has filesystem stress tests based on the fio tool. For example, to test runsc I/O performance with KVM+DirectFS, you could do:

# Build runsc:
$ mkdir bin
$ make copy TARGETS=runsc DESTINATION=bin/

# Install Docker runtime with KVM + DirectFS enabled:
$ sudo bin/runsc install --runtime=runsc-bench -- --platform=kvm --directfs=true
$ sudo systemctl restart docker

# Run `fio` benchmark:
$ make RUNTIME=runsc-bench BENCHMARKS_TARGETS=test/benchmarks/fs:fio_test run-benchmark
BenchmarkFioWrite/operation.write/ioEngine.sync/jobs.1/blockSize.4K/directIO.false/filesystem.bindfs-20                    19762           1800039 ns/op         604614656 bandwidth.bytes_per_second           147611 io_ops.ops_per_second
[...]

To compare with unsandboxed, use RUNTIME=runc instead.

[Anjali05]

@EtiennePerot Thank you. Yeah, I have no problem running fio. I wanted to try Filebench as it has better and more flexible filesystem metadata stress tests such as creating and deleting large numbers of files etc. As far as I know, Fio is recommended mostly for testing raw I/O performance.

[Anjali05]

@EtiennePerot @nixprime @ayushr2 @milantracy Can someone suggest any pointers if I have to implement disabling ASLR in gVisor, how do I go about it? I am not very familiar with the codebase.

[nixprime]

If you just need to hack out ASLR for testing, you can change https://github.com/google/gvisor/blob/da7cd03064d787d6546919a438cb4e0c60349fd6/pkg/sentry/arch/arch_amd64.go#L197 to return 0.

[Anjali05]

If you just need to hack out ASLR for testing, you can change

gvisor/pkg/sentry/arch/arch_amd64.go

Line 197 in da7cd03

return hostarch.Addr(rand.Int63n(int64(max))).RoundDown() to return 0.

My goal is to make Filebench work on gVisor. I am under the impression that if ASLR is disabled in gVisor, it will work. The build seems to fail with the change you suggested.