[BUG] Invoking executable without args gives regex error

[TylerSeanRau]

Describe the bug When I run the following benchmark setup I always see the message Failed to match any benchmarks against regex: . when I invoke the executable (giving --help causes a segfault

System Which OS, compiler, and compiler version are you using:

• OS: Debian 10 for WSL2
• Compiler and version: gcc 8.3

To reproduce Steps to reproduce the behavior:

1. Create a new directory
2. Place this CMakeLists.txt in that directory:

cmake_minimum_required(VERSION 3.14)
project(my_project)

set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_STANDARD_REQUIRED YES)

include(FetchContent)

set(BENCHMARK_ENABLE_TESTING off)

FetchContent_Declare(
  googletest
    GIT_REPOSITORY https://github.com/google/googletest.git
    GIT_TAG release-1.12.0)

FetchContent_Declare(
  googlebenchmark
    GIT_REPOSITORY https://github.com/google/benchmark.git
    GIT_TAG v1.7.0)

FetchContent_MakeAvailable(
  googletest
  googlebenchmark)

find_package(OpenMP)

add_library(
  libbench
  bench.cpp)

target_link_libraries(
  libbench
  PRIVATE
    benchmark::benchmark
    pthread
    OpenMP::OpenMP_CXX)

add_executable(bench /dev/null)
target_link_libraries(
  bench
    PRIVATE
      libbench
      benchmark::benchmark_main
      OpenMP::OpenMP_CXX)

3. Use this .cpp file
   • or write your own as far as I can tell it doesn't matter
   • unsure if all these headers matter but I've left them in case they do

#include <iostream>
#include <string>
#include <unordered_map>
#include <vector>
#include <fcntl.h>
#include <unistd.h>
#include <sstream>
#include <string>
#include <benchmark/benchmark.h>
#include <vector>
#include <random>
#include <iostream>
#include <sys/mman.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <mutex>
#include <omp.h>
#include <array>
#include <memory>

static void BM_vector_push_back(benchmark::State& state) {
  for (auto _ : state) {
    std::vector<int> v;
    v.reserve(1);
    benchmark::DoNotOptimize(v.data()); // Allow v.data() to be clobbered.
    v.push_back(42);
    benchmark::ClobberMemory(); // Force 42 to be written to memory.
  }
}

BENCHMARK(BM_vector_push_back);

4. cmake -DCMAKE_BUILD_TYPE=Release -S . -B build
5. cmake --build build -- -j8
6. ./build/bench
7. See error, Step 8 should've run the BM_vector_push_back test but instead it says Failed to match any benchmarks against regex: . and fails.

BONUS: 8. if you run build/bench --help you'll see a segmentation fault

Expected behavior I expect all benchmarks in the cpp file to have been run. (in this case BM_vector_push_back) Additionally I expected command line argument parsing to function correctly

Screenshots If applicable, add screenshots to help explain your problem.

Additional context The project where I noticed this problem previously used gcc directly and referred to artifacts that were created manually. I want to use cmake's FetchContent in that project instead this.

[dmah42]

the crash on --help was fixed just after i cut 1.7.0. i should cut 1.7.1 to fix that.

i'm still looking into the regex thing.

[dmah42]

i've tracked it down to benchmarks not being registered.. i'm wondering if it's because the benchmarks are in a separate library from the main, but i agree that should work.

[dmah42]

confirmed. this works:

cmake_minimum_required(VERSION 3.14)
project(my_project)

set(CMAKE_CXX_STANDARD 14)
set(CMAKE_CXX_STANDARD_REQUIRED YES)

include(FetchContent)

set(BENCHMARK_ENABLE_TESTING off)

FetchContent_Declare(
  googletest
    GIT_REPOSITORY https://github.com/google/googletest.git
    GIT_TAG release-1.12.0)

FetchContent_Declare(
  googlebenchmark
    GIT_REPOSITORY https://github.com/google/benchmark.git
    GIT_TAG v1.7.0)

FetchContent_MakeAvailable(
  googletest
  googlebenchmark)

find_package(OpenMP)

add_executable(
  bench
  bench.cpp)

target_link_libraries(
  bench
  PRIVATE
    benchmark::benchmark
    benchmark::benchmark_main
    pthread
    OpenMP::OpenMP_CXX)

so i think there's an issue with ordering where the static constructors of the library are not running before main.

i also tried making libbench static but that didn't work, which is a little surprising.

[HFTrader]

This is the way shared libraries work. The registration symbol is emitted but it's not brought into the executable since by default static libraries's symbols are optional, which on shared libraries all symbols are brought in.

One could use -Wl,--whole-archive to force all symbols in a static library to be brought into the executable but that's a command line argument. There isn't really much we can do on code.

https://stackoverflow.com/questions/5202142/static-variable-initialization-over-a-library

This works:

add_executable(bench /dev/null)
target_link_libraries(
  bench
    PRIVATE
      -Wl,--whole-archive libbench -Wl,--no-whole-archive
      benchmark::benchmark_main
      OpenMP::OpenMP_CXX)

[HFTrader]

Optionally, if you are using CMake newer than 3.12, you could use OBJECT libraries.

This also works:

add_library(
  libbench OBJECT
  bench.cpp)