ClickHouse icon indicating copy to clipboard operation
ClickHouse copied to clipboard

23.8 Aarch64 builds with better introspection

Open Enmk opened this issue 1 year ago • 3 comments

Changelog category (leave one):

  • Build/Testing/Packaging Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Building aarch64 builds with '-no-pie' to allow better introspection

Enmk avatar May 02 '24 17:05 Enmk

This is an automated comment for commit 636a295c100c832eaaaa07d92fceec2ab106fd65 with description of existing statuses. It's updated for the latest CI running

❌ Click here to open a full report in a separate page

Successful checks
Check nameDescriptionStatus
ClickHouse build checkBuilds ClickHouse in various configurations for use in further steps. You have to fix the builds that fail. Build logs often has enough information to fix the error, but you might have to reproduce the failure locally. The cmake options can be found in the build log, grepping for cmake. Use these options and follow the general build process✅ success
Compatibility checkChecks that clickhouse binary runs on distributions with old libc versions. If it fails, ask a maintainer for help✅ success
Docker image for serversThe check to build and optionally push the mentioned image to docker hub✅ success
Install packagesChecks that the built packages are installable in a clear environment✅ success
Integration testsThe integration tests report. In parenthesis the package type is given, and in square brackets are the optional part/total tests✅ success
Mergeable CheckChecks if all other necessary checks are successful✅ success
Push to DockerhubThe check for building and pushing the CI related docker images to docker hub✅ success
Ready for releaseThere's no description for the check yet, please add it to tests/ci/ci_config.py:CHECK_DESCRIPTIONS✅ success
Stateful testsRuns stateful functional tests for ClickHouse binaries built in various configurations -- release, debug, with sanitizers, etc✅ success
Stress testRuns stateless functional tests concurrently from several clients to detect concurrency-related errors✅ success
Check nameDescriptionStatus
CI runningA meta-check that indicates the running CI. Normally, it's in success or pending state. The failed status indicates some problems with the PR⏳ pending
Stateless testsRuns stateless functional tests for ClickHouse binaries built in various configurations -- release, debug, with sanitizers, etc❌ failure

altinity-robot avatar May 02 '24 18:05 altinity-robot

Without no-pie, trace_log / crash_log etc. can't be directly symbolized after clickhouse restart (with ASLR enabled on OS level).

(still can be done using some math and guessing the 'old' base address by comparing adresses in the current & old stacktraces).

filimonov avatar May 03 '24 10:05 filimonov

For the reference, there is a draft of adding run-time warning at startup, but this doesn't compile against 23.8 because of incompatible linux headers baked into CH repo:

diff --git a/src/Interpreters/SystemLog.cpp b/src/Interpreters/SystemLog.cpp
index 23a5a96584c..7f049cda173 100644
--- a/src/Interpreters/SystemLog.cpp
+++ b/src/Interpreters/SystemLog.cpp
@@ -37,6 +37,10 @@
 #include <Common/logger_useful.h>
 #include <Common/setThreadName.h>
 
+#if defined(OS_LINUX)
+#include <sys/personality.h>
+#endif
+
 
 namespace DB
 {
@@ -295,7 +299,17 @@ SystemLogs::SystemLogs(ContextPtr global_context, const Poco::Util::AbstractConf
     if (part_log)
         logs.emplace_back(part_log.get());
     if (trace_log)
+    {
+#if defined(OS_LINUX)
+        if (!(personality(0xffffffff) & ADDR_NO_RANDOMIZE))
+        {
+            global_context->addWarningMessage("Table system.trace_log is enabled on PIC/PIE build. "
+                "Stack traces from previous launches (e.g. before ClickHouse restart) "
+                "would produce incorrect results when used with addressToSymbol/ToSymbol/ToLineWithInlines.");
+        }
+#endif
         logs.emplace_back(trace_log.get());
+    }
     if (crash_log)
         logs.emplace_back(crash_log.get());
     if (text_log)

Enmk avatar May 03 '24 12:05 Enmk