XcodeBenchmark icon indicating copy to clipboard operation
XcodeBenchmark copied to clipboard

MacBook Pro 14", M4 Max 16C, 128GB, Sequoia 15.1, Xcode 16.1 [87.533sec]

Open kevingriffin opened this issue 1 year ago • 13 comments

** BUILD SUCCEEDED ** [89.621 sec]

System Version: 15.1
Xcode 16.1
Hardware Overview
      Model Name: MacBook Pro
      Model Identifier: Mac16,6
      Total Number of Cores: 16 (12 performance and 4 efficiency)
      Memory: 128 GB

✅ XcodeBenchmark has completed
1️⃣  Take a screenshot of this window (Cmd + Shift + 4 + Space) and resize to include:
\t- Build Time (See ** BUILD SUCCEEDED ** [XYZ sec])
\t- System Version
\t- Xcode Version
\t- Hardware Overview
\t- Started 15:56:30
\t- Ended   15:58:02
\t- Date 2024年 11月 9日 土曜日 15時58分02秒 JST
スクリーンショット 2024-11-09 15 58 22

kevingriffin avatar Nov 09 '24 07:11 kevingriffin

@kevingriffin Thanks for submitting. Did you have high power mode enabled for the run?

alexandre-g avatar Nov 09 '24 07:11 alexandre-g

I did, but I wasn't sure if that was actually appropriate. Let me know and I can do another run if needed.

kevingriffin avatar Nov 09 '24 07:11 kevingriffin

No that's right, it's meant to be on. Just a little surprising that it got almost the same as M3 Max 16-core thats on the results, which is a 16" MBP. Maybe thermal throttling? Do try another run if you can!

alexandre-g avatar Nov 09 '24 07:11 alexandre-g

Agreed, I'll shut it down and let it cool off for a while for another run 👍

kevingriffin avatar Nov 09 '24 07:11 kevingriffin

Please post an update after a second run.

marvin-hansen avatar Nov 09 '24 09:11 marvin-hansen

Second run was 87.533, a minor improvement but not what we'd expect from Geekbench.

It's (obviously) new, but Spotlight indexing seems finished and the system sits at 99% idle before the test is run.

Is there anything that lets you monitor for thermal throttling? I know asitop had a lot of these features, but it doesn't seem to work well on more recent Apple Silicon.

スクリーンショット 2024-11-09 19 27 36

kevingriffin avatar Nov 09 '24 10:11 kevingriffin

This might be unrelated to the actual CPU. The Swift compiler is a frontend to LLVM and I made recently an observation the the latest tw LLVM version 18 and 19 are relatively slow especially. When I switched to a different compiler to build C dependencies for my project (Rust) things were a lot faster.

The recent 30% to 40% regression of Xcode 16 vs 15 indicates that something went seriously wrong with the Apple compiler. For reference, the Rust compiler team never ships any regression in excess of single digit % changes relative to the previous version so I wouldn't draw any conclusion about the hardware yet.

Instead, I think it would be useful to design a different benchmark without XCode where the compiler version is fixed and largely able to exhaust all available cores.

marvin-hansen avatar Nov 09 '24 10:11 marvin-hansen

I am only surprised as an M4 Max is consistently and by a good margin faster than both M2 Ultra and M3 Max in Geekbench benchmarks, including in a clang one. I haven't checked closely, but benchmarks for other models seem to roughly carry across to the performance differences between hardware in this benchmark, yet not this one

alexandre-g avatar Nov 09 '24 14:11 alexandre-g

The 16" got 79s here, which is the time one would expect (for the M4 Max 16c)

@kevingriffin did you run Geekbench benchmark on yours by any chance? Can you share the result run if so?

alexandre-g avatar Nov 11 '24 10:11 alexandre-g

No that's right, it's meant to be on. Just a little surprising that it got almost the same as M3 Max 16-core thats on the results, which is a 16" MBP. Maybe thermal throttling? Do try another run if you can!

I'm getting better results (around 3 secs) with my m4 max and auto mode

flowridr avatar Nov 12 '24 16:11 flowridr

Would someone please measure temperature when the 14" Max runs the benchmark?

According to this report, the M4 gained a massive 32% in performance per Watt.

https://www.jeffgeerling.com/blog/2024/m4-mac-minis-efficiency-incredible

It might be that the limited thermal design of the 14" model requires thermal throttling to keep everything stable as the M4 peaks now at about 100Watt vs about 80Watt with the M3.

I loosely remember that thermal throttling of the 14" already came up with the M3 series, but it was way less of an issue.

The 16" model always had more cooling capacity hence can run the CPU longer at higher frequency and therefore delivere better sustained performance.

marvin-hansen avatar Nov 13 '24 00:11 marvin-hansen

Is there anything that lets you monitor for thermal throttling? I know asitop had a lot of these features, but it doesn't seem to work well on more recent Apple Silicon.

@kevingriffin this guy says some patch is needed? Has a repo with it in the comments https://x.com/ivanfioravanti/status/1856110736314958064

https://github.com/ivanfioravanti/asitop

alexandre-g avatar Nov 13 '24 00:11 alexandre-g

Just a quick question, was the original run made on battery?

Apparently you get slightly better performance when on a power chord, as discussed in this review.

https://youtu.be/eqsuFlT6WoE

marvin-hansen avatar Nov 15 '24 09:11 marvin-hansen