maxtext icon indicating copy to clipboard operation
maxtext copied to clipboard

Fix generate_metrics_and_upload_to_big_query

Open ycchenzheng opened this issue 3 months ago • 2 comments

Description

Fix generate_metrics_and_upload_to_big_query flag and related dependencies.

FIXES: b/441984274, b/446097400

Tests

Project preparation:

  • Create a dataset in BigQuery or use an existed one in the target project

benchmarks/recipes/user_configs.py change:
Adding the following flags to

# Define the required configuration here
USER_CONFIG = UserConfig(
  • Set bq_enable to True
  • Set bq_db_project and bq_db_dataset to the user's target project and the dataset in the target project
    Run the command python3 -m benchmarks.recipes.pw_mcjax_benchmark_recipe and after 20 steps, one big query record will be written to the run_summary table of the target dataset

Checklist

Before submitting this PR, please make sure (put X in square brackets):

  • [x] I have performed a self-review of my code. For an optional AI review, add the gemini-review label.
  • [x] I have necessary comments in my code, particularly in hard-to-understand areas.
  • [x] I have run end-to-end tests tests and provided workload links above if applicable.
  • [x] I have made or will make corresponding changes to the doc if needed.

ycchenzheng avatar Oct 30 '25 18:10 ycchenzheng

Context https://github.com/AI-Hypercomputer/maxtext/pull/2502#issuecomment-3408415947 point 1

ycchenzheng avatar Oct 30 '25 18:10 ycchenzheng

Actually, please disregard my "approval". Meant to put in just comments. We still need to ensure we have full approvals for moving aotc code.

SujeethJinesh avatar Nov 06 '25 20:11 SujeethJinesh