proton icon indicating copy to clipboard operation
proton copied to clipboard

refine settings javascript_max_memory_bytes and correct set hard limit

Open yokofly opened this issue 1 year ago • 0 comments

fix #716 so this PR do 3 things:

  1. increase the default javascript_max_memory_bytes to 200MByte from 100MByte
  2. Correct v8 memory hard limit, previous we do not set a value, and will hit v8 abort if the uda try to alloc too many memory.
  3. Correctly apply settings javascript_max_memory_bytes. previous we pass a current_thread::context, now we use query_context.

for 3: in latest proton server below query will passed, because the value is always 100MByte instead we want 2MByte.

CREATE STREAM IF NOT EXISTS 99010_udf_types(`f32` float);

CREATE AGGREGATE FUNCTION test_sec_large_99010(value float32) RETURNS float32 LANGUAGE JAVASCRIPT AS $$
                  {
                    initialize: function() {
                       this.max = -1.0;
                       this.sec = -1.0
                    },
                    process: function(values) {
                      for (let i = 0; i < values.length; i++) {
                        if (values[i] > this.max) {
                          this.sec = this.max;
                          this.max = values[i]
                        }
                        if (values[i] < this.max && values[i] > this.sec)
                          this.sec = values[i];
                      }
                    }  ..... hide other function
              $$;

select sleep(1) FORMAT Null;
insert into 99010_udf_types(f32) values(2.0);   
select sleep(1) FORMAT Null;

Before this PR: Below CTE query will always use the default of 100MB. If the memory usage exceeds 100MB, trying to modify settings javascript_max_memory_bytes=2 to 200MB will not change the outcome. Now: An error will be reported.

with acc as (select test_sec_large_99010(f32) as res from table(99010_udf_types) settings javascript_max_memory_bytes=2) select min(res) as re from acc;

yokofly avatar May 14 '24 05:05 yokofly