constellation icon indicating copy to clipboard operation
constellation copied to clipboard

Running a Simple Evaluation

Open ashuthosh-mr opened this issue 2 years ago • 8 comments

Hi,

I am trying to run a simple evaluation using chipyard's standalone mode. My chipyard version is 1.8.1. When I execute the command "make SUB_PROJECT=constellation BINARY=none CONFIG=EvalTestConfig00 MODEL=EvalHarness run-binary-debug" inside /chipyard/sims/verilator, I get these: Constellation: test Checking full connectivity Constellation: test Checking virtual subnet connectivity Constellation: test Checking for possibility of deadlock Constellation: test Finished parameter validation Constellation: test Starting NoC RTL generation Constellation: test Finished NoC RTL generation And then, I get few errors such as: [error] Picked up JAVA_TOOL_OPTIONS: -Xmx8G -Xss8M -Djava.io.tmpdir=/home/chips/Storage/willi/chipyard/.java_tmp [error] WARNING: Empty *.mems.conf file. No memories generated In the end, I get: %Error: Specified --top-module 'TestHarness' was not found in design. %Error: Exiting due to 1 error(s) However, a noceval.cfg is generated. But I still get: %Error: Specified --top-module 'TestHarness' was not found in design. %Error: Exiting due to 1 error(s) after rerunning the simulation.

What might be going wrong? Also, I am refering https://constellation.readthedocs.io/en/latest/Evaluation/index.html

Thanks

ashuthosh-mr avatar Jun 06 '23 05:06 ashuthosh-mr

Update on this.

There was an issue with makefile which always took TestHarness as top-module. I changed it to EvalHarness, it worked.

However, I came up with new errors: /home/mypc/chipyard/.conda-env/bin/../lib/gcc/x86_64-conda-linux-gnu/12.1.0/../../../../x86_64-conda-linux-gnu/bin/ld: TrafficEval.o: in function netrace_traffic_eval_t::ingress_tick(unsigned long, unsigned long, char, bool, bool)': TrafficEval.cpp:(.text+0x15f5): undefined reference to nt_read_packet' and few more undefined references to others as well.

Finally an error: make: *** [Makefile:232: /home/mypc/chipyard/sims/verilator/simulator-constellation.test-EvalTestConfig00-debug] Error 2

ashuthosh-mr avatar Jun 06 '23 09:06 ashuthosh-mr

I'll try this right now,

jerryz123 avatar Jun 06 '23 17:06 jerryz123

Can you try chipyard 1.9.1?

jerryz123 avatar Jun 08 '23 16:06 jerryz123

I tried setting 1.9.1 from scratch. I got upto this point with no problems: Setup complete! To generate simulator RTL and run metasimulation simulation, source env.sh To use the manager to deploy builds/simulations on EC2, source sourceme-f1-manager.sh to setup your environment. To run builds/simulations manually on this machine, source sourceme-f1-full.sh to setup your environment. . . Then I got an error: /home/mypc/chipyard/software/firemarshal/logs/br-base-build-2023-06-09--15-45-23-PVXUBEWNN88O8K89.log . /home/mypc/chipyard/software/firemarshal/boards/firechip/base-workloads/br-base/host-init.sh . /home/mypc/chipyard/software/firemarshal/images/firechip/br.947c/br.947c.img TaskError - taskid:/home/mypc/chipyard/software/firemarshal/images/firechip/br.947c/br.947c.img PythonAction Error Traceback (most recent call last): File "/home/mypc/chipyard/.conda-env/lib/python3.10/site-packages/doit/action.py", line 461, in execute returned_value = self.py_callable(*self.args, **kwargs) File "/home/mypc/chipyard/software/firemarshal/boards/firechip/distros/br/br.py", line 193, in buildBaseImage wlutil.run(['make'], cwd=br_dir / "buildroot", env=env) File "/home/mypc/chipyard/software/firemarshal/wlutil/wlutil.py", line 527, in run raise sp.CalledProcessError(p.returncode, prettyCmd) subprocess.CalledProcessError: Command 'make' returned non-zero exit status 2.

ERROR: Failed to build workload br-base.json Log available at: /home/mypc/chipyard/software/firemarshal/logs/br-base-build-2023-06-09--15-45-23-PVXUBEWNN88O8K89.log ERROR: FAILURE: 1 builds failed . . . However, The env.sh was incomplete. So, I used the env.sh of 1.8.1 and made an env.sh file with necessary changes. Simulating a rocket core, hello world worked fine.

Coming to constellation's test harness, no errors. Evaluation framework worked further than it was before(1.8.1) but I still got an error with the make.

This is what I got: . . . chipyard/sims/verilator/generated-src/constellation.test.EvalHarness.EvalTestConfig00/gen-collateral/netrace.h:42:48: error: invalid conversion from 'void*' to 'nt_dep_ref_node_t**' {aka 'nt_dep_ref_node**'} [-fpermissive] 42 | #define nt_checked_malloc(x) _nt_checked_malloc(x,FILE,LINE) | ~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~ | | | void* . . . make[1]: *** [VEvalHarness.mk:90: netrace.o] Error 1 make[1]: Leaving directory '/home/mypc/chipyard/sims/verilator/generated-src/constellation.test.EvalHarness.EvalTestConfig00/constellation.test.EvalHarness.EvalTestConfig00.debug' make: *** [Makefile:231: /home/mypc/chipyard/sims/verilator/simulator-constellation.test-EvalTestConfig00-debug] Error 2

ashuthosh-mr avatar Jun 10 '23 04:06 ashuthosh-mr

Sorry, it looks like the way verilator invokes g++ just fundamentally breaks the netrace headers. If you have access to VCS, you can follow the VCS flow.

If not, then the only other solution would be to comment out all references to trace in the evaluation harness C++ driver, including the #includes on netrace.h.

jerryz123 avatar Jun 13 '23 05:06 jerryz123

Alright. What might cause on commenting out the references and netrace.h?

ashuthosh-mr avatar Jun 19 '23 08:06 ashuthosh-mr

You would not be able to run the C++ evaluation harness with netrace-provided trace files as the traffic source.

jerryz123 avatar Jun 19 '23 13:06 jerryz123

I solve this problem by two step: step 1. modifying the generated natrace.c in directory "/root/chipyard/sims/verilator/generated-src/constellation.test.EvalHarness.EvalTestConfig00/gen-collateral" below:

/*
 * Copyright (c) 2010-2011 The University of Texas at Austin
 * All rights reserved.
 *
 * Redistribution and use in source and binary forms, with or without
 * modification, are permitted provided that the following conditions are
 * met: redistributions of source code must retain the above copyright
 * notice, this list of conditions and the following disclaimer;
 * redistributions in binary form must reproduce the above copyright
 * notice, this list of conditions and the following disclaimer in the
 * documentation and/or other materials provided with the distribution;
 * neither the name of the copyright holders nor the names of its
 * contributors may be used to endorse or promote products derived from
 * this software without specific prior written permission.
 *
 * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
 * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
 * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
 * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
 * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
 * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
 * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
 * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
 * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
 * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
 * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
 */

#include "netrace.h"

const char* nt_packet_types[] = { "InvalidCmd", "ReadReq", "ReadResp",
				"ReadRespWithInvalidate", "WriteReq", "WriteResp",
				"Writeback", "InvalidCmd", "InvalidCmd", "InvalidCmd",
				"InvalidCmd", "InvalidCmd", "InvalidCmd", "UpgradeReq",
				"UpgradeResp", "ReadExReq", "ReadExResp", "InvalidCmd",
				"InvalidCmd", "InvalidCmd", "InvalidCmd", "InvalidCmd",
				"InvalidCmd", "InvalidCmd", "InvalidCmd", "BadAddressError",
				"InvalidCmd", "InvalidateReq", "InvalidateResp",
				"DowngradeReq", "DowngradeResp" };

int nt_packet_sizes[] = { /*InvalidCmd*/ -1, /*ReadReq*/ 8, /*ReadResp*/ 72,
				/*ReadRespWithInvalidate*/ 72, /*WriteReq*/ 72, /*WriteResp*/ 8,
				/*Writeback*/ 72, /*InvalidCmd*/ -1, /*InvalidCmd*/ -1, /*InvalidCmd*/ -1,
				/*InvalidCmd*/ -1, /*InvalidCmd*/ -1, /*InvalidCmd*/ -1, /*UpgradeReq*/ 8,
				/*UpgradeResp*/ 8, /*ReadExReq*/ 8, /*ReadExResp*/ 72, /*InvalidCmd*/ -1,
				/*InvalidCmd*/ -1, /*InvalidCmd*/ -1, /*InvalidCmd*/ -1, /*InvalidCmd*/ -1,
				/*InvalidCmd*/ -1, /*InvalidCmd*/ -1, /*InvalidCmd*/ -1, /*BadAddressError*/ 8,
				/*InvalidCmd*/ -1, /*InvalidateReq*/ 8, /*InvalidateResp*/ 8,
				/*DowngradeReq*/ 8, /*DowngradeResp*/ 72 };

const char* nt_node_types[] = { "L1 Data Cache", "L1 Instruction Cache",
				"L2 Cache", "Memory Controller", "Invalid Node Type" };

void nt_open_trfile( nt_context_t* ctx, const char* trfilename ) {
	nt_close_trfile( ctx );
	int i;
	int length = 20;
	for( i = 0; trfilename[i] != 0; i++, length++ );
	ctx->input_popencmd = (char*) nt_checked_malloc( length * sizeof(char) );
	sprintf( ctx->input_popencmd, "bzip2 -dc %s", trfilename );
	ctx->input_tracefile = popen( ctx->input_popencmd, "r" );
	if( ctx->input_tracefile == NULL ) {
		nt_error( "failed to open pipe to trace file" );
	}
	ctx->input_trheader = nt_read_trheader( ctx );
	if( ctx->dependency_array == NULL ) {
		ctx->dependency_array = (nt_dep_ref_node_t**)nt_checked_malloc( sizeof(nt_dep_ref_node_t*) * NT_DEPENDENCY_ARRAY_SIZE );
		memset( ctx->dependency_array, 0, sizeof(nt_dep_ref_node_t*) * NT_DEPENDENCY_ARRAY_SIZE );
		ctx->num_active_packets = 0;
	} else {
		nt_error( "dependency array not NULL on file open" );
	}
}

nt_header_t* nt_read_trheader( nt_context_t* ctx ) {

	#pragma pack(push,1)
	struct nt_header_pack {
		unsigned int nt_magic;
		float version;
		char benchmark_name[NT_BMARK_NAME_LENGTH];
		unsigned char num_nodes;
		unsigned char pad;
		unsigned long long int num_cycles;
		unsigned long long int num_packets;
		unsigned int notes_length;  // Includes null-terminating char
		unsigned int num_regions;
		char padding[8];
	};
	#pragma pack(pop)

	int err = 0;
	char strerr[180];

	// Read Header
	struct nt_header_pack* in_header = (struct nt_header_pack*)nt_checked_malloc( sizeof(struct nt_header_pack) );
	fseek( ctx->input_tracefile, 0, SEEK_SET );
	if( (err = fread( in_header, sizeof(struct nt_header_pack), 1, ctx->input_tracefile )) < 0 ) {
		sprintf( strerr, "failed to read trace file header: err = %d", err );
		nt_error( strerr );
	}

	// Copy data from struct to header
	nt_header_t* to_return = (nt_header_t*) nt_checked_malloc( sizeof(nt_header_t) );
	memset( to_return, 0, sizeof(nt_header_t) );
	to_return->nt_magic = in_header->nt_magic;
	to_return->version = in_header->version;
	strcpy( to_return->benchmark_name, in_header->benchmark_name );
	to_return->num_nodes = in_header->num_nodes;
	to_return->num_cycles = in_header->num_cycles;
	to_return->num_packets = in_header->num_packets;
	to_return->notes_length = in_header->notes_length;
	to_return->num_regions = in_header->num_regions;
	free( in_header );

	// Error Checking
	if( to_return->nt_magic != NT_MAGIC ) {
		if( nt_little_endian() != 1 ) {
			nt_error( "only little endian architectures are currently supported" );
		} else {
			nt_error( "invalid trace file: bad magic" );
		}
	}
	if( to_return->version != 1.0f ) {
		sprintf( strerr, "trace file is unsupported version: %f", to_return->version );
		nt_error( strerr );
	}
	// Read Rest of Header
	if( to_return->notes_length > 0 && to_return->notes_length < 8192 ) {
		to_return->notes = (char*) nt_checked_malloc( to_return->notes_length * sizeof(char) );
		if( (err = fread( to_return->notes, sizeof(char), to_return->notes_length, ctx->input_tracefile )) < 0 ) {
			sprintf( strerr, "failed to read trace file header notes: err = %d\n", err );
			nt_error( strerr );
		}
	} else {
		to_return->notes = NULL;
	}
	if( to_return->num_regions > 0 ) {
		if( to_return->num_regions <= 100 ) {
			to_return->regions = (nt_regionhead_t*) nt_checked_malloc( to_return->num_regions * sizeof(nt_regionhead_t) );
			if( (err = fread( to_return->regions, sizeof(nt_regionhead_t), to_return->num_regions, ctx->input_tracefile )) < 0 ) {
				sprintf( strerr, "failed to read trace file header regions: error = %d\n", err );
				nt_error( strerr );
			}
		} else {
			nt_error( "lots of regions... is this correct?" );
		}
	} else {
		to_return->regions = NULL;
	}
	return to_return;
}

void nt_disable_dependencies( nt_context_t* ctx ) {
	if( ctx->track_cleared_packets_list ) {
		nt_error( "Cannot turn off dependencies when tracking cleared packets list" );
	}
	ctx->dependencies_off = 1;
}

void nt_seek_region( nt_context_t* ctx, nt_regionhead_t* region ) {
	int err = 0;
	char strerr[180];
	if( ctx->input_tracefile != NULL ) {
		if( region != NULL ) {
			// Clear all existing dependencies
			nt_delete_all_dependencies( ctx );
			// Reopen file to fast-forward to region
			// fseek doesn't work on compressed file
			pclose( ctx->input_tracefile );
			ctx->input_tracefile = popen( ctx->input_popencmd, "r" );
			unsigned long long int seek_offset = nt_get_headersize( ctx ) + region->seek_offset;
			unsigned long long int read_length = 4096;
			char* buffer = (char*) nt_checked_malloc( read_length );
			while( seek_offset > read_length ) {
				if( (err = fread( buffer, 1, read_length, ctx->input_tracefile )) < 0 ) {
					sprintf( strerr, "failed to seek region: error = %d\n", err );
					nt_error( strerr );
				}
				seek_offset -= read_length;
			}
			if( (err = fread( buffer, 1, seek_offset, ctx->input_tracefile )) < 0 ) {
				sprintf( strerr, "failed to seek region: error = %d\n", err );
				nt_error( strerr );
			}
			free( buffer );
			if( ctx->self_throttling ) {
				// Prime the pump to read in self throttled packets
				nt_prime_self_throttle( ctx );
			}
		} else {
			nt_error( "invalid region passed: NULL" );
		}
	} else {
		nt_error( "must open trace file with nt_open_trfile before seeking" );
	}
}

nt_packet_t* nt_read_packet( nt_context_t* ctx ) {

	#pragma pack(push,1)
	struct nt_packet_pack {
		unsigned long long int cycle;
		unsigned int id;
		unsigned int addr;
		unsigned char type;
		unsigned char src;
		unsigned char dst;
		unsigned char node_types;
		unsigned char num_deps;
	};
	#pragma pack(pop)

	int err = 0;
	unsigned int i;
	char strerr[180];
	nt_packet_t* to_return = NULL;
	if( ctx->input_tracefile != NULL ) {
		to_return = nt_packet_malloc();
		if( (err = fread( to_return, 1, sizeof(struct nt_packet_pack), ctx->input_tracefile )) < 0 ) {
			sprintf( strerr, "failed to read packet: err = %d", err );
			nt_error( strerr );
		}
		if( err > 0 && err < sizeof(struct nt_packet_pack) ) {
			// Bad packet - end of file
			nt_error( "unexpectedly reached end of trace file - perhaps corrupt" );
		} else if( err == 0 ) {
			// End of file
			free( to_return );
			to_return = NULL;
			return to_return;
		}
		if( !ctx->dependencies_off ) {
			// Track dependencies: add to_return to dependencies array
			nt_dep_ref_node_t* node_ptr = nt_get_dependency_node( ctx, to_return->id );
			if( node_ptr == NULL ) {
				node_ptr = nt_add_dependency_node( ctx, to_return->id );
			}
			node_ptr->node_packet = to_return;
		}
		ctx->num_active_packets++;
		ctx->latest_active_packet_cycle = to_return->cycle;
		if( to_return->num_deps == 0 ) {
			to_return->deps = NULL;
		} else {
			to_return->deps = nt_dependency_malloc( to_return->num_deps );
			if( (err = fread( to_return->deps, sizeof(nt_dependency_t), to_return->num_deps, ctx->input_tracefile )) < 0 ) {
				sprintf( strerr, "failed to read dependencies: err = %d", err );
				nt_error( strerr );
			}
			if( !ctx->dependencies_off ) {
				// Track dependencies: add to_return downward dependencies to array
				for( i = 0; i < to_return->num_deps; i++ ) {
					unsigned int dep_id = to_return->deps[i];
					nt_dep_ref_node_t* node_ptr = nt_get_dependency_node( ctx, dep_id );
					if( node_ptr == NULL ) {
						node_ptr = nt_add_dependency_node( ctx, dep_id );
					}
					node_ptr->ref_count++;
				}
			}
		}
	} else {
		nt_error( "must open trace file with nt_open_trfile before reading" );
	}
	return to_return;
}

nt_dep_ref_node_t* nt_add_dependency_node( nt_context_t* ctx, unsigned int packet_id ) {
	if( ctx->dependency_array != NULL ) {
		unsigned int index = packet_id % NT_DEPENDENCY_ARRAY_SIZE;
		nt_dep_ref_node_t* dep_ptr = ctx->dependency_array[index];
		if( dep_ptr == NULL ) {
			ctx->dependency_array[index] = (nt_dep_ref_node_t*)nt_checked_malloc( sizeof(nt_dep_ref_node_t) );
			dep_ptr = ctx->dependency_array[index];
		} else {
			for( ; dep_ptr->next_node != NULL; dep_ptr = dep_ptr->next_node );
			dep_ptr->next_node = (nt_dep_ref_node_t*)nt_checked_malloc( sizeof(nt_dep_ref_node_t) );
			dep_ptr = dep_ptr->next_node;
		}
		dep_ptr->node_packet = NULL;
		dep_ptr->packet_id = packet_id;
		dep_ptr->ref_count = 0;
		dep_ptr->next_node = NULL;
		return dep_ptr;
	} else {
		nt_error( "dependency array NULL on node addition" );
	}
	return NULL;
}

void nt_read_ahead( nt_context_t* ctx, unsigned long long int current_cycle ) {
	unsigned long long int read_to_cycle = current_cycle + NT_READ_AHEAD;
	if( read_to_cycle < current_cycle ) {
		nt_error( "trying to read too far ahead... overflowed :(" );
	}
	if( read_to_cycle > ctx->latest_active_packet_cycle ) {
		nt_packet_t* packet;
		while( ctx->latest_active_packet_cycle <= read_to_cycle && !ctx->done_reading ) {
			packet = nt_read_packet( ctx );
			if( packet == NULL ) {
				// This is the exit condition... how do we signal it to the
				// network simulator? We shouldn't need to... It is tracking
				// whether there are packets in flight.
				// Just in case, we'll provide this global indicator
				ctx->done_reading = 1;
			} else if( nt_dependencies_cleared( ctx, packet ) ) {
				nt_add_cleared_packet_to_list( ctx, packet );
			//} else {
				// Ignore this packet, since the reader is already tracking it
			}
		}
	}
}

nt_packet_t* nt_remove_dependency_node( nt_context_t* ctx, unsigned int packet_id ) {
	if( ctx->dependency_array != NULL ) {
		unsigned int index = packet_id % NT_DEPENDENCY_ARRAY_SIZE;
		nt_dep_ref_node_t* dep_ptr = ctx->dependency_array[index];
		if( dep_ptr == NULL ) {
			return NULL;
		} else {
			nt_dep_ref_node_t* prev_ptr = NULL;
			for( ; dep_ptr != NULL; prev_ptr = dep_ptr, dep_ptr = dep_ptr->next_node ) {
				if( dep_ptr->packet_id == packet_id ) break;
			}
			if( dep_ptr == NULL ) {
				return NULL;
			}
			if( prev_ptr == NULL ) {
				ctx->dependency_array[index] = dep_ptr->next_node;
			} else {
				prev_ptr->next_node = dep_ptr->next_node;
			}
			nt_packet_t* packet = dep_ptr->node_packet;
			free( dep_ptr );
			return packet;
		}
	} else {
		nt_error( "dependency array NULL on node remove" );
	}
	return NULL;
}

nt_dep_ref_node_t* nt_get_dependency_node( nt_context_t* ctx, unsigned int packet_id ) {
	if( ctx->dependency_array != NULL ) {
		unsigned int index = packet_id % NT_DEPENDENCY_ARRAY_SIZE;
		nt_dep_ref_node_t* dep_ptr;
		for( dep_ptr = ctx->dependency_array[index]; dep_ptr != NULL; dep_ptr = dep_ptr->next_node ) {
			if( dep_ptr->packet_id == packet_id ) break;
		}
		return dep_ptr;
	} else {
		nt_error( "dependency array not NULL on node search" );
	}
	return NULL;
}

int nt_dependencies_cleared( nt_context_t* ctx, nt_packet_t* packet ) {
	if( ctx->input_tracefile != NULL ) {
		nt_dep_ref_node_t* node_ptr = nt_get_dependency_node( ctx, packet->id );
		if( node_ptr == NULL || ctx->dependencies_off ) {
			return 1;
		} else {
			return (node_ptr->ref_count == 0);
		}
	} else {
		nt_error( "must open trace file with nt_open_trfile before injecting" );
	}
	return 1;
}

void nt_clear_dependencies_free_packet( nt_context_t* ctx, nt_packet_t* packet ) {
	unsigned int i;
	if( ctx->input_tracefile != NULL ) {
		if( packet != NULL ) {
			// If self-throttling, read ahead in the trace file
			// to ensure that there are new packets ready to go
			if( ctx->self_throttling ) {
				nt_read_ahead( ctx, packet->cycle );
			}
			for( i = 0; i < packet->num_deps; i++ ) {
				unsigned int dep_id = packet->deps[i];
				nt_dep_ref_node_t* node_ptr = nt_get_dependency_node( ctx, dep_id );
				if( node_ptr == NULL ) {
					if( !ctx->dependencies_off ) {
						// TODO: check if this is a problem with short seeks
						nt_print_packet( packet );
						nt_error( "failed to find dependency node" );
					}
				} else {
					if( node_ptr->ref_count == 0 ) {
						nt_error( "invalid reference count on node while decrementing" );
					}
					node_ptr->ref_count--;
					if( ctx->track_cleared_packets_list ) {
						if( node_ptr->ref_count == 0 ) {
							// This test alleviates the possibility of a packet
							// having ref_count zero before it has been read
							// from the trace (node_packet = NULL)
							if( node_ptr->node_packet ) {
								nt_add_cleared_packet_to_list( ctx, node_ptr->node_packet );
							}
						}
					}
				}
			}
			nt_remove_dependency_node( ctx, packet->id );
			nt_packet_free( packet );
			ctx->num_active_packets--;
		}
	} else {
		nt_error( "must open trace file with nt_open_trfile before ejecting" );
	}
}

void nt_init_cleared_packets_list( nt_context_t* ctx ) {
	if( ctx->dependencies_off ) {
		nt_error( "Cannot return cleared packets list when dependencies are turned off" );
	}
	ctx->track_cleared_packets_list = 1;
	ctx->cleared_packets_list = NULL;
	ctx->cleared_packets_list_tail = NULL;
}

void nt_init_self_throttling( nt_context_t* ctx ) {
	if( ctx->dependencies_off ) {
		nt_error( "Cannot self throttle packets when dependencies are turned off" );
	}
	ctx->self_throttling = 1;
	ctx->primed_self_throttle = 0;
	nt_init_cleared_packets_list( ctx );
}

nt_packet_list_t* nt_get_cleared_packets_list( nt_context_t* ctx ) {
	if( !ctx->primed_self_throttle ) {
		nt_prime_self_throttle( ctx );
	}
	return ctx->cleared_packets_list;
}

void nt_prime_self_throttle( nt_context_t* ctx ) {
	nt_packet_t* packet = nt_read_packet( ctx );
	if( nt_dependencies_cleared( ctx, packet ) ) {
		nt_add_cleared_packet_to_list( ctx, packet );
	}
	ctx->primed_self_throttle = 1;
	nt_read_ahead( ctx, packet->cycle );
}

void nt_add_cleared_packet_to_list( nt_context_t* ctx, nt_packet_t* packet ) {
	nt_packet_list_t* new_node = (nt_packet_list_t*)nt_checked_malloc( sizeof(nt_packet_list_t) );
	new_node->node_packet = packet;
	new_node->next = NULL;
	if( ctx->cleared_packets_list == NULL ) {
		ctx->cleared_packets_list = ctx->cleared_packets_list_tail = new_node;
	} else {
		ctx->cleared_packets_list_tail->next = new_node;
		ctx->cleared_packets_list_tail = new_node;
	}
}

void nt_empty_cleared_packets_list( nt_context_t* ctx ) {
	while( ctx->cleared_packets_list != NULL ) {
		nt_packet_list_t* temp = ctx->cleared_packets_list;
		ctx->cleared_packets_list = ctx->cleared_packets_list->next;
		free( temp );
	}
	ctx->cleared_packets_list = ctx->cleared_packets_list_tail = NULL;
}

void nt_close_trfile( nt_context_t* ctx ) {
	if( ctx->input_tracefile != NULL ) {
		pclose( ctx->input_tracefile );
		ctx->input_tracefile = NULL;
		nt_free_trheader( ctx->input_trheader );
		if( ctx->input_popencmd != NULL ) {
			free( ctx->input_popencmd );
		}
		ctx->input_popencmd = NULL;
		nt_delete_all_dependencies( ctx );
		free(ctx->dependency_array);
		ctx->dependency_array = NULL;
	}
}

void nt_delete_all_dependencies( nt_context_t* ctx ) {
	int i;
	for( i = 0; i < NT_DEPENDENCY_ARRAY_SIZE; i++ ) {
		while( ctx->dependency_array[i] != NULL ) {
			nt_remove_dependency_node( ctx, ctx->dependency_array[i]->packet_id );
		}
	}
}

void nt_print_header( nt_context_t* ctx, nt_header_t* header ) {
	unsigned int i;
	if( header != NULL ) {
		printf( "NT_TRACEFILE---------------------\n" );

		printf( "  Benchmark: %s\n", header->benchmark_name );
		printf( "  Magic Correct? %s\n", (header->nt_magic == NT_MAGIC) ? "TRUE" : "FALSE" );
		printf( "  Tracefile Version: v%1.1f\n", header->version );
		printf( "  Number of Program Regions: %d\n", header->num_regions );
		printf( "  Number of Simulated Nodes: %d\n", header->num_nodes );
		printf( "  Simulated Cycles: %lld\n", header->num_cycles );
		printf( "  Simulated Packets: %lld\n", header->num_packets );
		printf( "  Average injection rate: %f\n", (double)header->num_packets / (double)header->num_cycles );
		if( header->notes_length > 0 ) {
			printf( "  Notes: %s\n", header->notes );
		}

		for( i = 0; i < header->num_regions; i++ ) {
			printf( "    Region %d:\n", i );
			printf( "      Seek Offset: %lld\n", header->regions[i].seek_offset );
			printf( "      Simulated Cycles: %lld\n", header->regions[i].num_cycles );
			printf( "      Simulated Packets: %lld\n", header->regions[i].num_packets );
			printf( "      Average injection rate: %f\n", (double)header->regions[i].num_packets / (double)header->regions[i].num_cycles );
			printf( "      Average injection rate per node: %f\n", (double)header->regions[i].num_packets / (double)header->regions[i].num_cycles / (double)header->num_nodes );
		}
		printf( "  Size of header (B): %u\n", nt_get_headersize( ctx ) );
		printf( "NT_TRACEFILE---------------------\n" );
	} else {
		printf( "NULL header passed to nt_print_header\n" );
	}
}

void nt_print_trheader( nt_context_t* ctx ) {
	nt_print_header( ctx, ctx->input_trheader );
}

nt_header_t* nt_get_trheader( nt_context_t* ctx ) {
	if( ctx->input_tracefile != NULL ) {
		return ctx->input_trheader;
	} else {
		nt_error( "must open trace file with nt_open_trfile before header is available" );
	}
	return NULL;
}

float nt_get_trversion( nt_context_t* ctx ) {
	if( ctx->input_tracefile != NULL ) {
		return ctx->input_trheader->version;
	} else {
		nt_error( "must open trace file with nt_open_trfile before version is available" );
	}
	return 0.0f;
}

nt_packet_t* nt_packet_malloc() {
	// TODO: v1.1?
	// Allocate large array (1+ pages) to reduce # system calls
	return (nt_packet_t*) nt_checked_malloc( sizeof(nt_packet_t) );
}

nt_dependency_t* nt_dependency_malloc( unsigned char num_deps ) {
	// TODO: v1.1?
	// Allocate large array (1+ pages) to reduce # system calls
	return (nt_dependency_t*) nt_checked_malloc( num_deps * sizeof(nt_dependency_t) );
}

int nt_get_src_type( nt_packet_t* packet ) {
	return (int) ( packet->node_types >> 4 );
}

int nt_get_dst_type( nt_packet_t* packet ) {
	return (int) ( 0xF & packet->node_types );
}

const char* nt_node_type_to_string( int type ) {
	if( type < NT_NUM_NODE_TYPES ) {
		return nt_node_types[type];
	} else {
		return nt_node_types[NT_NUM_NODE_TYPES];
	}
}

int nt_get_packet_size( nt_packet_t* packet ) {
	if( packet->type < NT_NUM_PACKET_TYPES ) {
		return nt_packet_sizes[packet->type];
	} else {
		return nt_packet_sizes[0];
	}
}

const char* nt_packet_type_to_string( nt_packet_t* packet ) {
	if( packet->type < NT_NUM_PACKET_TYPES ) {
		return nt_packet_types[packet->type];
	} else {
		return nt_packet_types[0];
	}
}

nt_packet_t* nt_packet_copy( nt_packet_t* packet ) {
	if( packet != NULL ) {
		nt_packet_t* to_return = nt_packet_malloc();
		memcpy( to_return, packet, sizeof(nt_packet_t) );
		if( packet->num_deps > 0 ) {
			to_return->deps = nt_dependency_malloc( to_return->num_deps );
			memcpy( to_return->deps, packet->deps, sizeof(nt_dependency_t) * to_return->num_deps );
		}
		return to_return;
	}
	return NULL;
}

void nt_packet_free( nt_packet_t* packet ) {
	if( packet != NULL ) {
		if( packet->num_deps > 0 ) {
			free( packet->deps );
		}
		free( packet );
	}
}

void nt_print_packet( nt_packet_t* packet ) {
	int i;
	if( packet != NULL ) {
		printf( "  ID:%u CYC:%llu SRC:%u DST:%u ADR:0x%08x TYP:%s NDEP:%u",
				packet->id, packet->cycle, packet->src,
				packet->dst, packet->addr, nt_packet_type_to_string( packet ),
				packet->num_deps );
		for( i = 0; i < packet->num_deps; i++ ) {
			printf( " %d", packet->deps[i] );
		}
		printf( "\n" );
	} else {
		printf( "WARNING: %s:%d: NULL packet printed!\n", __FILE__, __LINE__ );
	}
}

void nt_free_trheader( nt_header_t* header ) {
	if( header != NULL ) {
		if( header->regions != NULL ) {
			free( header->regions );
		}
		if( header->notes != NULL ) {
			free( header->notes );
		}
		free( header );
	}
}

int nt_get_headersize( nt_context_t* ctx ) {

	#pragma pack(push,1)
	struct nt_header_pack {
		unsigned int nt_magic;
		float version;
		char benchmark_name[NT_BMARK_NAME_LENGTH];
		unsigned char num_nodes;
		unsigned long long int num_cycles;
		unsigned long long int num_packets;
		unsigned int notes_length;  // Includes null-terminating char
		unsigned int num_regions;
		char padding[9];
	};
	#pragma pack(pop)

	if( ctx->input_tracefile != NULL ) {
		int to_return = 0;
		to_return += sizeof(struct nt_header_pack);
		to_return += ctx->input_trheader->notes_length;
		to_return += ctx->input_trheader->num_regions * sizeof(nt_regionhead_t);
		return to_return;
	} else {
		nt_error( "must open trace file with nt_open_trfile before header is available" );
	}
	return -1;
}

void* _nt_checked_malloc( size_t n, char* file, int line ) {
	void* ptr;
	ptr = malloc( n );
	if( ptr == NULL ) {
		fprintf( stderr, "ERROR: bad allocation at %s:%d\n", file, line );
		exit(0);
	}
	return ptr;
}

int nt_little_endian() {
	int to_return = 1;
	union {
		int number;
		char bytes[sizeof(int)];
	} u;
	unsigned int i;
	u.number = (int)0;
	for( i = 0; i < sizeof(int); i++ ) {
		u.number |= (int)(i + 1) << (8 * i);
	}
	for( i = 0; i < sizeof(int); i++ ) {
		if( (unsigned int)u.bytes[i] != i+1 ) {
			to_return = 0;
			break;
		}
	}
	return to_return;
}

void _nt_error( const char* str, char* file, int line ) {
#ifdef DEBUG_ON
	fprintf( stderr, "WARNING: In %s:%d: %s\n", file, line, str );
#else
	fprintf( stderr, "ERROR: In %s:%d: %s\n", file, line, str );
	exit(0);
#endif
}

// Backend functions for creating trace files
void nt_dump_header( nt_header_t* header, FILE* fp ) {

	#pragma pack(push,1)
	struct nt_header_pack {
		unsigned int nt_magic;
		float version;
		char benchmark_name[NT_BMARK_NAME_LENGTH];
		unsigned char num_nodes;
		unsigned char pad;
		unsigned long long int num_cycles;
		unsigned long long int num_packets;
		unsigned int notes_length;  // Includes null-terminating char
		unsigned int num_regions;
		char padding[8];
	};
	#pragma pack(pop)

	if( header != NULL ) {
		struct nt_header_pack* out_header = (struct nt_header_pack*)nt_checked_malloc( sizeof(struct nt_header_pack) );
		out_header->nt_magic = header->nt_magic;
		out_header->version = header->version;
		strcpy( out_header->benchmark_name, header->benchmark_name );
		out_header->num_nodes = header->num_nodes;
		out_header->num_cycles = header->num_cycles;
		out_header->num_packets = header->num_packets;
		out_header->notes_length = header->notes_length;
		out_header->num_regions = header->num_regions;
		fwrite( out_header, sizeof(struct nt_header_pack), 1, fp );
		fwrite( header->notes, sizeof(char), header->notes_length, fp );
		fwrite( header->regions, sizeof(nt_regionhead_t), header->num_regions, fp );
		free( out_header );
	} else {
		nt_error( "dumping NULL header" );
	}
}

void nt_dump_packet( nt_packet_t* packet, FILE* fp ) {

	#pragma pack(push,1)
	struct nt_read_pack {
		unsigned long long int cycle;
		unsigned int id;
		unsigned int addr;
		unsigned char type;
		unsigned char src;
		unsigned char dst;
		unsigned char node_types;
		unsigned char num_deps;
	};
	#pragma pack(pop)

	if( packet != NULL ) {
		fwrite( packet, sizeof(struct nt_read_pack), 1, fp );
		fwrite( packet->deps, sizeof(nt_dependency_t), packet->num_deps, fp );
	} else {
		nt_error( "dumping NULL packet" );
	}
}

Step 2. Run the makefile in directory "/root/chipyard/generators/constellation/src/main/resources/csrc/netrace" and generate the natrace.o file. Replace the generated natrace.o to the broken natrace.o in directory "/root/chipyard/sims/verilator/generated-src/constellation.test.EvalHarness.EvalTestConfig00/constellation.test.EvalHarness.EvalTestConfig00.debug"

The output is below:

No traffic matrix plusarg found. Falling back on default config str.
Results CSV:
ingress_id, egress_id, received, sent, throughput, median_latency, max_latency
0, 0, 16000, 19948, 0.802085, 2902, 4874

Min throughput: 0, 0, 0.802085
Median latency: 2902
Max latency: 4874
Latency hist:   0-10: 0
  10-20: 0
  20-30: 0
  30-40: 0
  40-50: 0
  50-60: 0
  60-70: 0
  70-80: 0
  80-90: 0
  90-100: 0
  100-110: 0
  110-120: 0
  120-130: 0
  130-140: 0
  140-150: 0
  150-160: 0
  160-170: 0
  170-180: 0
  180-190: 0
  190-200: 0
  200-210: 0
  210-220: 0
  220-230: 0
  230-240: 0
  240-250: 0
  250-260: 0
  260-270: 0
  270-280: 0
  280-290: 0
  290-300: 0
  300-310: 0
  310-320: 0
  320-330: 0
  330-340: 0
  340-350: 0
  350-360: 0
  360-370: 0
  370-380: 0
  380-390: 0
  390-400: 0
  400-410: 0
  410-420: 0
  420-430: 0
  430-440: 0
  440-450: 0
  450-460: 0
  460-470: 0
  470-480: 0
  480-490: 0
  490-500: 0
  500-510: 0
  510-520: 0
  520-530: 0
  530-540: 0
  540-550: 0
  550-560: 0
  560-570: 0
  570-580: 0
  580-590: 0
  590-600: 0
  600-610: 0
  610-620: 0
  620-630: 0
  630-640: 0
  640-650: 0
  650-660: 0
  660-670: 0
  670-680: 0
  680-690: 0
  690-700: 0
  700-710: 0
  710-720: 0
  720-730: 0
  730-740: 0
  740-750: 0
  750-760: 0
  760-770: 0
  770-780: 0
  780-790: 0
  790-800: 0
  800-810: 0
  810-820: 0
  820-830: 0
  830-840: 0
  840-850: 0
  850-860: 0
  860-870: 0
  870-880: 0
  880-890: 0
  890-900: 0
  900-910: 0
  910-920: 0
  920-930: 0
  930-940: 0
  940-950: 0
  950-960: 0
  960-970: 0
  970-980: 0
  980-990: 0
  990-1000: 0
  1000-1010: 6
  1010-1020: 22
  1020-1030: 68
  1030-1040: 15
  1040-1050: 14
  1050-1060: 17
  1060-1070: 28
  1070-1080: 24
  1080-1090: 11
  1090-1100: 14
  1100-1110: 20
  1110-1120: 23
  1120-1130: 18
  1130-1140: 42
  1140-1150: 18
  1150-1160: 51
  1160-1170: 43
  1170-1180: 30
  1180-1190: 36
  1190-1200: 46
  1200-1210: 32
  1210-1220: 66
  1220-1230: 77
  1230-1240: 162
  1240-1250: 90
  1250-1260: 48
  1260-1270: 37
  1270-1280: 31
  1280-1290: 38
  1290-1300: 32
  1300-1310: 70
  1310-1320: 77
  1320-1330: 32
  1330-1340: 24
  1340-1350: 25
  1350-1360: 107
  1360-1370: 88
  1370-1380: 37
  1380-1390: 31
  1390-1400: 26
  1400-1410: 34
  1410-1420: 68
  1420-1430: 34
  1430-1440: 36
  1440-1450: 20
  1450-1460: 42
  1460-1470: 23
  1470-1480: 32
  1480-1490: 57
  1490-1500: 44
  1500-1510: 93
  1510-1520: 170
  1520-1530: 141
  1530-1540: 83
  1540-1550: 74
  1550-1560: 68
  1560-1570: 36
  1570-1580: 87
  1580-1590: 27
  1590-1600: 30
  1600-1610: 35
  1610-1620: 160
  1620-1630: 127
  1630-1640: 69
  1640-1650: 34
  1650-1660: 29
  1660-1670: 14
  1670-1680: 33
  1680-1690: 22
  1690-1700: 49
  1700-1710: 49
  1710-1720: 27
  1720-1730: 61
  1730-1740: 55
  1740-1750: 64
  1750-1760: 41
  1760-1770: 21
  1770-1780: 17
  1780-1790: 32
  1790-1800: 57
  1800-1810: 41
  1810-1820: 46
  1820-1830: 27
  1830-1840: 24
  1840-1850: 21
  1850-1860: 44
  1860-1870: 31
  1870-1880: 58
  1880-1890: 47
  1890-1900: 73
  1900-1910: 40
  1910-1920: 63
  1920-1930: 99
  1930-1940: 106
  1940-1950: 59
  1950-1960: 19
  1960-1970: 42
  1970-1980: 96
  1980-1990: 28
  1990-2000: 16
  2000-2010: 44
  2010-2020: 25
  2020-2030: 14
  2030-2040: 22
  2040-2050: 14
  2050-2060: 20
  2060-2070: 26
  2070-2080: 13
  2080-2090: 23
  2090-2100: 18
  2100-2110: 21
  2110-2120: 30
  2120-2130: 30
  2130-2140: 33
  2140-2150: 21
  2150-2160: 20
  2160-2170: 15
  2170-2180: 15
  2180-2190: 13
  2190-2200: 12
  2200-2210: 12
  2210-2220: 26
  2220-2230: 27
  2230-2240: 14
  2240-2250: 27
  2250-2260: 26
  2260-2270: 56
  2270-2280: 39
  2280-2290: 65
  2290-2300: 37
  2300-2310: 33
  2310-2320: 34
  2320-2330: 32
  2330-2340: 19
  2340-2350: 24
  2350-2360: 32
  2360-2370: 19
  2370-2380: 25
  2380-2390: 18
  2390-2400: 18
  2400-2410: 37
  2410-2420: 15
  2420-2430: 22
  2430-2440: 37
  2440-2450: 15
  2450-2460: 22
  2460-2470: 27
  2470-2480: 76
  2480-2490: 74
  2490-2500: 38
  2500-2510: 19
  2510-2520: 39
  2520-2530: 72
  2530-2540: 89
  2540-2550: 79
  2550-2560: 75
  2560-2570: 35
  2570-2580: 51
  2580-2590: 91
  2590-2600: 61
  2600-2610: 18
  2610-2620: 56
  2620-2630: 98
  2630-2640: 23
  2640-2650: 20
  2650-2660: 19
  2660-2670: 23
  2670-2680: 27
  2680-2690: 16
  2690-2700: 22
  2700-2710: 86
  2710-2720: 85
  2720-2730: 17
  2730-2740: 41
  2740-2750: 21
  2750-2760: 12
  2760-2770: 14
  2770-2780: 16
  2780-2790: 13
  2790-2800: 34
  2800-2810: 23
  2810-2820: 44
  2820-2830: 115
  2830-2840: 41
  2840-2850: 16
  2850-2860: 19
  2860-2870: 62
  2870-2880: 78
  2880-2890: 42
  2890-2900: 48
  2900-2910: 51
  2910-2920: 75
  2920-2930: 72
  2930-2940: 54
  2940-2950: 47
  2950-2960: 20
  2960-2970: 15
  2970-2980: 22
  2980-2990: 18
  2990-3000: 15
  3000-3010: 70
  3010-3020: 41
  3020-3030: 87
  3030-3040: 166
  3040-3050: 128
  3050-3060: 131
  3060-3070: 120
  3070-3080: 42
  3080-3090: 40
  3090-3100: 13
  3100-3110: 11
  3110-3120: 34
  3120-3130: 54
  3130-3140: 35
  3140-3150: 50
  3150-3160: 14
  3160-3170: 19
  3170-3180: 18
  3180-3190: 15
  3190-3200: 35
  3200-3210: 40
  3210-3220: 44
  3220-3230: 110
  3230-3240: 35
  3240-3250: 18
  3250-3260: 19
  3260-3270: 18
  3270-3280: 17
  3280-3290: 26
  3290-3300: 22
  3300-3310: 15
  3310-3320: 13
  3320-3330: 49
  3330-3340: 50
  3340-3350: 24
  3350-3360: 34
  3360-3370: 17
  3370-3380: 92
  3380-3390: 22
  3390-3400: 17
  3400-3410: 35
  3410-3420: 13
  3420-3430: 19
  3430-3440: 30
  3440-3450: 54
  3450-3460: 40
  3460-3470: 106
  3470-3480: 16
  3480-3490: 39
  3490-3500: 30
  3500-3510: 19
  3510-3520: 42
  3520-3530: 35
  3530-3540: 34
  3540-3550: 71
  3550-3560: 55
  3560-3570: 83
  3570-3580: 30
  3580-3590: 25
  3590-3600: 12
  3600-3610: 13
  3610-3620: 34
  3620-3630: 63
  3630-3640: 44
  3640-3650: 20
  3650-3660: 36
  3660-3670: 28
  3670-3680: 39
  3680-3690: 20
  3690-3700: 30
  3700-3710: 33
  3710-3720: 19
  3720-3730: 23
  3730-3740: 75
  3740-3750: 111
  3750-3760: 54
  3760-3770: 22
  3770-3780: 32
  3780-3790: 42
  3790-3800: 29
  3800-3810: 24
  3810-3820: 31
  3820-3830: 32
  3830-3840: 62
  3840-3850: 111
  3850-3860: 107
  3860-3870: 53
  3870-3880: 10
  3880-3890: 14
  3890-3900: 15
  3900-3910: 32
  3910-3920: 20
  3920-3930: 26
  3930-3940: 70
  3940-3950: 72
  3950-3960: 24
  3960-3970: 60
  3970-3980: 37
  3980-3990: 23
  3990-4000: 29
  4000-4010: 21
  4010-4020: 16
  4020-4030: 31
  4030-4040: 25
  4040-4050: 19
  4050-4060: 26
  4060-4070: 12
  4070-4080: 15
  4080-4090: 16
  4090-4100: 15
  4100-4110: 11
  4110-4120: 23
  4120-4130: 50
  4130-4140: 77
  4140-4150: 40
  4150-4160: 47
  4160-4170: 35
  4170-4180: 32
  4180-4190: 18
  4190-4200: 14
  4200-4210: 35
  4210-4220: 11
  4220-4230: 16
  4230-4240: 21
  4240-4250: 61
  4250-4260: 81
  4260-4270: 47
  4270-4280: 63
  4280-4290: 36
  4290-4300: 65
  4300-4310: 23
  4310-4320: 59
  4320-4330: 55
  4330-4340: 13
  4340-4350: 23
  4350-4360: 26
  4360-4370: 41
  4370-4380: 54
  4380-4390: 20
  4390-4400: 23
  4400-4410: 27
  4410-4420: 14
  4420-4430: 51
  4430-4440: 18
  4440-4450: 13
  4450-4460: 21
  4460-4470: 20
  4470-4480: 19
  4480-4490: 28
  4490-4500: 59
  4500-4510: 46
  4510-4520: 37
  4520-4530: 23
  4530-4540: 29
  4540-4550: 19
  4550-4560: 14
  4560-4570: 20
  4570-4580: 32
  4580-4590: 13
  4590-4600: 46
  4600-4610: 52
  4610-4620: 21
  4620-4630: 25
  4630-4640: 47
  4640-4650: 135
  4650-4660: 175
  4660-4670: 156
  4670-4680: 52
  4680-4690: 109
  4690-4700: 34
  4700-4710: 13
  4710-4720: 16
  4720-4730: 35
  4730-4740: 25
  4740-4750: 34
  4750-4760: 18
  4760-4770: 37
  4770-4780: 58
  4780-4790: 81
  4790-4800: 117
  4800-4810: 95
  4810-4820: 32
  4820-4830: 15
  4830-4840: 50
  4840-4850: 61
  4850-4860: 35
  4860-4870: 15
  4870-4880: 5
- /root/chipyard/sims/verilator/generated-src/constellation.test.EvalHarness.EvalTestConfig00/gen-collateral/TestDriver.v:158: Verilog $finish
Results CSV:
ingress_id, egress_id, received, sent, throughput, median_latency, max_latency
0, 0, 16000, 19948, 0.802085, 2902, 4874

Min throughput: 0, 0, 0.802085
Median latency: 2902
Max latency: 4874
Latency hist:   0-10: 0
  10-20: 0
  20-30: 0
  30-40: 0
  40-50: 0
  50-60: 0
  60-70: 0
  70-80: 0
  80-90: 0
  90-100: 0
  100-110: 0
  110-120: 0
  120-130: 0
  130-140: 0
  140-150: 0
  150-160: 0
  160-170: 0
  170-180: 0
  180-190: 0
  190-200: 0
  200-210: 0
  210-220: 0
  220-230: 0
  230-240: 0
  240-250: 0
  250-260: 0
  260-270: 0
  270-280: 0
  280-290: 0
  290-300: 0
  300-310: 0
  310-320: 0
  320-330: 0
  330-340: 0
  340-350: 0
  350-360: 0
  360-370: 0
  370-380: 0
  380-390: 0
  390-400: 0
  400-410: 0
  410-420: 0
  420-430: 0
  430-440: 0
  440-450: 0
  450-460: 0
  460-470: 0
  470-480: 0
  480-490: 0
  490-500: 0
  500-510: 0
  510-520: 0
  520-530: 0
  530-540: 0
  540-550: 0
  550-560: 0
  560-570: 0
  570-580: 0
  580-590: 0
  590-600: 0
  600-610: 0
  610-620: 0
  620-630: 0
  630-640: 0
  640-650: 0
  650-660: 0
  660-670: 0
  670-680: 0
  680-690: 0
  690-700: 0
  700-710: 0
  710-720: 0
  720-730: 0
  730-740: 0
  740-750: 0
  750-760: 0
  760-770: 0
  770-780: 0
  780-790: 0
  790-800: 0
  800-810: 0
  810-820: 0
  820-830: 0
  830-840: 0
  840-850: 0
  850-860: 0
  860-870: 0
  870-880: 0
  880-890: 0
  890-900: 0
  900-910: 0
  910-920: 0
  920-930: 0
  930-940: 0
  940-950: 0
  950-960: 0
  960-970: 0
  970-980: 0
  980-990: 0
  990-1000: 0
  1000-1010: 6
  1010-1020: 22
  1020-1030: 68
  1030-1040: 15
  1040-1050: 14
  1050-1060: 17
  1060-1070: 28
  1070-1080: 24
  1080-1090: 11
  1090-1100: 14
  1100-1110: 20
  1110-1120: 23
  1120-1130: 18
  1130-1140: 42
  1140-1150: 18
  1150-1160: 51
  1160-1170: 43
  1170-1180: 30
  1180-1190: 36
  1190-1200: 46
  1200-1210: 32
  1210-1220: 66
  1220-1230: 77
  1230-1240: 162
  1240-1250: 90
  1250-1260: 48
  1260-1270: 37
  1270-1280: 31
  1280-1290: 38
  1290-1300: 32
  1300-1310: 70
  1310-1320: 77
  1320-1330: 32
  1330-1340: 24
  1340-1350: 25
  1350-1360: 107
  1360-1370: 88
  1370-1380: 37
  1380-1390: 31
  1390-1400: 26
  1400-1410: 34
  1410-1420: 68
  1420-1430: 34
  1430-1440: 36
  1440-1450: 20
  1450-1460: 42
  1460-1470: 23
  1470-1480: 32
  1480-1490: 57
  1490-1500: 44
  1500-1510: 93
  1510-1520: 170
  1520-1530: 141
  1530-1540: 83
  1540-1550: 74
  1550-1560: 68
  1560-1570: 36
  1570-1580: 87
  1580-1590: 27
  1590-1600: 30
  1600-1610: 35
  1610-1620: 160
  1620-1630: 127
  1630-1640: 69
  1640-1650: 34
  1650-1660: 29
  1660-1670: 14
  1670-1680: 33
  1680-1690: 22
  1690-1700: 49
  1700-1710: 49
  1710-1720: 27
  1720-1730: 61
  1730-1740: 55
  1740-1750: 64
  1750-1760: 41
  1760-1770: 21
  1770-1780: 17
  1780-1790: 32
  1790-1800: 57
  1800-1810: 41
  1810-1820: 46
  1820-1830: 27
  1830-1840: 24
  1840-1850: 21
  1850-1860: 44
  1860-1870: 31
  1870-1880: 58
  1880-1890: 47
  1890-1900: 73
  1900-1910: 40
  1910-1920: 63
  1920-1930: 99
  1930-1940: 106
  1940-1950: 59
  1950-1960: 19
  1960-1970: 42
  1970-1980: 96
  1980-1990: 28
  1990-2000: 16
  2000-2010: 44
  2010-2020: 25
  2020-2030: 14
  2030-2040: 22
  2040-2050: 14
  2050-2060: 20
  2060-2070: 26
  2070-2080: 13
  2080-2090: 23
  2090-2100: 18
  2100-2110: 21
  2110-2120: 30
  2120-2130: 30
  2130-2140: 33
  2140-2150: 21
  2150-2160: 20
  2160-2170: 15
  2170-2180: 15
  2180-2190: 13
  2190-2200: 12
  2200-2210: 12
  2210-2220: 26
  2220-2230: 27
  2230-2240: 14
  2240-2250: 27
  2250-2260: 26
  2260-2270: 56
  2270-2280: 39
  2280-2290: 65
  2290-2300: 37
  2300-2310: 33
  2310-2320: 34
  2320-2330: 32
  2330-2340: 19
  2340-2350: 24
  2350-2360: 32
  2360-2370: 19
  2370-2380: 25
  2380-2390: 18
  2390-2400: 18
  2400-2410: 37
  2410-2420: 15
  2420-2430: 22
  2430-2440: 37
  2440-2450: 15
  2450-2460: 22
  2460-2470: 27
  2470-2480: 76
  2480-2490: 74
  2490-2500: 38
  2500-2510: 19
  2510-2520: 39
  2520-2530: 72
  2530-2540: 89
  2540-2550: 79
  2550-2560: 75
  2560-2570: 35
  2570-2580: 51
  2580-2590: 91
  2590-2600: 61
  2600-2610: 18
  2610-2620: 56
  2620-2630: 98
  2630-2640: 23
  2640-2650: 20
  2650-2660: 19
  2660-2670: 23
  2670-2680: 27
  2680-2690: 16
  2690-2700: 22
  2700-2710: 86
  2710-2720: 85
  2720-2730: 17
  2730-2740: 41
  2740-2750: 21
  2750-2760: 12
  2760-2770: 14
  2770-2780: 16
  2780-2790: 13
  2790-2800: 34
  2800-2810: 23
  2810-2820: 44
  2820-2830: 115
  2830-2840: 41
  2840-2850: 16
  2850-2860: 19
  2860-2870: 62
  2870-2880: 78
  2880-2890: 42
  2890-2900: 48
  2900-2910: 51
  2910-2920: 75
  2920-2930: 72
  2930-2940: 54
  2940-2950: 47
  2950-2960: 20
  2960-2970: 15
  2970-2980: 22
  2980-2990: 18
  2990-3000: 15
  3000-3010: 70
  3010-3020: 41
  3020-3030: 87
  3030-3040: 166
  3040-3050: 128
  3050-3060: 131
  3060-3070: 120
  3070-3080: 42
  3080-3090: 40
  3090-3100: 13
  3100-3110: 11
  3110-3120: 34
  3120-3130: 54
  3130-3140: 35
  3140-3150: 50
  3150-3160: 14
  3160-3170: 19
  3170-3180: 18
  3180-3190: 15
  3190-3200: 35
  3200-3210: 40
  3210-3220: 44
  3220-3230: 110
  3230-3240: 35
  3240-3250: 18
  3250-3260: 19
  3260-3270: 18
  3270-3280: 17
  3280-3290: 26
  3290-3300: 22
  3300-3310: 15
  3310-3320: 13
  3320-3330: 49
  3330-3340: 50
  3340-3350: 24
  3350-3360: 34
  3360-3370: 17
  3370-3380: 92
  3380-3390: 22
  3390-3400: 17
  3400-3410: 35
  3410-3420: 13
  3420-3430: 19
  3430-3440: 30
  3440-3450: 54
  3450-3460: 40
  3460-3470: 106
  3470-3480: 16
  3480-3490: 39
  3490-3500: 30
  3500-3510: 19
  3510-3520: 42
  3520-3530: 35
  3530-3540: 34
  3540-3550: 71
  3550-3560: 55
  3560-3570: 83
  3570-3580: 30
  3580-3590: 25
  3590-3600: 12
  3600-3610: 13
  3610-3620: 34
  3620-3630: 63
  3630-3640: 44
  3640-3650: 20
  3650-3660: 36
  3660-3670: 28
  3670-3680: 39
  3680-3690: 20
  3690-3700: 30
  3700-3710: 33
  3710-3720: 19
  3720-3730: 23
  3730-3740: 75
  3740-3750: 111
  3750-3760: 54
  3760-3770: 22
  3770-3780: 32
  3780-3790: 42
  3790-3800: 29
  3800-3810: 24
  3810-3820: 31
  3820-3830: 32
  3830-3840: 62
  3840-3850: 111
  3850-3860: 107
  3860-3870: 53
  3870-3880: 10
  3880-3890: 14
  3890-3900: 15
  3900-3910: 32
  3910-3920: 20
  3920-3930: 26
  3930-3940: 70
  3940-3950: 72
  3950-3960: 24
  3960-3970: 60
  3970-3980: 37
  3980-3990: 23
  3990-4000: 29
  4000-4010: 21
  4010-4020: 16
  4020-4030: 31
  4030-4040: 25
  4040-4050: 19
  4050-4060: 26
  4060-4070: 12
  4070-4080: 15
  4080-4090: 16
  4090-4100: 15
  4100-4110: 11
  4110-4120: 23
  4120-4130: 50
  4130-4140: 77
  4140-4150: 40
  4150-4160: 47
  4160-4170: 35
  4170-4180: 32
  4180-4190: 18
  4190-4200: 14
  4200-4210: 35
  4210-4220: 11
  4220-4230: 16
  4230-4240: 21
  4240-4250: 61
  4250-4260: 81
  4260-4270: 47
  4270-4280: 63
  4280-4290: 36
  4290-4300: 65
  4300-4310: 23
  4310-4320: 59
  4320-4330: 55
  4330-4340: 13
  4340-4350: 23
  4350-4360: 26
  4360-4370: 41
  4370-4380: 54
  4380-4390: 20
  4390-4400: 23
  4400-4410: 27
  4410-4420: 14
  4420-4430: 51
  4430-4440: 18
  4440-4450: 13
  4450-4460: 21
  4460-4470: 20
  4470-4480: 19
  4480-4490: 28
  4490-4500: 59
  4500-4510: 46
  4510-4520: 37
  4520-4530: 23
  4530-4540: 29
  4540-4550: 19
  4550-4560: 14
  4560-4570: 20
  4570-4580: 32
  4580-4590: 13
  4590-4600: 46
  4600-4610: 52
  4610-4620: 21
  4620-4630: 25
  4630-4640: 47
  4640-4650: 135
  4650-4660: 175
  4660-4670: 156
  4670-4680: 52
  4680-4690: 109
  4690-4700: 34
  4700-4710: 13
  4710-4720: 16
  4720-4730: 35
  4730-4740: 25
  4740-4750: 34
  4750-4760: 18
  4760-4770: 37
  4770-4780: 58
  4780-4790: 81
  4790-4800: 117
  4800-4810: 95
  4810-4820: 32
  4820-4830: 15
  4830-4840: 50
  4840-4850: 61
  4850-4860: 35
  4860-4870: 15
  4870-4880: 5

xianyi11 avatar Jul 11 '24 07:07 xianyi11