Karl Sundequist Blomdahl
Karl Sundequist Blomdahl
@sethtroisi From a quick survey of the paper, the Squeeze-Excite (SE) approach looks very similar to what @lightvector has been doing with global properties. The main difference seems to be...
The chess architecture is based on a denormalization of the features based on the kings position `(our_king_square, piece_square, piece_type, piece_color)`, and shogi uses a similar representation. The first challenge is...
All search algorithms needs to weight the benefits of quality vs quantity of nodes explored. NNUE is a step towards quantity, where as all recent Go algorithms trend towards quality...
All current NNUE architectures are pure _value_ networks which outputs `0` or `1` depending on whether the current state in a losing state or not. This is not necessarily a...
Continuing on the topic of the _MuZero Embedding representation_. It occurred to us that you can view the dynamics function as an RNN (or as it's more successful implementations GRU...
# Experimental results Experimental results of using a large representation network with a small dynamics network so far show negative results, the model is bottlenecked by the dynamics network and...
## Current implementation ```rust pub struct SmallChildrenImpl { /// The number of times each edge has been traversed. pub count: [i32; SMALL_SIZE], /// The number of virtual losses each edge...
Some preliminary benchmarks for the different memory layouts (using a simple UCT benchmark over the different structures, in this case using `f32x4` but performance was about the same with `f32x8`)....
For the cuDNN GRU implementation we need some references, since the official documenation is a bit lacking imo: - https://github.com/tensorflow/tensorflow/blob/eced720b05f71a46a721b19962decf7e0776cc54/tensorflow/stream_executor/cuda/cuda_dnn.cc#L1578 Fun fact, tensorflow seems to have two implementation of `GRUCell`...
Now that everyone is functionally correct it is time to do some performance tuning. We get the following GPU trace and profile for `initial_predict`: ``` 6.04518s 2.4000us - - -...