Karl Sundequist Blomdahl

Use a MuZero-style model for predictions

23

As mentioned in https://github.com/Chicoryn/dream-go/issues/56, change the prediction model to a MuZero style architecture. This should have several advantages, such as allowing us to have an asymmetric representation and dynamics network...

NNUE (ƎUИИ Efficiently Updatable Neural Network) for Go

5

NNUE (ƎUИИ Efficiently Updatable Neural Network) is a sparse shallow neural network architecture that can be calculated incrementally. This has proven incredibly successful in [Chess](https://github.com/glinscott/nnue-pytorch/blob/master/docs/nnue.md) and Shoigi. Historically algorithms that...

GPU vs CPU matrix multiplication

1

At the end of the neural network we use for prediction we perform a number of small matrix multiplication to calculate the final _policy_ and _value_ values. These might be...

Monte-Carlo tree search as regularized policy optimization

3

> The combination of Monte-Carlo tree search (MCTS) with deep reinforcement learning has led to significant advances in artificial intelligence. However, AlphaZero, the current stateof-the-art MCTS algorithm, still relies on...

MLP-Mixer: An all-MLP Architecture for Vision

7

https://arxiv.org/pdf/2105.01601.pdf > Convolutional Neural Networks (CNNs) are the go-to model for computer vision. Recently, attention-based networks, such as the Vision Transformer, have also become popular. In this paper we show...

Some impressive results that might be useful when transitioning to quantized models again. https://neuralmagic.com/blog/benchmark-yolov3-on-cpus-with-deepsparse/

In cuDNN 8.2 the swish activation function was introduced, this is an activation function that has been very successfully applied in networks such as MobileNetV3 and EfficientNet. It is worth...

We previously implemented quantized convolution in #8, but due being temporarily unsupported by tensor cores we switched to `TRUE_HALF_CONFIG` instead. Based on benchmarks it seems like we could acquire an...

Prune nodes from the search tree that are obviously bad

1

A lot of game engines in the 12th Computer Go UEC Cup [1] used a heuristic to prune "obviously bad" nodes from the search tree. This idea might be worth...

With the deprecation of the Sabaki specific analyze commands (which were very useful for debugging MCTS), we should implement the leela-zero analyze commands instead as they are widely supported by...