Policy or Value ? Loss Function and Playing Strength in AlphaZero
Por um escritor misterioso
Last updated 25 março 2025

Results indicate that, at least for relatively simple games such as 6x6 Othello and Connect Four, optimizing the sum, as AlphaZero does, performs consistently worse than other objectives, in particular by optimizing only the value loss. Recently, AlphaZero has achieved outstanding performance in playing Go, Chess, and Shogi. Players in AlphaZero consist of a combination of Monte Carlo Tree Search and a Deep Q-network, that is trained using self-play. The unified Deep Q-network has a policy-head and a value-head. In AlphaZero, during training, the optimization minimizes the sum of the policy loss and the value loss. However, it is not clear if and under which circumstances other formulations of the objective function are better. Therefore, in this paper, we perform experiments with combinations of these two optimization targets. Self-play is a computationally intensive method. By using small games, we are able to perform multiple test cases. We use a light-weight open source reimplementation of AlphaZero on two different games. We investigate optimizing the two targets independently, and also try different combinations (sum and product). Our results indicate that, at least for relatively simple games such as 6x6 Othello and Connect Four, optimizing the sum, as AlphaZero does, performs consistently worse than other objectives, in particular by optimizing only the value loss. Moreover, we find that care must be taken in computing the playing strength. Tournament Elo ratings differ from training Elo ratings—training Elo ratings, though cheap to compute and frequently reported, can be misleading and may lead to bias. It is currently not clear how these results transfer to more complex games and if there is a phase transition between our setting and the AlphaZero application to Go where the sum is seemingly the better choice.

AlphaZero Explained · On AI

How to build your own AlphaZero AI using Python and Keras, by David Foster, Applied Data Science

Representation Matters: The Game of Chess Poses a Challenge to Vision Transformers – arXiv Vanity

AlphaDDA: strategies for adjusting the playing strength of a fully trained AlphaZero system to a suitable human training partner [PeerJ]

AlphaDDA: strategies for adjusting the playing strength of a fully trained AlphaZero system to a suitable human training partner [PeerJ]

Multiplayer AlphaZero – arXiv Vanity

Policy or Value ? Loss Function and Playing Strength in AlphaZero-like Self- play

AlphaZero Explained · On AI

Frontiers AlphaZe∗∗: AlphaZero-like baselines for imperfect information games are surprisingly strong

Frontiers AlphaZe∗∗: AlphaZero-like baselines for imperfect information games are surprisingly strong

Policy or Value ? Loss Function and Playing Strength in AlphaZero-like Self- play

Reimagining Chess with AlphaZero, February 2022

Strength and accuracy of policy and value networks. a Plot showing the
Recomendado para você
-
Simple Alpha Zero25 março 2025
-
Google's self-learning AI AlphaZero masters chess in 4 hours25 março 2025
-
AlphaGo - How AI mastered the hardest boardgame in history25 março 2025
-
Efficient Learning for AlphaZero via Path Consistency Poster25 março 2025
-
Alpha S 2 Pickleball Paddle Bundle - Pickleball Paddle Shop25 março 2025
-
Dr. Rudolf Posch: Neural Network AlphaZero wins in Chess, Shogi and Go25 março 2025
-
DeepMind: the existence proof for RL at scale, by Nathan Lambert25 março 2025
-
Alpha Kappa Alpha Sorority, Incorporated - Rho Xi Omega Chapter25 março 2025
-
AlphaZero: Shedding new light on chess, shogi, and Go - Google DeepMind25 março 2025
-
Move over AlphaGo: AlphaZero taught itself to play three different games25 março 2025
você pode gostar
-
Let's PLAAY Tennis Fast-Action Cards — PLAAY Games25 março 2025
-
Impressão Em Tela Câmera Pencil Vs - Tigre, Coruja, Homem25 março 2025
-
Dragon Ball Kai - finally properly organized episodes : r/PleX25 março 2025
-
Papa's Hot Doggeria HD #39: Hot Diggity Dog25 março 2025
-
Experience all of DirectX 12 Ultimate with 3DMark – Hartware25 março 2025
-
McDonald's resgata Papa Burger para promover cardápio25 março 2025
-
Quem acharia legal um personagem Dendro? Genshin Impact25 março 2025
-
Electronic Arts - UEFA EURO 2024™ Comes to EA SPORTS FC™ 24, EA25 março 2025
-
How to peek in CS:GO – types of peeks, full guide25 março 2025
-
Mamahaha no Tsurego ga Motokano datta (DVD) (2022) Anime25 março 2025