Neural Teleportation

Neural Teleportation

Blog Article

In this paper, we explore a process called neural teleportation, a mathematical consequence of applying quiver representation theory to neural networks.Neural teleportation teleports a network to a new position in the weight space and preserves its function.This phenomenon comes directly from Cleanser the definitions of representation theory applied to neural networks and it turns out to be a very simple operation that has remarkable properties.

We shed light on the surprising and counter-intuitive consequences neural teleportation has on the loss landscape.In particular, we show that teleportation can be used to explore loss level cruiser-choc-chip-suede-spotted-hair curves, that it changes the local loss landscape, sharpens global minima and boosts back-propagated gradients at any moment during the learning process.