Operation

---

The behavior of the avatar while operating is similar to the way the self-play agent generates experiences, but simpler.

Once the neural network has been trained, tested and deployed, the avatar is ready for doing what is meant to do: maximizing rewards by the end of an episode. 

Let's recap the loop of interactions of the avatar with the environment, as shown in the image:

• Time Step: an interaction with the environment produces an observation and reward signal from the environment and the agent responds with an action to execute.
• Interval: a constant period of time between time steps.
• End Condition: a condition to stop the loop of time steps.
• Episode:  a sequence of Time Steps

Notice how we have unpacked the whole game framework to create experiences used for learning. During operations you are only interested in the episode. Echo has learnt to act in such a way that his behavior maximizes the rewards by the end of an episode. 

Deep Neural Network

We can't say much, obviously, about what makes Echo a powerful general purpose avatar. What we can say, though, is that the neural network's architecture is a state-of-the-art in AI. The network works with Transformers, instead of convolution or linear layers. This is a technique borrowed from the Natural Language Processing discipline, that works very well in this kind of avatar, for building cognitive and intuition capabilities for solving complex optimization problems, which is a thing Language models are not designed to do.

In general, the default design of the deep neural network would be enough to solve most complex optimization problems. The size of the network is also customizable to the complexity of the problem which is intended to be solved.

Computer Power

This version of the avatar, Echo, is a much powerful version than the previous one, Eva. Eva mimics the exact same design of the workings of the human brain, which implies simulating the imagination as well. Simulating the imagination is an expensive process in terms on compute power. Echo goes beyond the design of our human brain, to take advantage of the AI possibilities, and so it removes the imagination from the design.

This new design saves order of magnitude of computer power, for learning and for operating the avatar, because it saves the expensive simulation of the imagination and also reduces the size of the neural network.

Eva, because of the imagination, requires six deep neural networks that need to be trained, and when generating experiences must activate a sequence of simulations that implies the imagination. This is a very expensive a heavy process that requires strong computer power, which is a problem to use this technology widely. 

Echo, however, requires a single deep neural network. 

During operation, this improvement makes the responses of the avatar super fast when receiving an observation from the environment. With no imagination, the interaction with the environment  is very light, fast, making it possible to operate in most cases with a single CPU.

In summary, Echo is super efficient in terms of compute power, to train the avatar and to operate it.