Paired Open-Ended Trailblazer (POET): Endlessly Generating Increasingly Complex and Diverse Learning Environments and Their Solutions — part 6/6

Cleaned-text code points [77468, 80447). Bibliography omitted here; source.md includes it.

POET adopts a transfer mechanism that allows one agent (the emigrant ) to attempt to transfer its learned capabilities to a target environment , possibly replacing the current target environment’s agent. This transfer mechanism reflects the idea that skills gained when interacting with one environment might serve as a stepping stone to improved performance when interacting with another environment. We do not know at any given time which environment’s mastery might boost another’s, but we can always perform experiments to see whether such a transfer might improve performance in the target environment. As results support, the richness of simultaneous transfer attempts in a divergent coevolving system can lead to a succession of breakthroughs that would not be possible through a simple monotonic succession of improvements on a single environment.

Algorithm 4 illustrates the function EVALUATE_CANDIDATES() that facilitates this transfer process. Given a list of M M input candidate agents, denoted as θ 1 \theta^{1} , θ 2 \theta^{2} , …, θ M \theta^{M} , and a target environment E ⁡ ( ⋅ ) E(\cdot) , this function first concatenates the list with proposals , which are calculated by optimizing each of θ 1 \theta^{1} , θ 2 \theta^{2} , …, θ M \theta^{M} in E ⁡ ( ⋅ ) E(\cdot) with one ES step. Then each agent in the augmented list (the original M M agents and each of their M M proposals) is evaluated in E ⁡ ( ⋅ ) E(\cdot) and the one with the highest reward is returned. As illustrated in Algorithm 2 , the agent returned from this function will replace the agent currently paired with E ⁡ ( ⋅ ) E(\cdot) if the former performs better than the latter in E ⁡ ( ⋅ ) E(\cdot) . We refer to the replacement as transfer , and there are two types: the situation when the replacing agent is from the input candidate list (i.e. without an optimization step on E ⁡ ( ⋅ ) E(\cdot) ) is referred to as a direct transfer , while if the replacing agent is one of the proposals, it is called a proposal transfer .

## Instructions for reporting errors

We are continuing to improve HTML versions of papers, and your feedback helps enhance accessibility and mobile support. To report errors in the HTML that will help us improve conversion and rendering, choose any of the methods listed below:

Click the "Report Issue" ( ) button, located in the page header.

Tip: You can select the relevant text first, to include it in your report.

Our team has already identified the following issues . We appreciate your time reviewing and reporting rendering errors we may not have found yet. Your efforts will help us improve the HTML versions for all readers, because disability should not be a barrier to accessing research. Thank you for your continued support in championing open access for all.

Have a free development cycle? Help support accessibility at arXiv! Our collaborators at LaTeXML maintain a list of packages that need conversion , and welcome developer contributions .