Zhang Wenduo said, "I thought of that problem too. The traditional approach would definitely be to wait until an entire batch finishes training, update the model with the new parameters, and then run the next batch. But that's for ordinary models, where long-horizon tasks aren't numerous enough or long enough. In an experiment like mine, the traditional algorithm only gives an average Computing Card utilization of a little over twenty percent."
A little over twenty percent. Zhao Wenyuan repeated the figure to himself. Glutinous Rice Ball's current post-training tasks averaged a little over forty percent utilization-twice as much.
Zhang Wenduo looked around but found no pen or paper on Zhao Wenyuan's desk. He simply flipped over his own equipment request form, pulled a ballpoint pen from his trouser pocket, and started writing on it.
"Director, look. This is the new architecture I came up with."
He drew a circle in the center of the sheet, with two rows of little figures on either side, one labeled "Actor" and the other "Learner." The circle in the middle was the experience pool.
In the traditional learning method, once all the actors finished acting, they produced a batch of results. The learners incorporated those results into the original model, and the actors then acted based on the new version of the model.
The cycle repeated like that-model, actors, learners, model, actors, learners-until the final results reached a relative optimum with no room for improvement. At that point, training was considered complete.
In Zhang Wenduo's new architecture, however, the actors and learners did not wait for each other. The actors kept acting and deposited their results in the experience pool when they finished. Once the experience pool contained enough data, the learners incorporated that experience back into the model.
"You mean abandoning the synchronous