"What?" Han Luyi asked.
He knew that because of the similarity between Large Model and the human brain, many computer scientists who study Artificial Intelligence read books on neuroscience, and he himself had read several widely circulated bestsellers.
But when it comes to the details, he's completely lost.
Jiang Songran, who was standing next to him, was also completely baffled.
But seeing Zhao Wenyuan's excited expression in the video, both of them realized that the director of the Research Institute probably had something new to offer.
"It's the options themselves," Zhao Wenyuan said. "System 1 excels at yes/no judgments, and it can handle multiple-choice questions, but once the number of options exceeds a certain limit, System 1 becomes almost incapable of making a choice."
"Let me give you an example. If I asked you now if you wanted to drink Douzhi), wouldn't you be able to answer immediately?"
Jiang Songran, but Han Luyi's expression had already turned ugly.
This is indeed a question that can be answered with System 1.
"If I asked you whether Vim or Emacs is better, would you be able to give me an immediate answer?" Zhao Wenyuan continued.
Han Luyi and Jiang Songran spoke almost simultaneously. "Of course it's Vim." This was Han Luyi's answer.
"Emacs." That's what Jiang Songran said.
The two exchanged a quick glance, but the meaning in their eyes was different.
Han Luyi's eyes clearly said, "So you're an Emacs user." Jiang Songran, felt uneasy, realizing he had chosen the wrong side in the worst possible situation.
Zhao Wenyuan, who had deliberately asked the question, was secretly chuckling to himself. He had already noticed through observation that the two had different preferences for code editors.
"But if I asked you now to choose one jam from twenty kinds, you would almost certainly not be able to make a choice in an instant; if I asked you to write a two-hundred-word article right now, don't underestimate two hundred words, this is not a task that System 1 can complete. Do you see the problem?" Zhao Wenyuan said.
Han Luyi moment and said, "You mean the output format, right? Whether it's a yes/no question or a binary choice question, their output format is extremely simple; they don't need to be generated Token."
Jiang Songran added from the side, "It sounds like there's also the issue of the answer domain. The wider the range of the output answer domain, the more complex the calculation becomes."
At this moment, both of them realized why Zhao Wenyuan had brought up the topic of System 1 and System 2 for discussion.
Because the computation task involves complexity prediction, classification, and scheduling, it is inherently a System 1 task with an extremely narrow answer domain, essentially falling into three categories: simple, medium, and complex. If you want to categorize it more finely, five categories would be more than enough.
Han Luyi asked, "So, for this type of task, is there a better solution than using the lightweight version of Glutinous Rice Ball?"
In the video, Zhao Wenyuan wore a mysterious smile, "Yes, Boss Han, yes." Then he began to explain, "When the TransFramer architecture first came out, it used an encoder-decoder architecture. However, current Large Model similar to the PT architecture, including Glutinous Rice Ball, all use an architecture with only a decoder. This is, of course, because this approach is excellent enough to solve very complex problems. But in reality, different approaches are suitable for different problems. For the classification problem we just discussed, there is a more suitable architecture, which is an encoder-only architecture."
Jiang Songran exclaimed in surprise, "You mean Bert?"
Han Luyi beat slow to react. BERT is a technology approach proposed by Google in 2018. It was very popular in the field of machine learning before PT came out, but as PT-like models became the mainstream in the market, it has gradually fallen out of the spotlight.
"You mean?" Han Luyi asked.
"Traditional BERT algorithms typically operate on the scale of hundreds of millions of parameters. Moreover, because they don't require autoregression to generate predictions, they don't need to perform repeated forward computations; they can reach a conclusion in just one step. For the classification task we just discussed, this can be achieved in under 50 milliseconds."
"How much?" Jiang Songran's breathing became rapid.
"That's not all. What I just mentioned was a general classifier. Using this approach, we can train a classifier that specializes in scheduling tasks. The scheduling data we've accumulated since the Glutinous Rice Ball API was released is sufficient for training. In this case, we don't even need hundreds of millions of parameters; we can compress the parameters to the tens of millions level. This kind of model's single computation—"
Zhao Wenyuan spoke at length, becoming increasingly excited as he went on. He paused deliberately at this point, then held out both hands, fingers Zhang Kai, and said, "I'll say a number: 10 milliseconds."
"Impossible!" Jiang Songran blurted out.
Han Luyi didn't comment; they're still just discussing the technical direction. A task classification speed of 10 milliseconds is indeed astonishing. This speed can even match classification using hard-coded rule sets, let alone dynamic classification using models.
However, even if it doesn't reach 10 milliseconds, 50 milliseconds is enough to run this new classifier model for every Task Scheduling. If the accuracy of Glutinous Rice Ball 1.0 can be maintained, the efficiency of the entire scheduling layer can be improved by a lot.
With the same Hardware and the same Computing Power cluster, this optimization can achieve several times the performance of before. "Wenyuan, how long have you been thinking about this technical idea?" Han Luyi asked.
Zhao Wenyuan in the video paused for a moment, then chuckled twice and said, "Boss Han, I just came up with this idea during our discussion."
Wow, that sounds pretty good, and it was actually just brainstormed.
"How long will it take to do this technical verification?" Han Luyi continued to ask.
"With tens of millions of parameters, including the initial data cleaning and subsequent parameter tuning, I estimate we can produce a prototype in one or two weeks," Zhao Wenyuan said after thinking for a moment.
Then he glanced at Han Luyi and suddenly had a thought, "Boss Han, would you like to—"
Han Luyi interrupted him, "I've been very busy lately and don't have time to clean the data for you. You should first have someone build this model and let Old Jiang test it to see the classification accuracy and the actual latency in the production environment."
Zhao Wenyuan hesitated for a moment, then asked, "Boss Han, what is the priority for this?"
Han Luyi and said, "If it's really as useful as you say, you can save half of the training Computing Power you need. How much priority do you think that would be?"
Zhao Wenyuan's expression turned solemn.
Got it, highest priority.
Jiang Songran also spoke up at the opportune moment, "President Zhao, let me know as soon as your new model is ready, and I'll arrange for the Infrastructure Department to test it immediately." After giving Zhao Wenyuan and Jiang Songran their tasks, Han Luyi's thoughts drifted away again.
They were just discussing complexity prediction and classification tasks in Computing Power Scheduling, but in real-world use cases, there are many scenarios where the answer space is limited and fast computation is required.
For example, in Riding the Wind's Task Scheduling, if the backend uses a hybrid model, which tasks should use the flagship model and which should use the Lightweight Model? Or, when Glutinous Rice Ball answers questions, which tasks can utilize existing knowledge and which require online information searching? Even outside of Origin's existing product system, what if this new model were sold as a standalone product? Would it be suitable for certain business scenarios? Which user feedback could be automatically responded to, and which should be transferred to human customer service?
In short, this architecture is naturally suited for any task that involves classification.
"Boss Han, this new model doesn't have much to do with Glutinous Rice Ball). Should we give it a new name?" Zhao Wenyuan asked.
Han Lu thought, "Yes, it's definitely necessary."
Since Glutinous Rice Ball are already a food, let's use food as the model for the new one too.
He quickly thought of a name and said, "Let's call Douzhi..."