December 16th, 9:30 AM.
Zheng Xiaobo's office is on the 49th floor of Dingsheng Tower.
Cheng Yuan and Liu Dahai sat side by side on the sofa. Cheng Yuan was wearing a light gray suit with a black folder on his lap. Liu Dahai was still wearing the same gray hoodie.
Zheng Xiaobo sat opposite him, and there was a cup on the table containing Evian mineral water.
"Tell me about it, let's review what happened."
Cheng Yuan opened the folder.
"Boss Zheng, it's my fault. I wasn't in the flow of the entire meeting."
Let's put the pot down first.
Zheng Xiaobo raised an eyebrow; he hadn't expected Cheng Yuan.
Cheng Yuan turned to the first page where he had marked it.
"Dr. Liu gave the presentation first, and their reaction was lukewarm. Then he gave Dr. Liu and me a reverse presentation."
He glanced at Liu Dahai.
"I'm not very knowledgeable about the technical aspects; let Dr. Liu explain in detail."
Liu Dahai didn't understand the implication and honestly continued, "I first showed them Kun Yuan, then they pulled out their own model, saying it was an adjusted version of the open-source 7B model, and asked me to create some tests. I tested three questions, all of which I did in parallel using Kun Yuan. None of them were publicly available test questions, and their model performed better than Kun Yuan across the board."
"Then they showed us the benchmark scores, trying to understand this. They got 85.2 points, while Kun Yuan only got 78." Liu Dahai finished speaking, still with a hint of bitterness. Anyone would feel bad having their hard-earned results from leading a team for months crushed by someone else. Cheng Yuan Cheng Yuan that the blame had been shifted, quickly added, "But he didn't come to reject us. He brought his own plan, asking Dingsheng to convert the GPU usage rights into investment, in exchange for shares."
The implication is that their demands were outside my authority, making negotiation impossible; it wasn't my fault in the war.
Zheng Xiaobo frowned slightly. "Whose shares are we exchanging?"
"This is the key point; it's not Source Code Technology, it's a new company. He said he and Zhao Wenyuan are preparing for it, and their focus is on developing general-purpose base models."
Cheng Yuan closed the folder and placed his hands on it.
"They didn't provide the company name, registration status, or team structure. They only said 'under preparation,' which either means they haven't actually registered yet, or they're deliberately preventing us from conducting due diligence."
Zheng Xiaobo looked at Cheng Yuan quietly without saying a word.
Last week, he stood in front of the floor-to-ceiling window and thought Han Luyi was arrogant and ignorant.
Unexpectedly, it was I who was the one who lacked experience.
"Is the technology real?" Zheng Xiaobo asked.
As the helmsman of a tech giant like Dingsheng, he doesn't need to understand the technical details, but he must maintain a keen awareness of the boundaries of technology. What can be done, what can't be done, what is real, and what can be faked—these are his first reactions. Cheng Yuan didn't answer, but turned to look at Liu Dahai.
Liu Dahai said, "I've thought about it at first, but it should be true."
Zheng Xiaobo stared at him.
Is it possible that they are lying to us?
Liu Dahai shook his head. "That's impossible. You can improve your benchmark scores, but the on-site test is the one I set. There's no way to prepare for that, and the difference is, how should I put it, very obvious."
Zheng Xiaobo leaned forward.
What about overfitting?
Liu Dahai was taken aback; he hadn't expected Zheng Xiaobo to ask such a professional question.
"Boss Zheng looked into machine learning before?"
"Just tell me, is that even possible?"
Overfitting is not a new concept that originated in the era of Large Model.
In machine learning, data is divided into training sets and test sets. The training set is used to teach the model, and the test set is used to evaluate the model.
However, you can't directly use the training set as the test set, because the model will memorize the answers.
It's like trying to teach an elementary school student addition. If you teach one Add One and then test them on one Add One, you don't know if they've memorized it or if they truly know it. Only when you teach one Add One and then test them on twenty-three plus nineteen can you verify whether they truly know it.
Overfitting occurs when the model memorizes the answers from the training set, resulting in artificially inflated scores.
Liu Dahai organized his thoughts and replied:
"I've thought about it, but it's unlikely. The three test questions came from completely different sources. One was selected from Dingsheng's internal question bank, one I typed out on the spot, and the third was the most difficult one from my private question bank, which even few people inside the company have seen. Moreover, the three questions covered three different types of intent inference, and they all hit the mark accurately. Finally, they also showed me the complete baseline data, 1,200 questions, and cross-validation across 16 scenarios. They didn't just pick the best ones to show me."
"Well, to be honest, that 85.2 is definitely the real deal."
The office fell silent.
The sound from the air conditioner vents suddenly became very clear.
Cheng Yuan subconsciously closed the folder, swallowed hard, and then awkwardly heard the loud sound of his own swallowing. Liu Dahai stared at the vase on the coffee table, which contained a few withered cotton stalks.
Zheng Xiaobo's face remained calm, but his hand remained gripping the cup, neither lifting it nor letting go.
After a while, Zheng Xiaobo slowly spoke, "How did he do it? Do you have any ideas?"
Liu Dahai took out his laptop from his backpack, walked to Zheng Xiaobo's desk, turned on the computer, and the screen lit up.
The above is an open document.
<Tangyuan Model - Intent Understanding: Analysis of Possible Technical Paths>
"I wrote this last night," Liu Dahai said.
The tech person's attempt to take credit was a bit too direct.
He pointed the screen at Zheng Xiaobo and pushed it forward.
"Let's start with the process of elimination; specifying that it's not an architectural innovation is the first step."
"Intending to understand this track, OpenAI's GPT is now generally considered the ceiling, scoring just over 78 points. They took an open-source 7B architecture with only a fraction of the parameters and scored 85.2. Boss Zheng, let me put it this way: it's like a poor student from our county high school who self-studies using someone else's old textbooks and scores 30 points higher than the top scorer in the province on the college entrance exam. I can say with certainty, it's definitely not architectural innovation; the 7B architecture has only so much to offer, there's nothing particularly groundbreaking about it."
"The competition for Large Model has moved beyond the parameter arms race. With hundreds of billions or trillions of parameters, everyone's architecture is pretty much the same. The Transformer is still the same Transformer; it's just a matter of adding or subtracting layers, and adding or subtracting attention points."
As Liu Dahai spoke, he carefully observed Zheng Xiaobo's expression.
When technical staff report to their superiors, they are afraid of revealing too much technical detail and being misunderstood, or they are afraid of being too brief and unsatisfactory. They can only be cautious and adjust their approach as needed.
"The only explanation I can think of is that the students in this county aren't using other people's old textbooks, but rather a set of golden review materials that no other student in the world has."
Zheng Xiaobo remained expressionless.
At this Cheng Yuan interjected—he hadn't understood, but that was alright; his main concern was that even Boss Zheng might not have understood, and he couldn't let Boss Zheng ask such a question. "Dr. Liu, excuse me, what did you mean by that analogy?"
"It's Annotated Data." Liu Dahai realized he'd gone off-topic and quickly steered the conversation back on track. "Boss Zheng already has a general understanding, so I'll just give a brief overview."
He turned the computer around, searched for a flowchart online, and then turned it back to face Zheng Xiaobo.
"Our training of Large Model mainly consists of several stages. The first is pre-training, which involves piling up data; we want to include everything on the Internet."
"Then comes the supervision and fine-tuning, where a group of people demonstrate for the model, teaching it to understand what good and bad words are."
"The next step is alignment, teaching the model to align itself according to human thought, so that its responses sound more like normal human speech."
Liu Dahai pointed to the flowchart on the screen.
"In the first step, everyone uses roughly the same corpus; there's only so much data on the internet, so everyone can use it. The real difference lies in the last two steps: whoever has higher quality data annotation will train a stronger model."
"That's why I said... the other party's results are so good, they must have good Annotated Data."
Liu Dahai forgot Han Luyi's name and the company's name, so he had to change his mind halfway through.
No, I need to check out this company and the technical lead when I get back, and look at his papers.
Creating a model trained to this level requires substantial academic background.
Upon hearing this, Zheng Xiaobo, who had been silent all along, suddenly spoke up, "ScaleAl."