To avoid the peak of the return-to-work period, Han Luyi planned to set off for Haicheng on the fifth day of the Lunar New Year, and had already discussed it with Zhou Min in advance.
Strangely, Zhou Min, who never did this in previous years, specially invited a God of Wealth to her home on the evening of the fourth day of the Lunar Han Luyi.
The God of Wealth was invited in and set up. Zhou Min burned incense and then persuaded Han Luyi to also pay his respects.
Han Luyi thought to himself, "My mother is usually the most opposed to feudal superstitions. Is it because she's getting old that her beliefs have changed?"
After finishing her prayers, Zhou Min sat down on the sofa, picked up the remote control, but didn't immediately turn on the TV.
Han Luyi prepared to return to her room to work.
Zhou Min suddenly called out to him.
"Lutou, are you busy with work in Haicheng?"
Han Luyi was taken aback; the question seemed abrupt. "It's alright."
Zhou Min hummed in agreement, then said, "Lutou, don't overwork yourself. No matter what you do, your health is the most important thing."
Han Luyi hidden meaning in his mother's words and said, "Mom, what are you trying to say? Just say it, I can handle it."
He knew it wasn't a health issue. The first thing he did when he got home was to use a 360-degree Sight to give his mother a full-body scan, and she looked even healthier than she had during the National Day holiday.
Zhou Min stood up, looked at Han Luyi, and slowly said, "I know you were laid off last year, but you didn't tell me because you didn't want me to worry."
She paused, then continued, "Lutou, starting a business isn't easy. Don't be afraid of any big storms. I've saved some money over the years, and I've also saved the money you've sent me. If things really don't go well, we can quit. You can come back, and Mom can support you, okay?"
Han Luyi this, but her throat also ached slightly. She turned her head away from her mother, a hint of surprise in her voice. "Mom, how did you know? Did Biao tell you?"
As soon as he spoke, he realized his voice was a little hoarse.
Zhou Min smiled faintly, with a hint of smugness. "I knew before you came back. You two kids, putting on an act for me. When you were little, I always noticed you didn't do your homework."
Then she explained, "Last time, Teacher Xiao Wang made a timetable using Open Things, and she really liked it. She keeps browsing your WeChat official account. Once, she showed me a video of your speech and kept praising you while watching it. I was quite proud at the time, but I didn't tell her that it was my precious son."
Han Luyi felt a little embarrassed when he heard this. He thought he had kept it a secret from his mother quite well, but it turned out that he was the one who had been kept in the dark.
"Mom----um----" Han Luyi didn't know what to say for a moment.
"You've always been a child who never caused us any trouble. You're so outstanding, Mom believes you'll only get better and better." Zhou Min said, her expression seemingly lost in some memories.
They were such tiny little ones back then, and in the blink of an eye, they've grown up so much.
He's no longer the Lutou who clings to his mother's leg, crying and begging for a hug.
"Go to sleep early, you have to drive tomorrow. Let Xiao Zhang drive more; I think he's more reliable than you."
Zhou Min shook her head, seemingly not intending to delve any further into the topic. She gently patted Han Luyi on the back, then turned and walked back to the bedroom.
After his mother went into the bedroom, Han Luyi turned his head back. He looked at the God of Wealth Statue in the cabinet and realized that his mother still had an extra layer of worry and concern.
But I felt a mother's concern.
Very nice.
Upon returning to Haicheng, the first thing he did was to check the progress with Zhao Wenyuan unexpectedly gave him a pleasant surprise.
Instead of going to the office, the two found a coffee shop near Zhao Wenyuan's house.
On the afternoon of the fifth day of the Lunar New Year, most people were still on holiday, and the coffee shop was much busier than on a regular weekday.
The three of them found a table and sat down, each ordering a hot latte.
Han Luyi and Zhang Biao had just finished a long drive and both looked a bit tired. Zhao Wenyuan, on the other hand, was full of energy.
"Boss Han, good news! The pre-training progress has stabilized these past two days," Zhao Wenyuan said excitedly. "The next step is annealing."
Zhang Biao had already taken out his notebook. Upon hearing the term, he thought he had misheard.
He's much more proactive now, knowing he'll have to deal with these things sooner or later, and even if he doesn't understand the principles, he wants to get a general idea.
So he raised his hand, like a primary school student asking a question in class.
Zhao Wenyuan looked at Zhang Biao and asked, "What's wrong?"
"What is annealing?" Zhang Biao asked.
"Annealing is—" Zhao Wenyuan thought for a moment, "Annealing."
Zhang Biao's head was covered in question marks. What's the difference between your explanation and no explanation at all?
Han Luyi couldn't help but chuckle and explained, "Annealing is originally a metallurgical term, but it was used in Model Training because of its descriptive nature."
Zhang Biao thought for a moment and said, "Is it the 'whoosh' sound when a blacksmith strikes iron into the water?"
Zhao Wenyuan couldn't help but correct him, "What you're talking about is quenching, which in Chinese means quenching, and it requires rapid cooling."
Annealing is different; it involves slow cooling.
Han Luyi said, "Don't worry about it, it's just a metaphor, that's roughly the idea."
Then Han Luyi took the notebook and pen from Zhang Biao and started drawing on them while saying, "As we've mentioned, Model Training is divided into pre-training and post-training. But in the final stage of pre-training, there's a special stage called annealing."
At this point, Han Luyi had already drawn a simple diagram on his notebook and marked one of the places with his pen.
"Here, the data fed in earlier is already plentiful and varied enough, but it's all background knowledge. To improve the model's basic capabilities, we need to provide a lot of high-quality corpora at this stage, such as selected code repositories and literary publications."
Han Luyi finished drawing, he felt that his explanation wasn't clear enough, so he added, "If the initial stage was like learning knowledge from elementary school to the second year of high school, now it's like reviewing past exam papers in the third year of high school. It's not about learning everything, but only the most classic and representative questions."
Then he handed the notebook back to Zhang Biao.
Zhang Biao took the notebook, but he didn't really empathize with Han Luyi's example because he hadn't practiced any past exam questions.
However, he felt that this time, listening to the two top students discuss things, he was no longer as confused as before.
"This stage is generally quite short," Han Luyi said, turning to Zhao Wenyuan, "How much longer do you estimate it will take?"
Zhao Wenyuan held out one hand. "At the slowest, five days will be enough."
Then he withdrew his hand, picked up his latte, and took a sip.
"The annealing data was prepared by my team beforehand: 30 million high-quality code snippets, a selection of books and academic papers from various fields."
"What's the reason?" Han Luyi asked. "Is it because the curve became more stable after the data issue was fixed on the first day of the Lunar New Year?"
"That's part of it," Zhao Wenyuan nodded. "The main reason is that the cleaning rules you wrote earlier yielded higher returns than expected, and the quality of the raw data was significantly higher than industry standards. With less invalid data, the actual effective learning efficiency is much higher for the same number of training steps."
Han Luyi nodded. "Products from Brother Sight are reliable."
As Zhao Wenyuan spoke, he took out his phone and brought up a graph.
"This is a comparison before and after the correction. I was worried about fluctuations in the final stage, so I left a relatively large margin of error. But now it seems unnecessary, and we can directly use the annealing data."
Han Luyi looked at the curve but didn't speak immediately.
Zhao Wenyuan continued, "After the annealing is complete, we can first conduct a round of basic ability assessments, including coding, mathematics,..."
We'll run through Chinese comprehension, long contextualization, and general knowledge tests. Once the foundational capabilities are met, we can move on to post-training, where the Annotated Data will then begin to take effect.
"Don't just look at publicly available rankings," Han Luyi said. "Evaluation sets should be divided into two layers: one is the basic capabilities that can be compared externally, and the other is our own product's preliminary tasks."
He leaned forward slightly as he spoke, "Unlike typical research institutions, we're not making Glutinous Rice Ball glutinous rice balls) for the purpose of publishing papers. We already have real-world applications, Riding the Wind or Open Things, which is our advantage. We must ensure that Glutinous Rice Ball has that potential."
Zhao Wenyuan was taken aback. Indeed, he hadn't considered this aspect of integrating it with the product.
When he was in the Research Institute, he did research, and he didn't need to consider the ability to integrate it with products. He felt that if the basic capabilities were in place, the application would naturally be found.
It's like making the hammer first, then finding the nails.
However, the situation at Source Intelligence is different now. Source Code Technology's two application-layer products have already opened up the market. Glutinous Rice Ball are made, there is no need to consider which scenarios they can be used in.
The scenes of Riding the Wind and Open Things have been prepared.
Zhao Wenyuan nodded, quickly jotted down a note on his phone, and said, "Okay, Boss Han, I'll arrange for the team to create our own assessment kit as soon as we start work."
After arranging the work, Han Luyi silently calculated the time in his mind: five days for annealing, which would be February 15th.
It started a few days earlier than February 20th, when training was initially expected.
This bought a few more days for the migration of the Kaiwu Overseas Edition backend.
Jiang Songran's side also needs to hurry up; whether or not they can successfully run the training using a domestically produced card cluster is also a crucial part of this plan.
After the pre-training is finished, another twenty days will be spent on fine-tuning and aligning the post-training. By then, the Glutinous Rice Ball that have been trained from scratch can be made by early March.
This is quite different from the Glutinous Rice Ball v0.1 version that was previously developed using the 7B open-source platform and eight thousand annotations.
At that time, the Large Model trained using Sight- class data cleaning standards and Sight- class Annotated Data will be unveiled for the first time.
With all the resources and advantages he had, Glutinous Rice Ball was eagerly anticipating how well he would fare.
Thinking of this, Han Luyi opened his Sight and glanced at the experience progress in the lower right corner.
Fifty percent.
It's halfway done.
Thinking this, Han Luyi picked up the latte on the table and took a sip.
Meanwhile, on the other side of Haicheng, Zhang Chi had just landed at Hongqiao.
He didn't go home; instead, he had his driver take him directly to Dingsheng Tower.
When he arrived at Dingsheng Tower, his secretary was already waiting for him downstairs.
He handed his bag to his secretary and asked as he walked towards the elevator, "Has everyone arrived?"
"Director Liu and the chief director are waiting for you in the conference room," the secretary replied quickly, following closely behind.
After exiting the elevator, Zhang Chi headed straight for the conference room.
As soon as the door opened, Liu Bo and Wang Hao were both standing. When they saw Zhang Chi come in, they both stepped forward and quickly greeted him.
Seeing their nervous expressions, Zhang Chi pulled up a chair opposite them and sat down. He then waved his hand and said, "Hey, what are you standing for? Sit down and talk."
So the two of them quickly pulled out chairs and sat down.
Once the two were seated, Zhang Chi asked, "So, what's the progress?"
--