If You Want to Ask, Go Ahead. I'm Not Asking Anyway
There's no rush to deal with this kind of alarm message. Since I've already seen it, I can deal with it when I get back.
Besides, he didn't bring his computer with him.
Let's give the graphics card clusters a few hours off on the first day of the Lunar New Year.
Han Luyi didn't put down his phone. Instead, he opened WeChat and sent Zhang Biao a message: "Biao, don't wait for me at the door. You really need to go out for a walk, otherwise my mom won't be so easy to fool."
It was two hours later when Han Luyi and Zhou Min returned home.
Han Luyi returned to his bedroom, turned on his computer, and logged into Dingsheng Cloud's account.
Looking at the recent activity logs, there have been access records over the past two hours, but training has not yet resumed.
It seems Zhao Wenyuan has also seen it and is currently inspecting it.
But it got stuck.
Han Luyi frowned. A bug that could keep Zhao Wenyuan stuck for two hours must be something special.
He glanced at Lark Chinese instant messaging platform), saw that Zhao Wenyuan was online, and then initiated a video call request.
The video call connected quickly, and Zhao Wenyuan's face appeared on the screen. He was wearing a bright red sweater, looking cheerful, but his expression was serious, contrasting with his clothes.
"So? Have you located the problem?" Han Luyi asked directly.
"Loss has been erratic since around 1 PM. Rolling back the checkpoint doesn't work, restarting doesn't work, distributed checks are fine, and the parameters are correct. I suspect it's a data issue. The newly added filtering rules were added in batches by Little Niu and his team a few days ago, and I'm currently reviewing the code against the rule documentation." Zhao Wenyuan spoke quickly, but remained calm.
At this stage, there is generally a ten-hour interval between each checkpoint. That is to say, even if it can be fixed in the end, it will have to start from the checkpoint ten hours ago again, plus the two hours of debugging, so at least twelve hours will be wasted.
"No rush," Han Luyi said. "Send me a copy of the data sample from the abnormal batch."
"Good."
Han Luyi received the document and quickly glanced at it.
Previously, Han Luyi had used the data cleaning pipeline created by Sight to clean the most obviously contaminated samples.
Later, in order to enhance the model's capabilities, Zhao Wenyuan and his data team continued to expand upon it, adding rule sets and data sources.
This is necessary. No matter how strong the original pipeline is, it cannot cover all new scenarios. As the sources of training data expand, the team will naturally engineer it.
However, the expanded rule set has a much lower coverage of the new scenarios than the pipeline that Han Luyi initially designed.
This discrepancy can cause the training accuracy to suddenly deviate significantly at a certain point in the process.
The key now is to find out where the problem lies in the extended rule set.
Han Luyi slid the sample, he mentally reviewed the pipeline structure.
On the other side of the screen, Zhao Wenyuan was still typing on the keyboard, occasionally pausing to stare at the output and frowning.
This is very difficult for ordinary people because the amount of data is too large, and the so-called good or bad, clean or messy data are all subjective judgment standards defined by people.
The only symptom Zhao Wenyuan can currently see is the sudden and drastic fluctuation in the loss value, but finding the problem in a huge set of rules that he is unfamiliar with and an even larger original corpus is like finding a needle in a haystack.
The only solution is to write scripts on the spot to aid in diagnosis, which is what Zhao Wenyuan is currently doing.
However, what is difficult for Zhao Wenyuan may not be difficult for Han Luyi.
Or rather, it's not difficult from Sight.
The Sight opens.
In an instant, the familiar colors surfaced.
Most of the samples are light blue, normal text.
A small amount of gray text represents duplicate text that wasn't removed, but it's not a major issue.
Several segments of low-quality content, highlighted in orange-red, have been flagged by the rules and should not have been included in the training main data.
But soon, Han Luyi's gaze fell on one of the strange colors.
Dark purple.
This is not a color that has appeared before.
He was slightly taken aback.
Shift your attention there.
A line of explanation pops up in the Sight.
[Suspected type: Pseudo-structured fragment]
[Mixed HTML/Markdown/JSON markup fragments and garbled characters are misclassified as code snippets by the current rule set. After Token, the proportion of repeated symbols is abnormal, which can easily lead to abnormally long sequences and cause fluctuations in the training curve.]
Han Luyi glanced at the raw data.
At first glance, these snippets do resemble code, but upon closer inspection, they are syntactically incorrect and have no practical meaning.
It's like someone deliberately poisoned it.
Han Luyi quickly updated several new rules to capture this data, and then tested them on the abnormal samples sent by Zhao Wenyuan.
Efficient.
While uploading the revised rule set, Han Luyi also shared the problems he discovered and their solutions with Zhao Wenyuan during the video call.
Zhao Wenyuan was engrossed in writing the detection script, and had only finished half of it when he heard Han Luyi's words. He looked up blankly.
What? Aren't I the one explaining the situation to you?
How come you've already solved it?
But he quickly came to terms with it.
Oh right, Boss Han my ever-present grandfather. Never mind then.
After quickly reviewing the newly added rules in the rule set, Zhao Wenyuan ran the original data again, then added the cleaned new data to the pipeline and started training again.
When he pressed start, he felt inexplicably at ease.
Meanwhile, at Dingsheng Tower.
Liu Bo knocked on the door and entered Director Wang's office.
"Mr. Wang."
"Mr. Liu."
The two are now in the same boat, and their revolutionary friendship feels even stronger than before.
Liu Bo didn't beat around the bush and said directly, "I just went down to take a look. The front-end is progressing well. Riding the Wind currently only has a terminal command-line tool, so adding the front-end gives us a competitive edge. However, on the model side, Qianyuan isn't working very well; we still need a good foundation."
Director Wang nodded without saying anything.
Liu Bo guessed he hadn't heard him at all, or hadn't understood him. "President Wang—shouldn't you report this to Boss Zhang?"
Director Wang smiled and said, "Shouldn't we trouble Boss Zhang with this? Boss Zhang made it quite clear before the holiday that we should hurry up and finish the work."
"Yes." Liu Bo smiled as well. "That's why I came to discuss this with you. To be honest, I might not be able to make it. What if Boss Zhang asks about it later?"
"Who does Boss Zhang think he can ask?" Director Wang sat up straight. "The development team is under your supervision."
Liu Bo's smile faded slightly, "President Wang, you were the one who dismantled the Riding the Wind, and your team also determined the requirements."
The office was quiet for a moment.
Director Wang took out a cigarette. "Want one?"
Liu Bo thought to himself, "Smoking in the office is quite exciting. Could this be considered a perk for working overtime?"
Pity-
"I've quit."
Director Wang put the cigarette away again. "How about this, let's observe a bit more. If there's still no progress by tomorrow afternoon, then we'll talk about it again."
"Alright." Liu Bo stood up. "Then I'll go back and keep an eye on things."
"Thank you for your hard work."
Watching Liu Bo walk away, Director Wang thought to himself: You can ask him if you want, I'm not going to.
In the end, no one called Zhang Chi.
Even if they did fight, Zhang Chi wouldn't be able to receive the attack.
His phone was locked in the locker room; he was having a spa treatment.
"Ah Yue, your parents aren't old-fashioned; they don't force you to go on blind dates."
Mrs. Gu sat at the other end of the sofa, with a plate of tangerines that had been mostly peeled on the coffee table in front of her. The tangerine peels were scattered on the table, forming a small mountain.
As she spoke, her hands didn't stop; she put the peeled oranges into the bowl, then took another orange and started peeling it.
Gu Siyue was curled up at the other end of the sofa, her legs crossed, a cushion resting on her knees, her phone screen constantly lit up. She was absentmindedly looking at the case files on her phone, not even lifting her eyelids, as if she hadn't heard anything.
"If you want to date freely, we support you." Gu's mother pushed a small bowl of peeled oranges in front of her daughter, with a plate of untouched strawberries next to it.
"The person I'm introducing you to is also the son of your dad's good friend. Mom met him when she went back to Beijing last time. He's quite promising. You can get to know him first. It's okay if you don't like him; just consider it making friends."
Upon hearing the word "Beijing," Gu Siyue paused her finger as she swiped across the screen.
She locked her phone, slung it over the cushion, and looked up.
--
"Why would I make friends in Beijing when I'm in Haicheng?"
Mrs. Gu was speechless for a moment after being choked by her daughter. She put down the orange in her hand and wiped her fingers on her apron.
"Sigh—" she drawled, "Your dad should be transferred back after the New Year. You can't stay away from Beijing forever, can you?"
Sure enough, this question concerned her more than blind dates. Gu's mother leaned forward, her speech quickening. "You like being Lawyer? You can be one in Beijing. I heard you say last time that you resigned from your previous firm? Isn't that perfect for coming back to Beijing?"
Gu Siyue didn't want to respond to that.
She reached into the bowl, took an orange, broke it in half, and put one half into her mouth.
The tangerines were very sweet, even a little too sweet. She squinted, looked away from her mother's face, and then picked up her phone to check the time.
Where's my dad?
"I just received a phone call and then went out again."
Gu Siyue's brows slowly furrowed.
She turned to look out the window; it was already completely dark.
"Working overtime again. Even during the Chinese New Year holiday, I'm still working overtime every day. I wish I hadn't flown here."
"How can you talk like that, child?"
Gu's mother's tone was tinged with reproach, but even more so with helplessness.
After saying that, she sighed again.
"Your dad's rarely home, can't you come and keep your mom company during the Lunar New Year?"
Gu Siyue listened without saying anything, but got off the sofa, walked to her mother's side, and hugged her.
Mother Gu didn't mention moving again, nor did she mention blind dates. The mother and daughter chatted casually.
While her mother went to the kitchen to tidy up, Gu Siyue picked up her phone and opened WeChat.
The first person on the chat list is Han Luyi.
Gu Siyue looked at her conversation with Han Luyi from yesterday.
The last message was the last two words she posted.
Kunming.
She scrolled up again and looked at the previous chat history.
Then she reached out and typed a few words.
I thought about it, but didn't send it, and then deleted it.