Although Han Luyi just mentioned Su Niannian, he had actually arranged a video conference for himself as well.
And Zhao Wenyuan.
Zhao Wenyuan hasn't come to the office since he took leave. He's been working at home setting up data pipelines and preparing for the completion of the program at Dingsheng.
The two went over the training strategy first. At the end of the meeting, Han Luyi asked about the timeline.
"Wenyuan, the account with Dingsheng should be ready in the next couple of days. When can you start running yours?"
On the other end of the phone, Zhao Wenyuan paused for a moment, seemingly organizing his thoughts, before saying, "Boss Han, even if the Computing Power is in place, there's still a hurdle before we can get started."
What hurdle?
"What I told you about last Friday was the pre-cleaning of the pre-training data."
Han Luyi sat at his desk and thought about it. Yes, that was the case.
"What do we have now? Where are the obstacles?" Han Luyi asked.
"We have data from CommonCrawl and publicly available books and materials. I want to optimize the cleaning process, but I've been working on it for several days and still have no clue what to do," Zhao Wenyuan explained.
"Hmm... I'll go back and check on it tonight."
After hanging up the video call, Han Luyi began to ponder in his office.
Han Luyi remembered Zhao Wenyuan's attribute panel.
[Zhao Wenyuan]
[Technology S (Research SS) "Collaboration C" Stability A]
[Special Skill: Large Model Training "Large Model Post-Training"]
Post-training is his forte, but he might not be as familiar with pre-training.
Of course, Large Model have been around for a few years now, and so many papers have been published on them. Even if you haven't tried them, you've seen them. Skills can be transferred; with a little trial and error, pre-training isn't that difficult.
The real bottleneck to Large Model from scratch lies in data and Computing Power.
The data is also divided into two parts: raw data and Annotated Data.
In terms of Annotated Data, Han Luyi has already obtained world-class labeling through Sight.
The quantity isn't quite enough yet, but with some effort, there will eventually be some.
But we still need to figure out how to get the raw data.
Manufacturers typically source data from several sources. Besides publications or industry-specific information, they also collect data from the internet. Internet data comes from various sources: you can crawl the internet yourself—this is where your skills come in, and what you find depends on your abilities; or you can use CommonCrawl.
CommonCrawl is an open-source corpus, a "digital public library" for the era of Large Model.
It's a non-profit organization whose goal is to "crawl and archive the entire internet" and make that data freely available to everyone. This vision is great, and it has indeed helped many people.
Currently, almost every Large Model on the market that performs general-purpose scenarios includes CommonCrawl in its training corpus; the difference lies only in the amount of CommonCrawl present.
However, the problem with CommonCrawl is that it's basically unfiltered; it contains all sorts of raw information from the internet. Take the Chinese internet, for example: what's the most prevalent content there? Practical knowledge? Life tips? Life insights? Touching literary works? No, it's garbage, all sorts of garbage.
Spinach GG, Sorry GG, are using repetitive keyword stuffing to optimize search results.
If you were to use these materials directly as raw materials to create Large Model, you could expect to see something like this.
You ask it, "What is Large Model pre-training?"
It replied, "Recently, everyone has been very interested in Large Model pre-training. So what exactly is Large Model pre-training? Let's take a look, and you'll be amazed! How should we evaluate this? Feel free to leave your thoughts in the comments section!"
Therefore, before the raw materials are used, they must undergo a complex cleaning process, which is not done manually. Generally, a traditional rule-based data pipeline is used to filter out completely useless or harmful information, and then semantically repetitive content is removed from the remaining information.
Han Luyi had no idea how to do it yet, but that didn't matter.
I'll ask Brother Sight what he thinks tonight.
Thinking of this, he took out his phone and checked the time; it was almost six o'clock.
That evening, we had dinner together in Dorm 404, organized by Zhang Haoran.
I had planned to meet up this weekend, but Han Luyi postponed it until today because he was preparing for today's board meeting with Su Niannian.
Zhang Biao went to the airport and hasn't returned yet. Han Luyi didn't take the subway; instead, she splurged and called a car herself.
Zhang Haoran booked a grilled fish restaurant, which was much more upscale than their usual barbecue stall. A solid wood sign hung at the entrance of the private room, reading "VIP Seating".
When Han Luyi arrived, Zhang Haoran was already sitting inside.
As the two were greeting each other, Ma Xiaofei walked in, holding his DJI OsmoPocket 4P. He said as he entered, "This has to be posted! From the barbecue stall to the VIP seats, this is a documentary of our dorm's growth."
"Can't you come in and sit down first?" Zhang Haoran said.
"Artists need to record things constantly; you don't understand."
"When did creating self-media content become an art form?" Zhang Haoran complained.
"It's probably abstract art," Han Luyi added.
"Laugh all you want, I just surpassed 200,000 followers last month," Ma Xiaofei said defiantly.
"Two hundred thousand?" Zhang Haoran was a little surprised. "Wasn't it just over one hundred thousand last time?"
"That was months ago. Recently, the tech sector has been getting a lot of attention, so I made a series called 'Xiaofei Teardown,' which specializes in disassembling various domestic AI Hardware, such as translation earphones, AI speakers, and smart glasses, to show everyone what's inside. Last week's video disassembling a certain AI speaker got over 600,000 views."
"Are you able to support yourself now?" Han Luyi asked.
"What are you talking about? My buddy just got some good news. GG (male escort) rates are 5,000 per person now, standard rates. He can get four or five a month, and we split the profits." Ma Xiaofei paused, unusually serious for a second. "Of course, it's not stable. A good month doesn't guarantee a good month next, but at least right now is my most comfortable situation."
Zhang Haoran glanced at Ma Xiaofei: "You're a graduate of Jiaotong University, working full-time as a self-media creator, don't your parents have any objections?"
"I did have some complaints, but they all stopped when they saw I was making money," Ma Xiaofei said matter-of-factly. "All parents are the same."
The three sat down and ordered two fish: a Qingjiang fish with a soy sauce flavor and a black carp with extra spice. This was mainly because Han Luyi didn't like spicy food, Ma Xiaofei said that coming to eat grilled fish without eating spicy food would be a waste. They also ordered side dishes such as vermicelli, bamboo shoots, and bean products. Finally, they ordered a large bucket of Coke. Zhang Haoran didn't mind; he had already decided to treat them later.
Before the food was served, Ma Xiaofei fixed the Pocket 4 to the table, found a spot where he could see three people at the same time, and even set up face tracking for his own face.
Then, Ma Xiaofei turned around and asked, "Zhang Haoran, what's going on? Why did you call us over in such a hurry? Is there anything your brother Ma can help with? Is Old Han's savings a bit low?"
Han Luyi also looked at Zhang Haoran with curiosity.
This was indeed the main topic Zhang Haoran wanted to discuss today. He ignored Ma Xiaofei's joke, his expression serious, and his voice low. "Didn't Lao Han give us a beta test account last time? Did you use it when you got back?"
Ma Xiaofei felt a little uncomfortable with his expression: "I opened it and took a look, isn't it the same as ChatGPT, Doubao, or something like that?" "No," Zhang Haoran said, "Old Han's model really can read minds."
Now, not only Ma Xiaofei but even Han Luyi was completely baffled. The mind-reading thing was just something Glutinous Rice Ball was really good at understanding intentions; he was just joking. But looking at Zhang Haoran's expression, it seemed like he actually believed it?
"Let me tell you the details. Our bank recently got a new president, and the head office is pushing for the migration of the domestic system. The assistant president and the vice president who directly supervises me have been at odds for two or three weeks. I've racked my brains over this, afraid of choosing the wrong side. But guess what? I told Glutinous Rice Ball about it, and she figured it all out."
Zhang Haoran lowered his voice even further, leaned forward slightly, and gestured for the other two to lean in as well. "I thought what she said made sense, so during the meeting, I jumped ship and supported the assistant president. And guess what? The assistant president really did represent the president's opinion. Now I'm directly on the committee, and besides me, the committee consists of a manager from head office and another vice president from our bank."
Upon hearing this, Ma Xiaofei exclaimed, "Holy crap, I'm getting goosebumps!"
He glanced at the Pocket 4P on the table, reached for it, took it back, and turned it off.
"Don't worry, I definitely won't broadcast that part," Ma Xiaofei assured him.
At this moment, Zhang Haoran raised his Coke and said to Han Luyi, "Old Han, I just wanted to thank you today. Your AI is really amazing; it saved me three to five years of hard work in the workplace."
Han Luyi, however, was not happy at all.
He first clinked glasses with Zhang Haoran, took a sip of cola, and then said, "I really didn't expect to be able to use it for this."
Han Luyi really hadn't expected this. He was somewhat limited by the capabilities of the Large Model currently available on the market. The application scenarios he envisioned were all about enabling the model to better understand the user's real needs when there was insufficient context, so as to help them solve problems.
Zhang Haoran's use of it was both unexpected and seemingly reasonable. After all, its Annotated Data came from a cheat code, making it a simplified version of mind-reading. Moreover, the Large Model could go a step further by providing action suggestions based on the analysis results. But this was precisely its most dangerous aspect.
Can other Large Model do what Zhang Haoran just mentioned?
Yes, it can be done.
But will people listen?
It's highly unlikely.
What could be the reason?
Because their analysis wasn't good enough.
However, once Glutinous Rice Ball's ability to understand intentions became exceptionally high, it could analyze others' intentions with remarkable clarity. As a result, when it offered action suggestions, people found it hard not to believe them.
But even the original cheat codes couldn't predict the future.
Moreover, it's a 7B model, plus a weakened version of the cheat codes?
If you're right, that's fine, but what if you're wrong?
In fact, Large Model have already performed very well in many scenarios. For example, in Medical diagnosis and legal analysis, they can provide very reasonable suggestions.
But why is it that applications are most slowly adopted in these specific scenarios?
These scenarios are all typical high-cost scenarios.
You can achieve a 99% accuracy rate, but a 1% error can have serious consequences.
In the past, professionals took responsibility for these consequences, but if AI is used, who will bear the responsibility?
Thinking of this, Han Luyi's expression turned serious, and he said to Zhang Haoran, "Don't use it like that again."
Zhang Haoran was taken aback. "What do you mean?"
Han Luyi organized his thoughts and said, "Didn't you realize? It didn't just analyze; it also gave you specific suggestions, and then you followed them."
Then he emphasized his words, emphasizing each one:
"You've relinquished your right to judge."
Zhang Haoran immediately understood Han Luyi's meaning and felt a chill run down his spine.
Yes, he chose the right side this time, but does he really know why?
What about next time?
He recalled that recently he had been discussing more and more workplace issues with Glutinous Rice Ball that he didn't understand, and what she said made sense, which he had generally adopted.
Han Luyi continued, "Haoran, once you become increasingly dependent on it, your brain will degenerate."
"By then, you'll be useless!"
Upon hearing this, Ma Xiaofei, who was standing next to him, couldn't help but swallow hard.
Zhang Haoran said, "Then... what should I do?"
Han Luyi and said, "Let's use it differently."
What is it used for?
"You can treat it as a friend, a teacher, but not a prophet. How did we study in school? When we encountered something we didn't understand, we would first try to figure it out ourselves. If we really couldn't figure it out, then we would ask the teacher, bringing with us our thought process and where we were stuck. Or when we talked to friends, we would first express our own thoughts before asking for their opinions. Don't give up on thinking for yourself; think first, and then communicate with the results of your thinking. That's the correct way to use it."
Hearing this, Ma Xiaofei couldn't help but say, "Old Han, that's your study method, isn't it? I just ask the teacher directly when I don't understand something."
Han Luyi strange expression. "Are the requirements for people from Haicheng to Jiaotong University into Jiaotong University really that low?"
Are you trying to make fun of me?
With Ma Xiaofei interrupting, the atmosphere in the private room was no longer as oppressive as before, and just then the dishes were served.
Han Luyi wasn't sure if Zhang Haoran had taken it to heart, but the three of them tacitly changed the subject and started talking about other things.
Although Zhang Haoran was not promoted, he gained the appreciation of the top leader and was given a job with real power, which is something to celebrate.
"Speaking of which, how's your attempt to quit smoking going?" Ma Xiaofei asked.
"It's alright, I didn't break my rule. It's just that I'm working overtime every night, which is quite tiring. Look, I didn't even order any alcohol today," Zhang Haoran said with a smile.
"No? Who asked you?" Ma Xiaofei raised his voice, "Don't bully us single dogs!"
At this point, Ma Xiaofei turned to Han Luyi and asked, "Uh, Lao Han, you're still on my side, right?"
Han Luyi nodded.
Ma Xiaofei seemed relieved, then asked, "So what's going on between you and Goddess Su? Haven't you officially started dating yet?"
"Can we not talk about this?" Han Luyi looked at the grilled fish.
"Alright..." Ma Xiaofei sighed, "I saw Goddess Su at the hospital on Saturday and even said hello."
"What? What's she doing at the hospital?" Han Luyi turned to Ma Xiaofei and asked anxiously.
"No, why don't you ask me what I went to the hospital for?" Ma Xiaofei chuckled, "You're prioritizing your girlfriend over your friends, aren't you?"
Then, glancing at Han Luyi's expression, he quickly said, "It seems like a family member of hers is in the hospital. I didn't ask in detail, but she doesn't look well; she's probably just too tired."
Han Luyi recalled Su Niannian's figure this morning. Before the meeting, she could see that Su Niannian was a little tired, but as soon as the meeting started, she went all out.
He probably just had to force himself to stay awake.
Han Luyi picked up his phone from the table, opened the chat window with Su Niannian, typed a few words, and then deleted it.
What should he ask? "Are your family members alright?" He doesn't even know who's in the hospital. Asking "Are you alright?" I just saw her this morning; her demeanor at the meeting showed she didn't want others to know.
In the end, he only sent one message: "Nian Nian, you've worked hard today, get some rest."
After sending the message, he looked at it and felt that he hadn't said enough, but he couldn't think of anything more suitable.
Su Niannian did not reply immediately.
Han Luyi his phone back on the table and continued eating his fish. A few minutes later, his phone lit up, and he glanced down at it.
"OK!"
You too!
Han Luyi stared at those six words for a while, then locked the screen and didn't reply.
After finishing his meal, Zhang Haoran paid the bill ahead of time. Before leaving, he said to Han Luyi, "Old Han, you should probably close my beta test account. I'm afraid I won't be able to resist."
Han Luyi and said, "Okay, I'll shut down the Server and adjust it again when I get back. It really can't be used by ordinary users like this." Ma Xiaofei quickly said, "Hey, just shut down his server. I haven't used it yet."
Seeing the unfriendly expressions on their faces, he quickly added, "At least shut it down tomorrow, let me try it tonight first." Back home, Han Luyi turned on his computer, pulled out some raw data from the files Zhao Wenyuan had shared, and then ran the data cleaning script that Zhao Wenyuan had placed in the same folder.
As he ran, Han Luyi expanded his Sight.
Brother Sight, let's see your skills.