Han Luyi received a message from Ma Xiaofei early in the morning.
Ma Xiaofei: Godfather! Sugar Daddy! Save me!
Han Luyi a question mark.
Ma Xiaofei sent the video directly.
When the video call connected, his long face covered in acne appeared on the screen, his expression full of bitterness.
"I really can't write anymore!" Ma Xiaofei said with a mournful face.
"If you can't write it, just abandon it," Han Luyi suggested, offering him a clear path. "When did you start writing novels?"
"What novel? I mean, I can't finish the script. Didn't you see I haven't updated my videos for three days?" Ma Xiaofei was completely confused by Han Luyi's explanation.
Han Luyi suddenly slapped his forehead.
He's been busy with Computing Power lately and has completely forgotten about Ma Xiaofei's small business worth a few million.
There's simply no time left to watch videos.
"If you're lacking inspiration, why don't you ask Glutinous Rice Ball? Asking me won't help," Han Luyi said.
To everyone's surprise, Ma Xiaofei said with a serious expression, "Old Han, it's not that I lack inspiration, it's that I lack knowledge!"
Han Luyi looked at Ma Xiaofei, who had a proud expression on his face, and wondered what he was so proud of.
Ma Xiaofei finally explained his purpose: he was there to ask for resources. "The 'Xiao Fei Talks AI' series you did last time has the highest viewership of all my series, and it's also the one with the most requests for updates. But apart from you, I really don't have any connections in the A-list industry. Could you come and record another episode for me?"
Han Luyi looked helpless. "Brother Ma, Pony Ma, you can't just keep taking advantage of me."
"Or you could introduce me to your company's computer scientist. Surely you're not the only one in your company that understands computer science, are you?"
Ma Xiaofei widened his eyes as much as possible, putting on a pitiful look. "Please, Godfather! You were the one who suggested this series in the first place. Didn't you promise to provide me with connections?"
Han Luyi, "Did I agree to that back then?"
They might have actually promised.
Then he thought of the people around him. Zhao Wenyuan was a good candidate, but he was in the Beijing.
All the researchers at the Research Institute are in Beijing right now.
Should we have Ma Xiaofei conduct a live interview?
--It's not that it's impossible, but Han Luyi and remembered someone else.
"Let me ask for you," Han Luyi said. "They're definitely working on weekdays, how about the weekend? I'll see if I can get them to come out for you."
"Sure!" Ma Xiaofei said generously, "I'll give him a break right now and let him set aside his weekend."
Han Luyi sensed from the words that Ma Xiaofei's new media company seemed to be poorly managed.
However, he silently repeated to himself: I am the best investor in the world; I only provide the money and do not interfere with management...
I forcefully suppressed my thoughts of criticizing Ma Xiaofei's management style.
Perhaps this is just the style of new media.
After hanging up Ma Xiaofei's call, Han Luyi immediately dialed another number.
The call was answered very quickly.
"Boss Han, what brings you here?" a warm voice rang out.
Han Luyi felt a surge of joy upon hearing the voice and couldn't help but laugh.
"Brother Dahai, I have a favor to ask. A friend of mine works in social media and is currently doing an interview program. They'd like to interview some experts in field A. I wonder if you'd have time to chat?"
"Is it that 'Xiao Fei Talks AI'?" Liu Dahai asked.
Now it was Han Luyi be surprised.
How come you know about this too?
Liu Dahai Harbin University: "Boss Han, every single person in China working on Artificial Intelligence has seen your interview! I'm so honored to be the second guest on the show, right? Being listed after you makes me the second person in China to be on the AI team, doesn't it? Here's what we'll do: send me Ma Xiaofei's contact information, and I'll discuss the time with him. Is that alright?"
Then he added, "However, regarding the interview, I need to report to Boss Zheng that matters concerning the company cannot be discussed."
Han Luyi readily agreed.
He came from Dingsheng himself, how could he not know this rule?
The deal was settled, but Han Luyi's first reaction after hanging up the phone was:
Oh no, will Old Zhao get jealous?
Remember the updated address so you don't get lost:???.,.,.,.,.,.,.,.,.,.,??.?,????.ɡ,????.??,????.?ɡ,???.??ɡ,????ɡ,????.?,???.??ɡ or:
On Saturday, Liu Dahai followed the navigation and drove to the address given by Ma Xiaofei.
The road became increasingly desolate.
The building he finally stopped at was so far off-center that it looked like he had gone back to his hometown in Northeast China.
Liu Dahai had no idea that there was such a desolate place in Haicheng.
The address seemed correct, so he even called Ma Xiaofei to confirm.
A short while later, Ma Xiaofei came down to pick him up with photographer Liu Yibo.
Thus, Ma Xiaofei's studio finally welcomed its first guest since Han Luyi.
After a brief self-introduction, Liu Dahai greeted Liu Yibo warmly, "Hello, hello, we're even from the same clan."
Liu Yibo seemed a little hesitant to shake hands.
One is a small-time photographer in an independent studio, and the other is the Chief Scientist at Dingsheng; the gap between them is too large.
In other words, Ma Xiaofei's carefree and reckless personality means he doesn't care about this.
Upon entering the studio's shooting area, Ma Xiaofei and Liu Yibo set up the lights together, then asked Liu Dahai, "Teacher Dahai, can we begin?"
Liu Dahai was taken aback. "Mr. Ma, don't we need to put on makeup?"
This isn't Liu Dahai's first time facing the camera; he's been to the recording studio before. Although the group consisted mostly of men, there was still a makeup and styling process involved, right?
Ma Xiaofei smiled and said, "Teacher Dahai, this is an educational program, not a show with good looks, so we won't do those things. We'll rely entirely on Teacher Dahai's knowledge to attract viewers."
"Um, okay."
Liu Dahai thought to himself, "I should have dressed a little more formally. I didn't expect there wouldn't be a makeup and styling step."
The main reason Liu Dahai was impressed was when he saw Han Luyi's interview video last time, where Han Luyi looked radiant and spoke eloquently.
Now that I think about it, they probably didn't rely on makeup.
Filming began quickly.
Ma Xiaofei addressed the camera in the center, saying, "Hello everyone, welcome to the second episode of 'Xiao Fei Talks AI.' In this episode, we've invited the renowned Artificial Intelligence expert, Dr. Liu Dahai. Dr. Liu Dahai previously worked at well-known overseas companies such as pnA and is currently the Chief Scientist at Dingsheng Artificial Intelligence Research Institute— "
Liu Dahai sat still and nodded at the camera, revealing his smooth scalp.
"We invited Professor Dahai today because I wanted to take this opportunity to ask him some questions I've never understood before," Ma Xiaofei said. "Because I also have a computer science background, when we were learning, the computer would output B if you input A. Although the logic might be complex, the principle that the output should be the same if the input is the same would never change."
Ma Xiaofei pointed to the air.
"But why did this pattern change in era A? Why do I get different answers every time I ask the same question?" Ma Xiaofei asked.
Liu Dahai looked at Ma Xiaofei's hands. "Mr. Ma, what are you doing?"
Liu Yibo covered his eyes.
Thankfully, it wasn't a live stream.
"Teacher Dahai, you can just call me Xiao Fei." Ma Xiaofei looked at his hands. "Aren't we adding special effects in post-production? I'm leaving room for that."
Liu Dahai suddenly realized. "Sorry, I don't have experience this first time. Shall we record it again?"
"No need, Mr. Dahai, this can be edited out in post-production. You can just answer my question directly."
Liu Dahai nodded and began to explain, "Xiao Fei, that's an excellent question. The essence of Large Language Model is actually a probability generator. It's essentially only responsible for one thing: predicting the next Token."
"Let's do a simulation now," Liu Dahai said. "I'll say a word, and you repeat it. You don't need to worry about what you mean, just make sure the last sentence makes sense."
"Today," Liu Dahai said, then paused.
"The weather," Ma Xiaofei continued.
"Real."
"Is Tian Tian being really nice today?" Ma Xiaofei thought for a moment and said, "Great!"
The sentence can be summed up as "The weather is really nice today."
Liu Dahai nodded. "You see, we just completed a Large Model reasoning exercise. After I said 'today,' you could actually say anything. You could say 'today's weather,' or you could say 'today and tomorrow.' Then, after I said 'true,' your choices all changed."
Liu Dahai stretched out his hand, pointing with each finger. "It can be good, it can be hot, or it can be cold."
"From the limited context we just discussed, relying solely on the phrase 'the weather is really nice today,' we can estimate that the probability of the next word being 'good' is about 40%, 'hot' is 30%, 'cold' is 20%, and other miscellaneous words add up to about 10%."
"If that's the case," Ma Xiaofei asked doubtfully, "Then why not just choose the one with the highest probability?"
"If every time I type 'What a great weather today' and a 'good' appears, is it still an A?" Liu Dahai asked. "Or rather, is it even useful?"
Ma Xiaofei frowned.
It's pointless to listen to that.
"In the example just now, the model's distribution ratio of 40, 30, 20 is fine. If you try a few more times, several options may appear. But what if the statistical distribution is 80, 10, 8? Wouldn't only the one with the highest probability appear?" Ma Xiaofei asked.
Liu Dahai looked at Ma Xiaofei in surprise. "Xiao Fei, you're very talented! We have a tool specifically designed to solve this problem in machine learning!"
He mimicked Ma Xiaofei's earlier gesture, reaching his hand into the air.
"In all model settings, there will be a parameter called Tmpatu, which is the Chinese word for temperature."
Then he said to Ma Xiaofei, "This part will have special effects added in post-production."
Ma Xiaofei smiled.
No wonder you have a PhD, you learn really fast.
"So how do we solve this temperature problem?" Ma Xiaofei asked.
"Simply put, this is an operation that can flatten the probability distribution," Liu Dahai said. "Suppose your data probability distribution is very extreme, with the top four options corresponding to probabilities of 90, 3, 3, and 3 respectively. Then no matter how you choose, you might always end up with the one with a 90. But by increasing the temperature, we can change the overall probability to 30, 22, 22, and 22. After the adjustment, the order of the options remains the same, but the distribution is more even, and your probability of choosing the smaller option also increases."
"The fundamental reason we do this is because we know that a statistically high probability of a result is not necessarily the best. Therefore, we manipulate the probability to introduce more variables into the outcome."
Ma Xiaofei, with a "I understand" expression, said, "Teacher Dahai, I've seen this temperature option on various interfaces before, but I never understood what it did. Now that you've explained it, I understand. So, the lower the temperature, the calmer the model, and the more consistent the output; the higher the temperature, the more energetic the model, and the more random the output."
Was my conclusion correct?
Liu Dahai hesitated.
The summary is correct, but is the use of the word "passion" appropriate?
"Besides temperature, what other commonly used parameters are there in the model?"
"There are still two parameters that frequently appear related to temperature: TpK and TpP," Liu Dahai said. "Let's talk about TpK first."
"You know, current Large Models are huge, with trillions of parameters. A single input can have tens of thousands of possible outputs. If we were to calculate them all, it would be slow and wasteful of resources. To avoid this waste, we choose to put only the top K into the candidate pool and leave the rest out."
Ma Xiaofei nodded.
This is simple and clear.
What about Tp P?
Liu Dahai, perhaps because he hadn't given a lecture in a long time, became increasingly enthusiastic as he spoke.
"The P here stands for probability, Pbablty. Let's first talk about what problem it solves. For example, if I input the sentence 'What is the capital of China?', what do you think the answer would be?"
"It's the Beijing," Ma Xiaofei replied instantly.
"Yes, this is a factual answer, and it cannot be wrong. However, there are other options in the model that have a very small probability, such as 'Jinling' and 'Chang'an,' which are also predicted. According to what we just said, these options may also be output."
Liu Dahai said, "But we don't want it to be output; we only want the most certain answers. So we can set P=0.95. That way, as long as the probability of the options we collect exceeds 0.95, we won't continue collecting. In the previous answers, the original probability of 'Beijing' might be as high as 0.99, so naturally, no other options will appear in the answer set."
This is a bit of a brain teaser for Ma Xiaofei.
However, he still tried to summarize it.
It's not a bad thing to not know; if you don't understand, you should ask.
After all, only if he can understand it can the audience understand it.
"So, TpK is about the number of options, and TpP is about the total probability. When the number of options selected by these two numbers is different, which one should we listen to?" Ma Xiaofei asked.
"We'll listen to whichever is smaller," Liu Dahai replied. "We do all this to prune the answer; anything that doesn't meet any of the criteria is cut off."
Hearing this, Ma Xiaofei breathed a sigh of relief.
What a thoroughly enjoyable teaching session!
He glanced at the questions he had prepared and continued, "Teacher Dahai, thank you very much for your insightful and clear explanation. I have one last question."
"Go ahead and ask," Liu Dahai said with a smile.
"As we just discussed, the reason a model's output is different each time is because the settings of these parameters introduce a certain degree of randomness into each inference." Ma Xiaofei glanced down, then looked up again. "Is there a possibility that a model might suddenly become less intelligent?"
Liu Dahai's smile froze on his face.
"Mr. Ma, you shouldn't ask that question."