Zhao Wenyuan coughed lightly, drawing everyone's attention, and then spoke:
"Since everyone agrees to try following Boss Han's approach, let me first explain in detail our current achievements and how the two sides can be integrated."
After hearing Zhao Wenyuan's words, the people at Mengnao Technology took them seriously.
Glutinous Rice Ball and Origin are household names in the field of Artificial Intelligence, but as the saying goes, "every profession has its own secrets," and it's hard to say how much of their technology can be applied to the field of Brain-Computer Interface.
Now that the other party's Chief Scientist has spoken, Zhan Xiang feels extremely conflicted. He hopes that Dr. Zhao can bring a new solution from a different perspective, but he is also unwilling to let someone else easily solve the problem that has been stuck for several years.
Zhan Xiang to himself that it would be best if they could provide some ideas, but he could complete the whole task himself.
Zhao Wenyuan stood up, stood in front of the whiteboard in the conference room, picked up a marker, and started drawing on it.
He didn't use any pre-prepared slides, nor did he need to. The training strategies for all the models under Origin were in his mind, and he could draw architecture diagrams with ease.
Before long, the training framework for Glutinous Rice Ball was drawn on the whiteboard.
"President Meng, President Zhan, you see, this is how Glutinous Rice Ball 1.0 was trained. From building a base model from the original corpus to adjusting the weights with high-precision annotation, it developed its intent understanding and problem-solving capabilities. Following the idea that Boss Han just proposed, we will first use the EEG data that Mengnao has accumulated to train a base model. This model does not need to be able to accurately understand the meaning represented by every EEG wave. It is more like a pre-trained model that can distinguish which patterns are recurring general signals and which are random noises from the 30,000 hours of original data you provided."
Zhan Xiang to follow Zhao Wenyuan's line of thought, but after listening to him, he shook his head. "President Zhao, I'm afraid that won't work. Although the amount of data is large, it's too messy. It's not feasible to directly summarize patterns from it." Zhao Wenyuan glanced at him, thinking to himself, "Who understands machine learning better, you or me?"
Self-supervised pre-training does not require uniform labels; that's something needed in the post-training stage. Self-supervision itself can learn general structures from messy data.
However, he didn't voice his doubts. The new method was too different from Mengnao's current method, so it was normal for their team to have some doubts. But the old method was clearly useless, otherwise Mengnao Technology wouldn't be stuck here.
This reminded Zhao Wenyuan of the history of the field of translation.
When machine translation was first started, people began with the methods of translators and linguists, trying to teach computers to translate using the methods humans use. Grammar, syntax, and clause structure were all written into the rules.
How effective was it?
It's a long story.
Later, people took a different approach, using large-scale data to statistically analyze probabilities, i. e., machine learning, for training, which significantly improved the results.
Frederick Jelinek, a leading figure in speech recognition, famously said, "Every time I fire a linguist from my team, the performance of speech recognition improves dramatically."
Incidentally, the Transfrmer architecture was originally invented for translation, and its effects were indeed amazing. It was later proven that this architecture could be used for more than just translation.
Zhao Wenyuan didn't intend to persuade Zhan Xiang face-to-face, as verbal exchanges were pointless. He simply said, "President Zhan, why don't we start the first round of training as soon as possible? With the results of the first round, we can know whether this direction is feasible."
He believed that Zhan Xiang would be convinced once the results were in. Zhao Wenyuan then asked, "What is the sampling rate of these EEG data? How large is the corresponding data volume?"
Zhan Xiang aback, not expecting Zhao Wenyuan to avoid answering his question. Zhao Wenyuan was clearly very confident in the training results, and Zhan did not answer Zhao Wenyuan's question for a moment.
It was Meng Yao who spoke first, "It's 256 Hz, the standard sampling rate. Our microelectrode array will aggregate the acquired signals, with a maximum of 1024 acquisition channels, using 24-bit precision."
She realized that Zhao Wenyuan was asking about the size of the data, so she provided all the parameters of the original data at once.
Zhao Wenyuan pondered for a moment, then quickly made an estimate in his mind.
30,000 hours, calculated at 256 Hz:
30000 x 3600 x 256 = 27648000000
A single channel has approximately 28 billion sampling points.
With 1024 channels, the total data volume is 28 trillion potential signals, with 24-bit precision, which is approximately 84 TB of data.
As EEG data, this amount is huge; but as data for training a model, this amount is negligible.
Having worked with training systems for a long time, one develops an almost instinctive sensitivity to the magnitude of data. The estimation of these data points took Zhao Wenyuan only a dozen seconds to process.
Then he said, "We don't need to start with too much data. We can start with a model with 500 million parameters." Zhan Xiang subconsciously repeated, "500 million?"
The model their team manually extracted features from still has tens of millions of parameters.
When Zhao Wenyuan heard him speak, he assumed that the child felt it was too small.
After all, with 500 million parameters, it only has 0.5 bytes. Any open-source model on the market is larger than that. Glutinous Rice Ball 1.0 is Big Mac with trillions of parameters, and this is still far from being the largest model on the market.
This EEG model, which is expected to undergo its first iteration, is practically adorable compared to the Large Language Model.
Zhao Wenyuan offered words of comfort, "Let's start small, and if it works well, we can upgrade."
Zhan Xiang choke for a moment, and couldn't say anything more.
Five hundred million—is that small? Then what is our current model? Very small?
Meng Yao wasn't bothered by the number; her field is neuroscience, and algorithms aren't her specialty.
So from the beginning, she has been focusing more on the second point Han Luyi mentioned earlier: achieving rapid translation from intention to action through the new model architecture of Origin.
Two hundred milliseconds.
She wasn't sure Han Luyi just made up the number, but it happened to be the average time it took for a human to go from having a thought in their brain to a muscle response and finally to making a movement. In other words, if Brain-Computer Interface could really achieve this level of signal transmission delay, it would allow people with disabilities to use external artificial devices just like they would use real limbs.
That's what truly solves the difficult problem of Assisting People with Disabilities.
Furthermore, if this level of performance can be achieved, coupled with the inherent harmlessness of non-invasive brain-computer interfaces, the application scenarios for civilian and entertainment uses will be greatly expanded.
At least players can use it to play games now.
Mengnao Technology's current average latency is between 800 milliseconds and one second, which is definitely unacceptable.
The problem is, this is already the level they could achieve by doing everything they could.
To achieve 200 milliseconds, the speed needs to be increased by four to five times.
To be honest, Neralink's ability to collect electrical signals directly from areas of high brain activity using an Invasive solution is already quite remarkable.
Given Mengnao Technology's current technological approach, how can it achieve this?
Meng Yao couldn't help but ask, "President Zhao, let's assume the first step of the general model is successful—"
"But what's the next step? How can we achieve two hundred milliseconds?"