The Journey of Reinventing the Wheel from Scratch

Standing under the dormitory building with bags in both hands, Natsukawa Sosuke let out a long sigh of relief. It was already 3:00 PM, and after following the map on his phone the entire way, he had finally arrived. Carrying his computer had nearly exhausted him. Looking at the modern-looking dormitory building before him, he felt it was far more luxurious than his university dorm in his previous life. This was the "newbie village" where he would spend the next three years.

He walked toward the dormitory lobby on the first floor, heading to the manager's window alongside Ayanokoji Kiyotaka. After the dorm manager verified his student information, she handed a room card to Natsukawa: "Room 403. The rulebook is inside the room; read it carefully."

"403, thank goodness it's not 404." Natsukawa took the card, his mouth twitching slightly. "If it were 404, you might never be able to find me one day, Ayanokoji. What's your room number?"

"401."

"Looks like we're neighbors then. We should visit each other often." Natsukawa patted Ayanokoji on the shoulder. Only after lifting his arm did he realize his hands were practically dead from carrying his bags all the way here. If he had kept holding them, he might have managed, but once he put them down, picking them back up wasn't going to be easy. He looked at the heavy computer and miscellaneous items on the floor, then glanced at Ayanokoji's empty hand. His eyes rolled, and a plan formed.

"Ayanokoji, my dear friend!" he said with a sincere tone. "As the saying goes, 'hesitation leads to defeat, decisiveness leads to... freebies'... wait, no, 'decisiveness leads to friendship'! Look at this body of mine that would topple over in a light breeze; could you help a brother out and carry these up to the fourth floor?"

Ayanokoji looked at him silently for two seconds, seemingly processing these words: "Are we already good friends?"

"Of course!" Natsukawa began to spout nonsense with a straight face. "Good friends are all about mutual support. You help me out today, I push you along tomorrow—that's just the daily routine for good brothers."

Perhaps a keyword had been triggered, or perhaps he simply felt this small favor was no trouble at all, but Ayanokoji silently took the heaviest supermarket shopping bag. "...Let's go."

The two took the elevator to the fourth floor. Swiping the card to push open the door to 403, a standard single-apartment layout came into view, complete with a kitchen and bathroom. After placing his computer on the bedroom desk, Natsukawa arranged his toiletries in the bathroom.

"Thanks, Ayanokoji, catch!" Natsukawa tossed him a drink he'd grabbed at the supermarket.

"No need to thank me. I'm heading back to my room." Ayanokoji caught it with one hand, glanced at the drink, and opened the door to leave without a word.

"Alright, you may withdraw; I am weary as well," Natsukawa joked as he closed the door, finally leaving him alone in the room.

The world quieted down. Natsukawa walked to the desk, unpacked his laptop, and plugged it into the power supply.

"Alright, time to test the quality of this 'newbie starter pack'." He rubbed his hands together, his eyes flashing with the unique gleam of a tech geek. "Although I specialized in embedded systems in my past life and wasn't very familiar with big data crawlers, the principles are the same! Isn't the whole point of choosing this to be able to build my own wheels in an environment where 'nothing is available and everything has to be hand-crafted'?"

Windows, start!

Natsukawa's fingers flew across the keyboard. The laptop came with an original version of Windows, saving him from having to find a bootable USB drive somewhere. He first installed the classic Notepad++ and FileZilla.

No VS Code? No big deal, Notepad++ will fight for the future!

Next was the main event. He skillfully downloaded the Python 2.7 installer.

Although Python 3 was already available online, it seemed to have just been released and the ecosystem wasn't that rich yet, so he'd stick with 2.7 to save himself some trouble.

The installation went smoothly. Then, he began configuring the necessary libraries. Fortunately, the school's internal network resources were quite rich, providing a local mirror source for pip.

pip install requests

pip install beautifulsoup4

pip install lxml

With a few commands, the core dependency libraries were successfully installed.

Thanks to the school's IT department! As expected, standing on the shoulders of giants is comfortable; no need to hand-craft everything starting from the HTTP protocol.

But soon, he hit his first hurdle. When he tried to install pandas for data analysis, the command line threw a headache-inducing pile of dependency errors.

As expected, configuring a Python environment on Windows will always give you 'surprises'. I was spoiled by Anaconda in my past life; only now do I realize the cruelty of a native environment.

He sighed and decisively gave up on pandas.

"Without pandas, I can't write data analysis? You're underestimating my ability to manually optimize!" He rolled up his non-existent sleeves. "Isn't it just dictionaries, lists, and loops? I'll write the statistical logic myself; the performance might even be better! (Big lie)"

Core development, begin!

He created a new py file and started typing the code.

#Import the life-saving libraries first

import requests

from bs4 import BeautifulSoup

import time

import datetime

import re

Fortunately, the school forum could be accessed via a computer browser, not just the mobile version.

"First, I need to simulate a login and get the cookie..." he muttered to himself while checking the forum's login interface. "This form structure is a bit retro."

Soon, he finished the network request portion and successfully scraped the HTML of the forum's homepage.

"Next is parsing the post list and extracting the posting time." He stared at the dense HTML source code and frowned. "This DOM structure... it's written a bit carelessly. No unified CSS class names? I'll have to brute-force it with regex and parent-child node positioning."

BeautifulSoup isn't omnipotent either. When you encounter non-standard HTML, you have to rely on sheer willpower! It's like being given a Swiss Army knife, only to find that most of the time you have to use the toothpick inside to pick a lock.

He threw himself entirely into the struggle with the HTML structure. Time slipped away amidst the sound of clacking keys.

He didn't know how much time had passed before the protest from his stomach pulled him back to reality. Outside the window, the sky had already darkened.

"As expected, coding is fun for a moment, but coding for too long... makes you hungry." He rubbed his sore eyes and decided to solve the dinner problem first.

Fortunately, the school had more than just one cafeteria. Natsukawa found a decent-looking ramen shop on the commercial street to satisfy his appetite.

After finishing dinner, Natsukawa decided to stop by the supermarket to buy some easy-to-prepare ingredients. After all, he wouldn't always want to go out to eat, and it was better to have some supplies on hand just in case.

He picked up some rice, noodles, pork belly, eggs, and other ingredients from the supermarket, then walked back to the dorm through the cool night breeze.

The night was deep, and the lights in room 403 flickered on once more. Natsukawa took a quick shower and returned to the world of code. He optimized the parsing logic and personally wrote the core functions for time string conversion and hourly statistics.

Nearing one in the morning, he ran the final test. The script executed successfully, and the console clearly output the time-distribution statistics for the number of forum posts over the past 24 hours.

"Done!" He let out a long sigh of relief, the sense of accomplishment dispelling his fatigue. "Although it's just a prototype, the core pipeline is connected! Requests for scraping, BeautifulSoup for parsing, manual data processing... a complete end-to-end service!"

This feeling was like successfully assembling a functional machine with limited tools, where every gear had been tuned by his own hands.

He saved the code and closed his laptop. Outside the window, the entire academy was silent.

"As expected, some things are just written in one's DNA." Natsukawa shook his head with a smile and turned off the room's lights. "For instance, spending the night resolving dependencies and debugging."

The room quietly plunged into darkness, but his campus life, accompanied by the first "information collector" he had built with his own hands, was quietly getting on track. Although he knew the true nature beneath the school's polished exterior, he couldn't simply state it outright; he needed tangible evidence to support his so-called 'deductions.' He had to make the buyer believe that the information he provided was worth the price. And this crude web crawler would become the eye that allowed him to 'gain the upper hand' in this peculiar school.