Voldemort's astonishment toward Vizet transformed into hatred for Vizet, and hatred for Dumbledore.
Back when he had possessed Quirinus Quirrell, he had learned some of Vizet's past through publications like The Daily Prophet, as well as the reason Vizet had entered Hogwarts.
Vizet's experiences were actually similar to his own, except he had grown up in an orphanage, while Vizet had grown up being tortured by a Dark Wizard;
Yet after he and Vizet entered Hogwarts, Dumbledore had shown completely different attitudes.
After he entered Hogwarts, Dumbledore's attitude toward him had been more of an omnipresent vigilance;
No matter how outstanding his grades were, no matter how much praise and esteem he received from the other professors, Dumbledore had always kept his distance from him;
Dumbledore always looked him over with suspicion, as though he believed that as long as he cast a spell, it had to be Dark Magic, and as long as he gathered friends, they had to be discussing some evil plan;
Even during that "Chamber of Secrets Heir Incident," when Rubeus Hagrid, the culprit, had clearly been caught, and Armando Dippet had clearly no longer been going to close the school, Dumbledore had still clung to him and refused to let go;
When he had completed his studies and wanted to join Hogwarts as a professor, Dumbledore, who had already become headmaster, had still firmly opposed it, refusing to let him become the Defence Against the Dark Arts professor;
He still remembered this matter vividly even now. Dumbledore had actually brought up what happened when he first entered school, brought up how he had burned a wardrobe to frighten him. It was utterly hateful!
Yet Dumbledore treated Vizet with another attitude entirely: tolerance... tolerance everywhere.
First was his tolerance of Vizet's identity as an Obscurial.