Sigh...
How had things ended up like this?
Xie Wen had left Makino's tavern and was following Garp toward Mount Colubo with the cats.
After hearing Xie Wen say that he "didn't know how to raise kids," and receiving Luffy's enthusiastic agreement, Garp had insisted on dragging Xie Wen up the mountain with them, saying he would show Xie Wen what he could do.
Clearly, Garp intended to have Xie Wen fight Ace up on the mountain to prove there was nothing wrong with his teaching abilities.
As for why his opponent wasn't Luffy...
Xie Wen glanced at the rubber boy, who was pestering the cats with endless questions and a stupid look on his face, and at his height, barely more than a head taller than Meow Jūshirō. The reason was obvious.
Leaving aside the fact that Ace as he was now definitely wouldn't be a match for him, Garp had also misunderstood what Xie Wen meant... though his teaching abilities really weren't much good.
But what Xie Wen had mainly meant was-
He had wanted to raise Luffy into a Marine, only to produce one of the Four Emperors instead; he had promised to look after Roger's child, only to dump Ace in a bandit den while he was still a baby... even leaving him with Makino or Foosha Village's mayor would have been better.
He barely saw his grandson a few times a year, and when they did meet, it was mostly punches and kicks. No wonder Luffy had no interest whatsoever in becoming a Marine.
Later, he did keep Koby and that "son of the Axe Emperor" by his side to train them, only to produce a "King of Deserters." Then, after Koby was captured by Blackbeard, he even led troops specifically to rescue him, going