Why doesn't anyone refer to Sarumon as "Sarumon of Many Colors"?

Ive only read the first book and dont care about spoilers if it comes up later. When we first hear of him, he's Sarumon the White but from the first point he appears he identifies himself as Sarumon of Many Colors. Feels strange that I've never heard this before. Seems like a classic LotR "Um, Actually" opportunity.