No matter how many fret polishes I do with steel wool, some frets have this weird layer on them. Do I need a refret?

Had this Fender Strat for a few years now. When I purchased it, I had to spend many hours polishing and sorting it out, it was much worse. The seller didn't show the fret detail very well and I drove many hours to pick it up, but it is my main guitar and plays well, however, it is always bugging me having this weird stuff on some of the frets. Steel wool / metal polish did no good (2nd time now) Has anyone encountered this or know what it could be? Thank you!