Jack of all trades master of none?

doesn't this hamper your progress when people say this thing to you.For instance , I'm sort of all rounder, good in sports , academic and any other thing that interests me.But when thing seems to look overly competitive or reward oriented it kills all my excitement to pursue it further because it feels draining to me lol, is it normal?.People often label me as 'jack' but in reality I don't want to 'master' those things after some point at first place and it bugs me how no one gets me.