Worried about using upt

I've been using my upt and leaving and hour or 2 early every shift, well last night my manager came up to me and said he noticed I've been leaving early and wanted to know why!

I thought if I had the time I can use it as I please? Why is me leaving early a big deal to the manager.. now I feel uncomfortable to use it and I wanted to leave early tonight.. I'm so sick of this place