Slick tires that can handle a little bit of gravel too?

Hey everyone, i got myself a gravel bike 4 months ago.

I wanted to go with a road bike, but given my location, i went with gravel (Lots of trails, questionable road condition nearby).

After quite some time i find myself mostly riding flat, okay-ish condition bike roads and roadways. I'd say 85% of my riding is on those surfaces.

I was thinking if changing my tires for 25-28C slicks (from 37C WTB Riddler). I'm wondering if any of you have any experience with slicks handling rougher terrain (gravely-bumpy).

Im averaging 15mph on 20 mile commutes to my workplace. Is changing tires to slick going to make the commute easier and/or faster, or should i explore other upgrades, like aero stuff first?