Town specialization should allow additional buildings to be purchased with gold

Currently town specialization feels lackluster. There are few scenarios where you don't just convert to a Farming or Mining Town. I'd love to see more differentiation. Allow us to buy libraries/amphitheatres in Urban Centers. Barracks/Blacksmith in Fort Towns. Markets/Lighthouse in Trade Outposts and maybe a discount on merchants bought with gold. Anything to make the choice more interesting and strategic.