Hi anavn!

If you mouse over the upgrade button, it likely has a tooltip saying it is because you're in combat. Since many Keep bonuses have combat related stats, you aren't able to upgrade during active combat. Try exiting your battle, and see if it lets you do it then :)