Vigilance only matters when Armory Guard is being declared as an attacking creature. If Armory Guard is already attacking, losing control of your only Gate won’t cause Armory Guard to become tapped or to leave combat.