For most games I have a set of boxes drawn on a piece/s of paper off table, with numbers or description of the buildings on table on them. A unit moves into a building – it disappears from the table and sits in the relevant box until it exits or dies. Yes people forget they are there – attackers suddenly get shot to pieces from what they had assumed was an unoccupied farm complex or defenders don’t shoot at troops outside they could have. Two things I regard as a bonus not a problem. Declutters the table and you add a small layer of fog of war. But if you want people to know at a glance; your system certainly does a good job.
I like the idea of your fog of war element and can see a benefit to that. I play mostly solo so I know anyway where each unit is, unless I specifically plan for some hidden deployment/surprise deployment in a specific scenario.