Definitely recoil results exist, but normally (as far as I can recall) in bigger battle rules and for earlier periods. Modern period close combat rules tend towards once you are committed, that’s it…definitely not saying that a recoil mechanism is wrong mind.
My own view is that…it is complicated. If you get close enough (say within 20-30m) and there is no cover to pull back into, troops are going to shoot it out (or freeze, or surrender); if there is some kind of possible escape route, then that option is always open. And often there is a possible escape route, since hopefully the attacking troops used it to get close in the first place.
Anyway, really looking forward to having a look at your rules. What is the specific issue with the reserves? Hard to design a mechanism to reflect how hard it is to control committed squads/sections?