"No control input" is synonymous with "control inputs neutral".
No, neutral means all stick inputs have a 0.0 value.
The FC takes over and does all the twitchy hard work to stay in place.
Exactly, the FC is computing the angles required to hold position.
Give some input, and the FC gets (mostly) out of the way
How much out of the way? E.g. if roll increases from 0.0 to 0.01 is that enough? Or, maybe increases to 0.0314159?
I'm going to take a chance here and speculate that you don't have much experience with DJI drones prior to the FPV. It's well established that prior DJI drones when in GPS mode will operate as has been described here by others and myself. The FPV in N/S mode does the same.
