What is supposed to happen is that the unused part of the payment is carried forward. This carried forward amount should show on the Apply Payment screen right above the invoice list and when you create a new invoice for the same customer you should be offered the chance to apply that amount to the new invoice.
That said, I am aware of several bugs in this process that I need to work with NCH. There are cases where not fully apply payments remain in the list but are neither shown in the Apply Payment screen nor in any way marked in the payment list. 'Aslo, even if a unapplied payment is shown in the payment screen it will not be applied to a recurring invoice.
I have written code to analyze the payment list and identify unapplied payment amounts. It run an MS Access. Let me know if you want it.
++PLS