Indeed, isContract can only be used to assert that an address is a contract, and should never be used to assert that an address is not a contract.
There is relevant discussion in Bypassing Smart Contract Timelocks. An alternative method proposed there is to require a signature from the address in order to verify that it is an EOA, and thus conclude it isn’t a contract.