Contract Upgrades with The Factory Pattern


I have a contract that acts as a factory. When called, it instantiates two other contracts. Also, it maintains a mapping of the instantiated contracts and the owner's address.

I am thinking of making the contracts upgradable. My first question is: given the setup above, is it feasible to make the contracts upgradable? given that the contracts are instantiated at run time and not know at deployment time. Can the factory use "new" to instantiate other contracts and then use "initializer" methods to set the parameters on the new contract?

If the above is doable, I assume that the factory itself has to be upgradable. What about the contracts it instantiates? I assume they, too, should be upgradable. Is this correct?