Openzeppelin upgrades - new instance in a contract

Yes, that sounds exactly right!

Check out this thread, there are some factory code samples: