If you want to deploy via Truffle or Buidler, then I would recommend using the Upgrades plugins.
If you want to deploy upgradeable contracts using Solidity then I recommend inheriting from ProxyFactory. You just need to pass in your implementation address, your admin address (could be a ProxyAdmin if you wanted) and any initialization.
To upgrade proxy contracts you do need to upgrade each contract individually. So you would need to keep a list either on or off chain if you wanted to upgrade multiple contracts.
If you want to upgrade all the upgradeable contracts in one go, you could look at a different upgrade pattern.