Hi @EvilJordan,
As you found, the contracts are kept in project.json.
I really like the Preset contracts. They can be deployed without any Solidity code being written and cover many use cases. (e.g. Create an ERC20 without writing Solidity, using OpenZeppelin CLI). We can use the CLI to deploy upgradeable versions (we just deploy the proxy and proxy admin as the logic contract is already deployed).
I created an Issue for the documentation: https://github.com/OpenZeppelin/openzeppelin-sdk/issues/1540