TL;DR: How can we merge contracts and contracts-ethereum-package into a single offering, making it transparent to the user whether they are using upgradeable contracts or not?
If you are here in the OpenZeppelin forum, dear reader, you are probably familiar with our @openzeppelin/contracts library, which sports popular contracts like SafeMath, Ownable, or the most widely used ERC20 implementation.
However, it's possible that you have missed our @openzeppelin/contracts-ethereum-package. This is a fork of the vanilla contracts package, where all contracts have been adapted to be upgradeable. This means, for instance, that they use initializer functions instead of constructors.
Note that the
ethereum-packagevariant not only includes upgradeable versions, but also provides the addresses of reusable pre-deployed logic contracts, to save gas on deployments in public networks. But we will not focus on this here.
This duplicity has caused much confusion among our community, so we want to explore alternatives for removing it, and unifying everything under a single package. But let's first recap how we got here in the first place.
The SDK and upgradeable contracts
The OpenZeppelin SDK (formerly known as ZeppelinOS) shipped since its version with upgradeable contracts as its main selling point. However, for a contract to be upgradeable, and thus used within the SDK, it needs to conform to a set of rules required by the proxy pattern we use:
- It cannot have a constructor, nor can any of the contracts it extends from
- It cannot assign initial values in variable declarations
- It cannot remove or change the type of any existing storage variable when upgrading
- Any contracts created from Solidity using
MyContract.new()cannot be upgradeable
Since contracts did not conform to this pattern, and we did not want to remove constructors (they are useful after all!), we built the contracts-ethereum-package version.
To illustrate this, here is the constructor of the Ownable contract in the contracts package:
constructor () internal {
_owner = msg.sender;
emit OwnershipTransferred(address(0), _owner);
}
And the corresponding initializer of the Ownable version in the contracts-ethereum-package:
function initialize(address sender) public initializer {
_owner = sender;
emit OwnershipTransferred(address(0), _owner);
}
This duplicity in the contracts packages meant that users needed to use a different one depending whether they were building with our SDK or in a different platform. As explained in the SDK documentation:
Make sure you install
@openzeppelin/contracts-ethereum-packageand not the vanilla@openzeppelin/contracts. The latter is set up for general usage, while@openzeppelin/contracts-ethereum-packageis tailored for being used with the OpenZeppelin SDK. This means that its contracts are already set up to be upgradeable.
This has lead to much confusion among the community, not to mention the efforts to port every change in the vanilla contracts repository to its upgradeable counterpart. Thus, we have had in the back of our minds different options to solve this problem.
Towards frictionless upgradeability
Over a year ago, @frangio wrote a fantastic blogpost explaining different approaches to automating the process of making a regular contract upgradeable. Go read it if you want to learn more about this!
The main takeaway of the post is that there are two main approaches to accomplish this feat:
- Manipulate the Solidity source code to change all
constructors toinitializers. This can be done through the AST that the compiler exposes, taking care to preserve the constructor semantics involved with multiple inheritance. - Extract the constructor bytecode from the compilation result. This can be either deployed as an initializer contract, or passed as the proxy's constructor. This is more direct, but requires some transformations directly on the assembly.
The first approach is more sensitive to changes in the Solidity source code, while the second to changes in the compiler codegen. Either way, both seemed promising at the time, and still do.
Our requirements
At this point, it's worthwhile pausing and understand what are our requirements for what we are trying to accomplish.
- We want users of the SDK to just install
@openzeppelin/contractsand use them. - We want the SDK to support both upgradeable and vanilla contracts.
- We want the deployed proxy and logic contracts to be verifiable on Etherscan.
The last item means that we cannot freely manipulate the bytecode, but we need to be able to produce a Solidity contract that compiles to it. The second one implies that we need to watch out for name clashing between upgradeable and vanilla contracts on the same project.
And the first one leads us to two paths: transforming the contracts at the package and shipping duplicated versions, or adding these automated transformations directly on the SDK. Let's explore both approaches.
Plan A: Ship upgradeable versions within @oz/contracts
The seemingly easiest option is to package all the upgradeable versions in a separate folder in the contracts package. Thus, we could have both ownership/Ownable.sol and upgradeable/ownership/Ownable.sol. The upgradeable versions could be generated manually, as we do today, or via a script that modifies the source code automatically, and then validated by us.
However, this has the problem that the user needs to knowingly import from a different path - not much of an improvement over the current situation. We could try to automatically manipulate the import paths if the user is writing an upgradeable contract, but since Solidity imports bring into the current namespace pretty much anything they find, this could easily get out of hand.
Furthermore, this is bound to end up in name clashings. If we have two contracts with the same name in the same project, and they accidentally get both imported into the same file, there is no way to refer to one or the other, and the compilation fails.
This means that we would need to ship all contracts with a prefix, ending up with Ownable and UpgradeableOwnable in the contracts package. Users would be able to install a single contracts package, but they would need to be explicit on which contract they use when writing their own, depending on whether they are writing vanilla or upgradeable contracts.
It's unclear how much of an improvement we get here. At least, we can identify when the user is writing an upgradeable contract, we can walk through the base contracts, and warn them if they are using a non-upgradeable version.
It's worth discussing as well how to detect when the user is writing an upgradeable contract. We have a few options here: 1) adding a magic comment to signal it, 2) extending a marker interface, or 3) deciding based on the
createcommand issued by the user. While (3) is less cumbersome for the user, it has the downside of not having information on how the contract will be used until deploy-time.
Plan B: Automate initializers in the SDK at compile-time
The alternative is to build the process of automating initializers into the SDK itself. When we detect that a contract is marked as upgradeable (note that this needs to be only on the most-derived contracts, not the base ones) we can reprocess it to remove the constructors from it and all its ancestors, modifying either the source code or the bytecode. The resulting artifact can be compiled into a separate build/upgradeable folder to avoid name clashes, and then pulled from either that folder or the vanilla build/contracts one depending on which version the user wants to use.
The main advantage of this approach is that any contract that the user pulls in to their repo becomes automatically upgradeable, not just those from @openzeppelin/contracts. As a bonus, we could also inject storage gaps in intermediate contracts to make upgrades easier during these transformations.
On the other hand, this requires more effort than Plan A, since it requires (just to begin) a production-grade automatic script for transforming contracts on-the-fly.
There is also the risk of turning the SDK into too much of a black box. Would you as a developer feel comfortable having a tool that deploys a contract different to the one you've written? If the transformation is done at the Solidity level, we could store all the modified source codes in a separate folder (such as build/upgradeable-sources) so users can review them.
Also, there is the matter of tooling. How would a debugger behave in this case, given that sources are in a non-standard path? We would need to check that tools properly honour the sourcePath property in the build artifacts.
There's also the matter of creating upgradeable contracts from Solidity itself. How should we manage a MyContract.new(params) in Solidity code, when making these transformations? An option is replacing it for a sequence of instance = MyContract.new(); instance.initialize(params);, but transformations start becoming more complex - more than we would like.
A solution to this would be to autogenerate the upgradeable versions in Solidity within the contracts source folder in the project, in a separate /upgradeable subfolder. This way users could explicitly refer to these versions. Note that this would also imply renaming each contract to avoid clashes.
As you can see, there are many approaches to this problem. We could start with Plan A, and as the tool for autogenerating initializable contracts matures, then shift into B. However, Plan A introduces some inconveniences for the user that we would need to patch in the SDK, that would go away if we go straight with B. Alternatively, we could stay with separate packages for a while, with the difference that we autogenerate contracts-ethereum-package using this tool.
We also need to settle on whether to make the transformations at the bytecode or the source code level. While bytecode level sounds more straightforward, since the constructor code is already packaged into a single chunk, making changes at the source level make it more auditable and easier to interact with.
What are your thoughts? And what have been your experiences around the duplicity of the contracts packages?
)