SDK 3.0 Design Document

We’ve decided to release a new major version of the SDK with significant improvements. This is a living document of the changes we’re planning. Most of it has come out of conversations with others in the team, but a few things are my own ideas that haven’t been discussed yet. This is a work in progress so please share your thoughts!

Goals

I believe the main goal of 3.0 should be to iron out the quirks that have accumulated since the initial prototype of ZeppelinOS.

Being a major version we will feel free to make breaking changes. However, it is also a goal that there should be a migration path for current 2.x users. We want everyone to benefit from the improved experience.

Automatic Initializers

So far users have had to write their upgradeable contracts in a special way with initializer functions. The conversion from constructors to initializers is completely mechanical, though, so we want to automate it for the user. We will do this behind the new deploy command.

The special thing about deploy is its --kind option, which can have different values: regular, upgradeable, minimal, and potentially more in the future, such as deterministic (using create2).

Transpiler

The conversion to initializers is done by a transpiler that will process Solidity files and generate other Solidity files. The transpiled contracts are renamed to …Upgradeable, for example Foo results in FooUpgradeable.

These files are placed next to the user’s contracts in the directory contracts/__upgradeable__. The reasoning behind placing them there is that it will interoperate better with other tooling that expects contracts to be under the contracts directory.

It will be possible to disable the transpiler for a project, and users migrating from 2.x might want to do this since their contracts will already be converted to initializers.

Removing Aliases

Since day 1 the CLI has had the concept of aliases. The idea is that you can assign an alternative name to a contract and its instances. We don’t expect a lot of people to know about this feature because it’s nowadays hidden in the oz create workflow, and contracts are almost always referred to by their name in the source code.

We want to fully remove aliases now, since we don’t think they really contribute to a good experience, and they add a lot of complexity internally. We do recognize that aliases as a building block can be used for many purposes, but in my opinion they are probably never the right abstraction. Here are some examples and alternative abstractions, that we may implement in the future.

Example Use Case #1
Aliases can be used when keeping multiple versions of the same contract side by side. For example, you may have FooV1 and FooV2 among your contracts in order to test the correctness of an upgrade. Initially you may have aliased FooV1 as Foo, but once you’re confident in the upgrade you can change the alias Foo to refer to FooV2, and run oz upgrade Foo.

A better abstraction for this use case would be the concept of snapshots. We could provide a command to store a named snapshot of a contract source: oz snapshot Foo v1. Later this snapshot could be used wherever we need to refer to a contract implementation, for example oz deploy Foo@v1.

In the meantime, we will add as an alternative a command that can be used to change a proxy’s implementation to another contract, such as oz upgrade-to FooV2.

Example Use Case #2
You can alias ERC20 as MyERC20 and deploy an upgradeable instance of MyERC20. If you later change the source for ERC20 and run oz upgrade MyERC20, this will upgrade only the MyERC20 instance.

This doesn't work so well for other kinds of commands. For example, it would not be reasonable to run oz call --to <alias> --method totalSupply, because the alias potentially refers to multiple contracts, and it's unlikely that the user intends to call a function on multiple contracts.

A better abstraction for this use case would be assigning names to instances. Similarly, we’ve seen the need to assign names to arbitrary addresses. Both of these things may be served by the same feature.

Removing App/Package/ImplementationDirectory

These are the contracts that support on-chain Ethereum Packages (initially called EVM Packages). Package and ImplementationDirectory were created as a way to store an on-chain versioned registry of contract implementations for a package, and App as a way to connect to multiple packages and act as a factory of proxies for those packages’ implementations.

Nowadays, they are mostly unused because of the overhead they add. The CLI normally works in a mode where all of that information is simply stored locally in JSON files, and proxies are created directly rather than through the App as a factory. The local JSON files work well or even better, so we want to remove support for these contracts, which contribute a lot of code and complexity internally.

There is one use case for which the contracts do serve a purpose, though, which is dynamically creating proxies on-chain. Before removing them, we need an alternative offering for this use case. We considered many options, a lot of them involving using the transpiler to convert a Solidity new statement automatically into a proxy creation. One central issue that kept coming up was how to decide who should be the admin of the newly created proxy. Ultimately, we decided that such an automatic conversion is too magic and hides important detail. We came to the conclusion that if a contract creates other proxies, then proxies are part of its domain, and therefore it is better to make it an explicit concern.

As an alternative to App, we will give users the tools to create their own proxy factories, in a Solidity library of proxy factories, and to hook in to the transpiler to get hold of the upgradeable source code for the contract they want to deploy dynamically.

// This import lets the CLI know that it needs to have Foo.sol transpiled
// and up to date. It puts FooUpgradeable in scope.
import "./__upgradeable__/Foo.sol";

contract MyFactory {
    using Proxies for Proxies.ProxyFactory;

    // This struct contains the address of the implementation that is
    // used for the proxies it creates.
    struct Proxies.ProxyFactory factory;

    constructor() public {
        // Initializes the factory struct by deploying the implementation
        // and storing its address.
        factory = Proxies.newFactory(FooUpgradeable.creationCode);
    }

    // initializerData is the encoded call to the initializer function.
    function deployInstance(address admin, bytes initializerData) public {
        factory.deploy(admin, initializerData);
    }
}

Argument Conventions

Commands that receive arguments for functions (including initializers) currently do so through an option --args whose value is the arguments separated by commas. For example:

oz create --args 1,2,a_string Foo
oz call --to <address> --method bar --args 1,2,a_string

This feels out of place in a shell interface where arguments are normally separated by spaces and the shell does the splitting into words. The new deploy command follows this convention:

oz deploy Foo 1 2 a_string

We should strive for a similar interface for all commands, where the primary arguments are specified as positional arguments. oz call, for example, should look like:

oz call <address> bar 1 2 a_string

Implicit Initialization

Users of the CLI currently have to initialize a project with oz init before doing anything (except compiling). This step basically doesn't serve any purpose for the user. They have to provide a name and a version which are not really used for anything (other than the Package contract mentioned above that we intend to remove). Additionally, they can specify options that will be stored in the config, but we really shouldn't expect users to know at this stage what options they will want for their project.

Internally the command is necessary because it initializes the .openzeppelin directory. This can and should be done implicitly when needed.

We should remove oz init and aim to be a tool that works without initialization. This will contribute to making oz feel like a leaner tool than it is today. Additionally, we should try to make it possible to drop oz into a project to use one or two features that the user finds useful, such as oz call or oz deploy. This will allow users to try the tool progressively and adopt it more and more as they discover the cool features it offers.

2 Likes

I can’t wait for Automatic Initializers, they will make life a lot easier for the community.

I have only used aliases when showing upgrading and testing versions (use case #1). I like oz upgrade-to FooV2.

I didn’t do anything with App/Package/Implementation so like the simplicity of ProxyFactory.

I’ve added a new section on argument conventions. I’ve also added some more rationale on why aliases are a bad abstraction in use case #2.

1 Like

@spalladino I’d like to hear your thoughts on sessions. I recall you mentioning that you don’t like them.

We do recognize that they can be useful to avoid repeating options all the time though. My impression is that the notion of opening a session that is then implicitly picked up until it expires is completely foreign to the shell. The fact that it happens implicitly can be dangerous and should be a bit worrying.

I was thinking that it would be better to simply have default values specified in config. For example, the networks.js file can have a default: "<network>" field. I’m not sure each network has attached to it the gas information and a default address for transactions, but those should be configurable as defaults too.

Those defaults would never expire, so I would change the interactive prompts to accomodate that. My current thinking is that we should have a flag --no-defaults and a corresponding question asking the user “Use default configuration for transactions?”. Based on that answer we can then query for network and all the other parameters if necessary.

1 Like

I’ve added a new section titled “Removing Initialization” about removing the oz init command.

Hi @frangio,

I like the idea of being able to install the CLI and use it to compile without having to do any setup.

Assuming oz init is removed, I would like there to be a way to create the project directories and network.js though perhaps that could be via a starter kit. Save having to remember which directory needs to be singular and which plural: contracts and test.

Also having network.js with commented out networks for public networks and useful tools e.g. Infura and @truffle/hdwallet-provider

On starter kits, it would also be nice to not have to use the CLI to unpack a starter kit which could contain the CLI.

1 Like

I think that sessions became a lot less useful once interactive commands came along. The fact that the last network used is pre-selected helps a lot, as its just an extra [enter] when running an operation. This makes me think that we could very well kill sessions for the purpose of maintaining the network selection.

Where sessions do help is in non-interactive commands. However, they are now much rarer, and should be found only in the context of a script. With that in mind, I'd drop the network selection from sessions, and just let the user set it via an environment variable (like $OPENZEPPELIN_NETWORK), so they have better control of its scope.

Now, sessions also keep track of other stuff, such as sender and timeouts. I'd move these to network.js instead (as you say), to make sure they are used consistently across all devs in a project.

I don't particularly like the idea of specifying a default network at the network.js. I think that we could expect that most usage would be in the development network, with occasional deployments to a testnet or mainnet. I'm not sure in which cases it would make sense to have a default that is not development.

1 Like

That's exactly right, which is why I'm suggesting the default setting.

I agree it wouldn't make a lot of sense, but we don't know the name of the development network, it's not necessarily always development (although it could be the default value for the default setting---not sure about this).

Environment variables are dynamic scope, as opposed to lexical scope. It results in the behavior of sessions but with an automatic expiration, which is interesting and certainly better. Is dynamic scope what users would want? If they want lexical scope they can implement it by defining a local bash function which calls oz with fixed options.

Either way I think we agree that we can replace sessions with something better.

2 Likes

Hey, community:
I like this design very much. I for one have never used aliases and have trouble remembering the , argument convention. And implicit init would be good, as I never seem to know the answers yet and just end up editing the config later anyway.

Mostly I’m excited for the new regular/upgradeable features. A couple of simple questions or points of clarification regarding planning for current projects in development:

  • Is it fair to conclude that when CLI 3.0 ships it will, going forward, sync up with the current version (3.0) of contracts, since presumably any initialize logic will now be automated through the transpiler process?
  • Users of the CLI should be orienting new development towards constructors rather than initializers?

Hi @kev.

I’m not sure what you mean by “sync up” exactly, but yes any initializer logic should be automated through the transpiler process. There are still some details we have to work out regarding the way storage changes across versions in Contracts will translate into the transpiled code. We will keep everyone updated on this.

I think it’s only slightly early to begin working with constructors rather than initializers. I would say to at least wait until we’ve released an initial usable version.

1 Like