Mutable storage structs and mappings by mappings to strings or byte arrays?

What I am trying to achieve is to account for possible increased future complexity of the storage, not just increased future complexity of the logic.
Apparently, as far as I understand it, by current upgradeability standards, if I want to introduce a more complex storage to an already functioning storage, I have to either deploy a side storage contract, or, to put that complexity in a newer version of implementation contract. Basically what I have to do is to build a storage on top of a storage. And if there will be a need to increase complexity of the storage even more in the future, it will require to either deploy yet another side storage contract, or it will require to move all storage from older implementation to a newer one. Something like that? But how would you actually operate within all that possible complexity spread over many contracts? Isn’t it just a flawed solution? Isn’t it more convenient to have a mapping to a string, which can contain possibly an infinite amount of complexity?
So as far as I understand how upgradeability standards are designed, we could have this as an example of a storage which could be upgraded:

mapping (address => uint256) private _balances;
uint public reserved;
struct Proposal {
    uint id;
    bool executed;
}

And if we need to upgrade, alter that storage, for example something of this gets deprecated, or add more complexity in the future, then what we will have to do is to deploy:

mapping (address => uint256) private _balances;
mapping (address => bool) private _staked;
struct Proposal {
    uint id;
    bool executed;
    bool pending;
}

Correct?

==============================================================================

And what I am thinking of, is that why bother, just use strings as reserve, so the first storage could look like this instead:

mapping (address => uint256) private _balances;
mapping (address => string) private _stringValues; // reserved for future unaccounted values
struct Proposal {
    uint id;
    bool executed;
    string stringValues;
}

an example of how it is stored could be:

_stringValues[msg.sender] = "{bool_authorized=true,bool_staked=true,uint_lended=1.000000001000000000}";

it also could be bytes32 type instead and look like this:

_bytes32Values[msg.sender] = "{b_a=t,b_s=t,u_l=1.000000001}";

And it could be even better, like an infinite array of bytes32 values, could be cheaper and easier, no?

mapping (address => bytes32[]) private _bytes32Values;

So surprisingly for me what am I actually talking about is reserving as many possible variations of arrays of bytes, could be like this:

mapping (address => bytes32[]) private _bytes32Values;
mapping (address => mapping (address => bytes32[])) private _bytes32ValuesAddyAddy;
mapping (address => mapping (uint => bytes32[])) private _bytes32ValuesAddyUint;
// etc

There will be less need to upgrade storage, add side storage, lower possibility of storage collision in general, it will all be in one contract and it seems more convenient to me.
I could still be terribly wrong, it could be that side storage, or putting a part of a storage into newer implementation is easier or more effective? What am I missing? Is it actually a viable solution, or I just should try harder to understand current upgradeability standards better?

I am thinking that when OpenZeppelin has decided to go with unstructured storage solution, using byte arrays was maybe on the table, and for some reason was rejected. Then the question is, why? Was it because it is more comprehensive in some way than using infinite arrays? I don’t want to ruin possible work of all my life by accidentally mixing up storage in my complex project.

Also, if anybody has any examples of current live upgradeable contracts repositories on GitHub, please link. I am really new to all this, and you guys will be doing God’s work, if you help me to understand what’s better.

3 Likes

Hello @Solidity-Snake and welcome to the community! What you described is called Eternal storage and it was indeed considered for our upgrades solutions.

From Santiago Palladino's The State of Smart Contract Upgrades (suuuuper recommended lecture):

While it guarantees no issues during upgrades, it requires a major change in how all contracts are coded, incompatible with contracts that do not follow this convention, and produces far more awkward code. Using strings for identifying variables can also lead to errors due to typos, unless constants are used for the mapping keys.

2 Likes

Glad to be here! I am quickly becoming a fan of OpenZeppelin while studying your GitHub guys. Thank you for the link, it appears that OZ blog is a treasury of knowledge and I should have been check it out earlier.

2 Likes

Glad to know! Stay around then :slight_smile:

1 Like

A very good article, got it!

2 Likes

Just to confirm, need to make absolutely sure before moving forward: is this correct representation of upgradeability architecture? Is it viable to try and merge AdminUpgradeabilityProxy for several contracts, or better go with unique instance for every contract?

1 Like

Unifying AdminUpgradeabilityProxys would imply that all of your logic contracts are behind a single proxy, which is a single address, and a single storage. I wouldn’t do that for a few reasons

  • The Transparent Proxy Pattern is not ready to handle multiple implementations
  • You as a developer will have to be super aware of avoiding storage clashes among many implementations (wouldn’t be easier to have a single contract inheriting from all of those logics instead?), which would become harder and harder to keep track in future upgrades/implementations
  • For multi-implementation upgrade proxies, I suggest you check out the Diamond pattern, although we don’t provide any implementation nor recommend its usage.
2 Likes

True, it probably will be easier. Diamond pattern? Man, do I love reinventing the wheel all day :sweat_smile: So pretty much it finally clicked. Unstructured Storage is the best solution so far.

2 Likes

However, while using bytes32 arrays for storing more values than one as a string could produce more awkward code, which is harder to keep track of and could increase the cost of audits, it seems that in the end it could be times cheaper for users to interact with some functions. Basically this and a few more solutions could possibly allow for the cheapest gas on the market. Currently, I am inclined to actually do that like a madman.
Like, a boolean array in bytes32 string could allow to write 11 boolean variables(could be even more actually, up to 32) for only 20000 gas, and extract them at the cost of one. If so many boolean variables are not needed, then bytes32 could include block-timestamp or something else important which could fit.

bytes32 booleanArray = "st,af,pt,bf,mf,rf,wt,lf,ff,tt,dt";

For example, I could optimize voting by allowing only whole numbers of votes(or simply truncate), which at it’s simplest form could be a value of vote and a boolean if it’s for or against:

bytes32 vote = "10000,support=false";

And for a particular proposal in general, votes against and votes for could be stored in bytes32 string divided by comma. So it would be 2 storage writes instead of normally 3, 50% cheaper. Basically allowing more participation. Also users could vote for numerous proposals in one cheap transaction, but that should probably be a string type.

1 Like

Or, I could just pack bytes8 or less in one storage write. A hashed string could contain a lot of values.
For further gas savings I could divide logic contracts in smaller parts. Like, GovernanceV1 contract could be divided in GovernanceProposeV1 and GovernanceVoteV1, both controlled by same AdminUpgradeabilityProxy. As far as I could understand how that could work, there can’t possibly be any clashes between functions of GovernanceProposeV1 and GovernanceVoteV1. Is that correct?

1 Like

So apparently uint256 can store up to 256 boolean values. Xdai is not even required.
Could also be switches, could also be booleans and switches dependent on each other which would allow to pack even more in one uint, like 300+ variables
Hex with capital letters is suboptimal in my case. Uint could store a string with length of 51. And it could be optimized even further in some ways, one of which could be to include most common 2 letter variations.

1 Like