Proxy Implementation Library Contract modify calling contract storage Uint as struct

Hi Andrew @abcoathup,

I trust that you are well.

I’m playing around with a proxy contract system in Remix connected to Ganache via Metamask. I want the function in the implementation library contract to modify the state of the calling contract. So, I need to pass in the storage variable as a parameter (otherwise, how will the library know).

I’ve started with a simple setup where there is a single uint variable to modify. I started by trying to pass in the uint directly, but the compiler threw up an error as it didn’t accept that the uint could be a storage varible (only structs, mappings can be storage variables?). Then I came across an example in a Medium article (https://medium.com/@kidinamoto/library-driven-development-of-solidity-ec7b0b41f38) where they enclosed the uint in a struct. I assume that this is a workaround to be able to give a uint a storage modifier. Does that sound right? (The immediate below is the example from the article)

library TestLib {
  struct Data {
    uint n;
  }

Relatedly, assuming that the above is corrrect, I need to encode a function call so that I can pass it from the proxy state contract to the implementation contract. My testing contracts are below.

The examples from the Solidity documentation (https://solidity.readthedocs.io/en/v0.4.24/abi-spec.html#function-selector-and-argument-encoding) don’t refer to how to deal with structs. If I was able to call a function with a uint, then it would be no problem. Unfortunately, I don’t know what to hash to produce the function selector. Would I hash ‘setNumber(Data)’? (The first four bytes of this hash are 278de5b8, by the way)

And what about the parameter? Would that just be a uint? So, 0000000000000000000000000000000000000000000000000000000000000045 would be 69.

Actually, I just realized that the parameter is only a reference to a storage variable in the calling contract. So, I shouldn’t have to pass in an encoded parameter. How does that work?

I tried throwing 278de5b80000000000000000000000000000000000000000000000000000000000000045 at the state contract using Remix to send it in the transaction payload. As the state contract inherits, proxy.sol it should forward it on to the implementation contract, but it didn’t work.

Transaction failed! Error:[ethjs-query] while formatting outputs from RPC

 library ImplementationTesting {
        struct Data {
        uint x;
    }
   function setNumber (Data storage x)
    public
    {
        x.x = 1;
    }

}
pragma solidity ^0.7.0;

import './AdminUpgradeabilityProxy.sol';

contract StateTesting is AdminUpgradeabilityProxy {
    
    uint public x;

    constructor(address _logic, address _admin, bytes memory _data) AdminUpgradeabilityProxy(_logic, _admin, _data)
    {
    x = 32;
    } 
}

I realize that this is a bit convulted. If you can make sense of any of my issues and provide some direction, it would be appreciated. Cheers.

Sincerely,
Craig

1 Like

Hi @cjd9s,

I wanted to take a step back and understand what you are trying to do? As I am not sure I see the need to do this.

I wasn’t sure why you wanted to extend the proxy contract? Is there functionality that you need?

Hi Andrew @abcoathup,

I appreciate the reply.

Essentially, I’m trying to delegatecall a function in the implementation contract (library type) from a proxy contract. My understanding is that I need to encode the function call so that I can send it in the payload of a transaction (i.e. msg.data) to the proxy contract. When the proxy contract receives this transaction, it will invoke its fallback function which will delegatecall the function call to the implementation contract.Ultimately, this transaction will be generated through web3.js, but I am creating them manually at the moment and sending them via Metamask.

I am trying to get the most basic form of delegatecall to work (i.e. just with a single uint variable) so I can start to work on making it work with the more complicated data structures that I’m employing (e.g. ERC1155 mappings).

I want to extend the proxy contract because that is where I will store my election state variables. For example, the token mappings for ERC1155. FYI, there will be a contract factory that deploys a proxy election state contract for each election.

I’m not sure if that’s a sufficient picture for you to understand what I’m trying to do. Let me know. Thanks again for everything. Cheers.

Sincerely,
Craig

1 Like

Hi @cjd9s,

I am still unsure why you need to extend the proxy contract rather than using as is. I must be misunderstanding something.

I would recommend using Upgrade Plugins (https://docs.openzeppelin.com/upgrades-plugins/1.x/) to experiment with upgradeable contracts (including testing and upgrade safety).

To experiment with deploying proxy contracts you could first deploy using Remix as in this post:

Hi Andrew @abcoathup,

I appreciate the reply and the example. It’s easier to get on the same page when I have an example to work through.

BREAKTHROUGH!!!

I had a major breakthrough while reading an article about delegatecall (https://medium.com/coinmonks/delegatecall-calling-another-contract-function-in-solidity-b579f804178c). A portion of the article discusses how variable memory slot differences between the implementation contract and proxy contract can lead to incorrect values assigned to variables.

In the article, the implementation contract is called Calculator and the proxy state contract is called Machine. The Calculator contract declares uint calculatorResult (assigned to slot 0) and address user(assigned to slot 1). The Machine contract declares Storage (an ABI object used earlier in the article, but unused now - assigned to slot 0), uint calculatorResult (assigned to slot 1) and address user(assigned to slot 2). As delegatecall is used, the Machine contract is the context, but the slot number is based on the Calculator contract.

So, when a function is delegatecalled from Machine to calculator, the calculatorResult from the Calculator (slot 0) contract overwrites Storage (slot 0) from the Machine (slot 1) contract and the user address (slot 1) from the Calculator contract overwrites the calculatorResult from the Machine contract.The system fails a test that checks that the calculatorResult is 3, but is, in fact, the user address. FYI, the solution is to order the variable declarations in your implementation and proxy state contracts so that they align.

Anyway, I realized that libraries don’t have storage so how I could I use a library as my implementation contract. Once I switched to a normal contract, I realized that I just can just call functions in a straightforward manner. I didn’t need a struct to encapsulate a uint. Anyway, for the moment, I’m moving along after weeks of smashing my head into a wall. And I owe it all to the example that you gave me that forced me to work though the process. So, again, thanks for that. You’re making a big difference to me.

Now, with that said, I do have an issue with the example you provided. And it might lead to the heart of some of the misunderstandings between us. I’ve pasted a quick graph to better explain what’s going on.

Andrew

So, although you sent the encoded function call as part of the constructor function, it didn’t do anything. That is because the constructor function used a delegatecall which used a function found in the Box contract (store), but in the context (aka using the state variables) found in the TransparentUpgradeableProxy contract. As no such state variable exists in the TransparentUpgradeableProxy contract, nothing happened. If the constructor function had used .call instead, the context would have been the Box contract and the state variable in the Box contract would have been changed to 42.

I would note that if I call the store function directly in the Box contract, it modifies the Box state varaiable as expected.

So, I made a slight change. I inherited the TransparentUpgradeableProxy contract to a newProxy contract (see below) and I added a uin256 state variable.

contract newProxy is TransparentUpgradeableProxy {
    
    uint256 public value;
    
    constructor(address _logic, address _admin, bytes memory _data) public payable TransparentUpgradeableProxy (_logic, _admin, _data) {
        value = 2;
    }
}

Now, when I send your encoded function call to newProxy, either through Metamask or directly through Remix, it changes the state variable in newProxy to 42. So, to revert to our misunderstanding about why I would extend the proxy contract, it’s because I need to add state variables so that the when the proxy contract delegatecalls functions from the implementation contract, it has something to change. The delegatecall functions are all occuring in the context of the proxy contract.

Let me know if this sounds right to you or if I misunderstand something. Cheers.

Sincerely,
Craig

1 Like

Hi Craig (@cjd9s),

I am still either not understanding what you are trying to do, or I haven’t explained properly how Transparent upgradeable proxies work.

I recommend reading (if you haven’t already):

I am going to walk through the example using Remix to hopefully make it clearer

  1. Deploy the implementation contract (Box.sol). I deployed to Rinkeby.
// Box.sol
// SPDX-License-Identifier: MIT
pragma solidity ^0.6.0;

contract Box {
    uint256 private value;

    // Emitted when the stored value changes
    event ValueChanged(uint256 newValue);

    // Stores a new value in the contract
    function store(uint256 newValue) public {
        value = newValue;
        emit ValueChanged(newValue);
    }

    // Reads the last stored value
    function retrieve() public view returns (uint256) {
        return value;
    }
}

Transaction deploying the contract:

Implementation contract (Box): https://rinkeby.etherscan.io/address/0xd8400192f5f7b2bdba6ec3b5f6a838aa2b5b5b5e#code

  1. Deploy the proxy contract
    I used GitHub imports to get the contract.
// OpenZeppelinContracts.sol
// SPDX-License-Identifier: MIT
pragma solidity ^0.6.2;

import "https://github.com/OpenZeppelin/openzeppelin-contracts/blob/v3.2.0/contracts/proxy/TransparentUpgradeableProxy.sol";

Using the parameters of:
_LOGIC: 0xd8400192f5f7b2bdba6ec3b5f6a838aa2b5b5b5e (implementation contract - Box)
_ADMIN: 0x4924b7e51F71f2D3a57711C7a2Ac8a5ca3b2DBED (an account I control, we could also deploy a ProxyAdmin and use that)
_DATA: 0x6057361d000000000000000000000000000000000000000000000000000000000000002a (call store(uint256) with a value of 42)

Transaction deploying the contract: https://rinkeby.etherscan.io/tx/0xa8cfc31ef0e4c7c34277f258b0ed618fb5cb58325a3d54d86831ca37a2089183

Proxy: https://rinkeby.etherscan.io/address/0xfc38e2f5cf00dbab4884abb26bf21fe23d8af803#code

  1. Interact with the upgradeable contract
    Interact using the ABI of the implementation contract with the address of the proxy

Use the Box contract with the address of the proxy: 0xfc38e2f5cf00dbab4884abb26bf21fe23d8af803

Then press retrieve to get the initialized value

We can then store a value e.g. 23 (https://rinkeby.etherscan.io/tx/0xedf7ac46f8270301215cdf58c2615ee546690f0ec0ab8dd43760f2ae44ac26f0) and retrieve it.

A. (Optional) Verify the implementation contract on Etherscan
If you use Rinkeby you won’t need to do this as I have already done this.

B. (Optional) Verify the proxy contract on Etherscan
If you use Rinkeby you won’t need to do this as I have already done this.
I used Hardhat but used similar to this: Verify Upgrades Plugins proxy on Etherscan

arguments.js

// arguments.js
module.exports = [
    "0xd8400192f5f7b2bdba6ec3b5f6a838aa2b5b5b5e",
    "0x4924b7e51F71f2D3a57711C7a2Ac8a5ca3b2DBED",
    "0x6057361d000000000000000000000000000000000000000000000000000000000000002a"
  ];

Verify using Hardhat

$ npx hardhat verify --constructor-args arguments.js --network rinkeby 0xfc38e2f5cf00dbab4884abb26bf21fe23d8af803
Nothing to compile
Successfully submitted source code for contract
contracts/proxy/TransparentUpgradeableProxy.sol:TransparentUpgradeableProxy at 0xfc38e2f5cf00dbab4884abb26bf21fe23d8af803
for verification on etherscan. Waiting for verification result...
Successfully verified contract on etherscan

C. (Optional) Flag on Etherscan as a proxy
You would need to do this for your contract

D. (Optional) Interact using Etherscan
Read: https://rinkeby.etherscan.io/address/0xfc38e2f5cf00dbab4884abb26bf21fe23d8af803#readProxyContract

Press retrieve to read the value set earlier.

Hi Andrew @abcoathup,

I really appreciate the work that you’ve put into trying to get us on the same page. I’m know that this is frustrating for you so thanks for staying engaged and doing your best.

Unfortunately, the issue that I raised in my previous post remains.

To illustrate, I followed your first two steps. It’s better to not go to the third step to better isolate the issue. I used Remix Javascript VM, but the results will be the same.

  1. Deployed the implementation contract (Box.sol) exactly as you have written it. Check.
  2. Deployed the proxy contract exactly as you have written it (using Github import) using the same respective parameters of logic contract address (copied from the Remix interface), admin address (different Remix account), and _DATA (exact copy of the encoded function call of the function selector for the store function and an argument of 42). Check.

Now, here is the problem: when I invoke the retrieve function in the Box contract immediately after deploying the proxy contract, it returns 0. However, I imagine that you’re expecting this value to be 42. Otherwise, why would you send the encoded function call as part of the constructor function for the TransparentUpgradeableProxy contract. Do you get the same result?

This 0 makes sense because, as you can see in the UpgradeableProxy contract, which TransparentUpgradeableProxy inherits, the encoded function call sent as part of the constructor of TransparentUpgradeableProxy is delegatecalled, not called, to the implementation contract (Box.sol). So, the store function contained in Box.sol is invoked in the context of the TransparentUpgradeableProxy contract. So, it can’t modify the uint value variable in the Box contract.

However, if you added a uint value2 variable to the TransparentUpgradeableProxy contract (as I did in my previous post in NewProxy), that encoded function call sent as part of the constructor of TransparentUpgradeableProxy will set value2 to 42.

Again, thank you for your patience. If we can’t connect on our next post, perhaps we should just put this issue to bed and move on. Thanks again, as always. Cheers.

Sincerely,
Craig

1 Like

Hi @cjd9s,

Ok great, it looks like we have narrowed the problem down that you got different results when testing which then leads to the confusion between us.

I get 42 on JavaScriptVM (though I did get 0 when I tried yesterday when I was switching between contracts and possibly messed up my JavaScriptVM environment, so I can understand how that happened to you.)

42

I recommend trying on a public testnet (specifically Rinkeby as I verified the contracts). That way you can see all the transactions on Etherscan and use Etherscan's proxy support.

Hi Andrew @abcoathup,

Although this is harrowing for both of us, it’s forcing me to learn some new things about Remix, Etherscan, and the test networks. So thanks for all of this. It’s great stuff and you’re doing a good job.

I checked our your contracts on Rinkeby. I can see the transactions where you deployed Box(…1206), Proxy (…1255) (along with the delegatecall to Box with a value of 42), and where you delegatecall to Proxy with a value of 23 (…1403). I can see the associated emitted events that show these values so we know that the store function is being called in the Box contract (but, crucially, not in the context of the Box contract as we’re using delegatecall). However, I can’t actually see the value of the variable in the Box contract in the transactions.

So, I then load your Rinkeby-deployed Box contract on Remix. I click retrieve and the value is 0. And this is what I would expect as we’re using delegatecall.

To confirm that the store function works, I called store directly through the ABI exposed in Remix. I used the value 1234 and then used retrieve to confirm that the variable had changed to 1234. The link for that transaction is just below.

To illustrate how I expect this to work, I decided to do the same with the proxy contract slightly modified as previously described in the thread (newProxy). You can find the addresses and transactions below. All I have done is deployed the Box contract and then deployed the NewProxy contract (with your encoded function call as a parameter). If you load the Box contract and hit retrieve you’ll get 0. If you load the NewProxy contract and hit the getter function for the value variable, you’ll get 42.

My EOA
0x256200b7553ADd8c8544229fb9241459Ba6cf57b

Deployment of Box (0x653D19c64C2B5Af5e9578ed4d99Cc2DA48475cdB)

Deployment of NewProxy (0x67Bc7BC5468B2b01C4C0e8CB43D876598a18cE61)

Does this all seem right to you? Let me know if I’m misunderstanding something. Cheers.

Sincerely,
Craig

1 Like

Hi Andrew,

I just noticed something that help explain our misunderstanding. In your previous post, you have a screen grab that shows the contract name Box, but has the address for your proxy contract. I then figured out this is what you meant in step 3 by “Interact using the ABI of the implementation contract with the address of the proxy”.

42

So, I recreated this situation and, using your method, got the same result (42). However, if you use the implementation contract ABI with the implementation address to directly call receive(), you get a value of 0. So, it’s seems that the value variable in the box contract is 0.

So, how do we explain this discrepancy? Why do you get 42 when using the proxy contract address, but with the implemenation contract ABI? I believe that this is related to my earlier discussion about address slot differences between the proxy and implementation contracts. So, it seems that the encoded function call used during the deployment of the proxy contract did store a value in the proxy contract somewhere (slot 0). And when you use the retrieve function when using the proxy contract address with the implementation ABI, you’re accessing the slot 0 variable value in the proxy contract.

Now, I don’t know exactly which variable in the proxy contract is storing 42. Maybe none of the named ones and it’s just in slot 0 (or the first availabe non-constant slot, I don’t know how that works) As far as I can tell, the only variables available in TransparentUpgradeableProxy are constants. So, that’s not clear to me. What is clear to me is that if I extend TransparentUpgradeableProxy and add a uint variable, that variable is set to 42 during deployment of the proxy contract.

What do you think about that? Cheers.

Sincerely,
Craig

1 Like

Hi Andrew @abcoathup,

After some further thought I think I now understand why you don’t understand why I wanted to extend the proxy contract. In your example, it’s not necessary to add state variables to the proxy contract. You simply sended the encoded function call during deployment of the proxy contract and, I think, slot 0 in the proxy contract, which is undefined, is set to 42. You can then access slot 0 using the retrieve() function (because slot 0 is the location of the value variable in the Box contract). Is that your understanding of how this is supposed to work?

This also explains why you’re not concerned about setting the state variable in the Box contract which is what I thought you were trying to do.

Can you see how it also works if you extend the proxy contract and add a uint variable? That 42 is then written to that uint variable in the proxy contract.

I’d appreciate any comments. Cheers.

Sincerely,
Craig

1 Like

Hi Andrew @abcoathup,

I believe the gist here is that you’re not supposed to define the state variables in the proxy contract. The state variables are stored in the proxy contract in the slot defined in the implementation contract. For example, the uint256 private value variable in the Box contract is defined in slot 0 in the implemetation contract, but actually stored in slot 0 in the proxy contract. Is that correct?

So, the implementation contract is simply your whole contract (functions, state variables, events, enums, modifiers). And the proxy contract holds the variable values in the slots defined by the variable declarations in the implemenation contract. Is that correct? Cheers.

Sincerely,
Craig

1 Like

Yes yes yes!!!

Logic and state are defined in the implementation contract, but the actual storage is in the proxy.

This means when you upgrade, you actually get your proxy to point to a new implementation contract which can have different functions and additional state variables, as long as the original variables don't change type or order.

Hi @cjd9s,

Just wanted to check if you have any more questions on this?

Hi Andrew @abcoathup,

No. Thanks to your help, I now have a solid understanding of how proxy contract work. I’m now powering ahead with my project. Thank you for your assistance and patience. Cheers.

Sincerely,
Craig

1 Like