Hardhat deploy on xDAI "TransactionMinedTimeout [Error]: Timed out waiting for transaction"

Looks like Ive got a similar problem as

Were migrating into hardhat. Im using xdai if it matters.

local deploys work every time.
mainnet deploys get to same place before timing out each time.

The deploy script deploys some other contracts as part of the project, 2 of them also upgradable, and pre deployed 4 libraries, which are given to the third contract which fails with a txhash that doesnt exist…


  const DC = await ethers.getContractFactory("DC", {
    libraries: {
      a: a.address,
      b: b.address,
      c: c.address,
      d: d.address,
    },
  });




  const dC = await upgrades.deployProxy(
    DC,
    [init1, init2, init3, init4],
    { unsafeAllowCustomTypes: true, unsafeAllowLinkedLibraries: true }
  );// <-time outs here

    You are using the `unsafeAllowLinkedLibraries` flag to include external libraries.
    Make sure you have manually checked that the linked libraries are upgrade safe.

TransactionMinedTimeout [Error]: Timed out waiting for transaction 0x2035336a41b178ec16035e93621acd3f0107ec20e877ff4fc2215d909c138f49
    at Object.waitAndValidateDeployment (/contracts/eth/node_modules/@openzeppelin/upgrades-core/src/deployment.ts:61:15)
    at fetchOrDeployGeneric (/contracts/eth/node_modules/@openzeppelin/upgrades-core/src/impl-store.ts:42:5)
    at Proxy.deployProxy (/contracts/eth/node_modules/@openzeppelin/hardhat-upgrades/src/deploy-proxy.ts:45:18) {
  deployment: {
    address: '0xE86FB472D997bD489EEd806f6b138518d5473De0',
    txHash: '0x2035336a41b178ec16035e93621acd3f0107ec20e877ff4fc2215d909c138f49',
    layout: { storage: [Array], types: [Object] }
  }
}
    "@nomiclabs/hardhat-ethers": "^2.0.1",
    "@nomiclabs/hardhat-waffle": "^2.0.1",
    "@openzeppelin/contracts-upgradeable": "^3.3.0",
    "@openzeppelin/hardhat-upgrades": "^1.5.0",
    "hardhat": "^2.0.7",
    "hardhat-typechain": "^0.3.4",
2 Likes

Got the DEBUG enabled but mostly says same thing, its looking for txhash that it thinks it submitted but doesnt exist?

    You are using the `unsafeAllowLinkedLibraries` flag to include external libraries.
    Make sure you have manually checked that the linked libraries are upgrade safe.

  @openzeppelin:upgrades:core fetching deployment of implementation f3b04f614a3a1e805085ed0d5729854e07fa045ae4fbaf7bd7a23d8ca77e0e3a +53s
  @openzeppelin:upgrades:core deployment of implementation f3b04f614a3a1e805085ed0d5729854e07fa045ae4fbaf7bd7a23d8ca77e0e3a not found +2ms

  @openzeppelin:upgrades:core initiated deployment 0x2035336a41b178ec16035e93621acd3f0107ec20e877ff4fc2215d909c138f49 +1m
  @openzeppelin:upgrades:core verifying deployment tx mined 0x2035336a41b178ec16035e93621acd3f0107ec20e877ff4fc2215d909c138f49 +4ms
  @openzeppelin:upgrades:core waiting for deployment tx mined 0x2035336a41b178ec16035e93621acd3f0107ec20e877ff4fc2215d909c138f49 +250ms
  
  @openzeppelin:upgrades:core verifying deployment tx mined 0x2035336a41b178ec16035e93621acd3f0107ec20e877ff4fc2215d909c138f49 +5s
  @openzeppelin:upgrades:core waiting for deployment tx mined 0x2035336a41b178ec16035e93621acd3f0107ec20e877ff4fc2215d909c138f49 +223ms
...
  @openzeppelin:upgrades:core verifying deployment tx mined 0x2035336a41b178ec16035e93621acd3f0107ec20e877ff4fc2215d909c138f49 +5s
  @openzeppelin:upgrades:core waiting for deployment tx mined 0x2035336a41b178ec16035e93621acd3f0107ec20e877ff4fc2215d909c138f49 +221ms
TransactionMinedTimeout [Error]: Timed out waiting for transaction 0x2035336a41b178ec16035e93621acd3f0107ec20e877ff4fc2215d909c138f49
    at Object.waitAndValidateDeployment (/contracts/eth/node_modules/@openzeppelin/upgrades-core/src/deployment.ts:61:15)
    at fetchOrDeployGeneric (/contracts/eth/node_modules/@openzeppelin/upgrades-core/src/impl-store.ts:42:5)
    at Proxy.deployProxy (/contracts/eth/node_modules/@openzeppelin/hardhat-upgrades/src/deploy-proxy.ts:45:18) {
  deployment: {
    address: '0xE86FB472D997bD489EEd806f6b138518d5473De0',
    txHash: '0x2035336a41b178ec16035e93621acd3f0107ec20e877ff4fc2215d909c138f49',
    layout: { storage: [Array], types: [Object] }
  }
}
2 Likes

Hi @jacobrosenthal,

Welcome to the community :wave:

I was able to deploy a simple upgradeable contract to xDAI: OpenZeppelin Upgrades: Step by Step Tutorial for Hardhat so assume it is the linked libraries which results in the timeout of 60 seconds.

Can you try updating to the latest version of @openzeppelin/hardhat-upgrades in case that resolves the issue. (You are using 1.5.0 and the latest is 1.6.0).

Tried. no change.

Seems to be a bug in the “deployment of implementation” “not found” path.
Thats the only time that path is hit in the logs, and the one that has a nonexistant txhash.

1 Like

How are you connecting to xdai?

1 Like

https://rpc.xdaichain.com

1 Like

The contract at the address seems to have been created, but the transaction hash is a different one.

https://blockscout.com/poa/xdai/address/0xE86FB472D997bD489EEd806f6b138518d5473De0

I don’t really know what could be going wrong here. We obtain the deployment transaction hash from ethers.js:

const instance = await Contract.deploy(...);
instance.deployTransaction;
1 Like

Hi @jacobrosenthal,

I’d like to be able to reproduce the issue. I am not familiar with how libraries are linked in ethers.js and I couldn’t find an example in the ethers.js documentation.

Are you able to create a simple example that reproduces the problem?

Thanks. I wrote up a github issue. Let me know if that shows linking enough.

1 Like

I believe @abcoathup was asking for code to reproduce the issue. The code you shared in github issue is not executable as is, there’s no contracts and the script code is incomplete. Bear in mind that “reproducible code” is meant to be executed, not read.

In order for you to get the best help in the shortest time possible, I suggest you make this as simple for other people as possible: either by simplifying your script and adding dummy contracts or directly setting up a full working repo.

1 Like

Hi @jacobrosenthal,

I wasn’t able to reproduce with a simple example.
I found how to link libraries in Hardhat: https://hardhat.org/plugins/nomiclabs-hardhat-ethers.html#library-linking

I got the same timeout using your code.

I updated the issue with this information.

1 Like

@jacobrosenthal I was able to deploy your contracts using your deploy:prod script.

I did get a timeout when deploying DarkForestTokens, but in my case it was apparently just a timeout, and I worked around it by rerunning the script, with modifications to avoid redeploying what was already on chain. The key is that if you rerun the same deployProxy or upgradeProxy function call, it will resume where it had previously timed out. You can wrap deployProxy around a try..catch block to rerun it if it times out until it succeeds. (We can make the timeout configurable but this is a quicker workaround you can implement.)

I’m still really surprised that the contract address that you shared was found on chain but deployed with a different transaction hash than the plugin had observed. Since we are seeing all of these other deployments on xDai succeed, I’m thinking this was an outlier error that must have been caused by an unrelated reason, maybe the transaction was replaced by one with a higher fee or something?

To sum up, I believe the timeout that you are seeing can be worked around by using try..catch to rerun deployProxy, which will resume waiting for the already sent transaction that caused the timeout.

1 Like

Ive never seen a timeout on DarkForestTokens, mine was always the same error on the final Core deploy.

However trying again now (without try catches or any changes) I actually see it complete successfully-- but only 1 out of 4 times. I havent had the timeout with no tx hash anymore. now I usually get one of

  • hangs forever after deploying whitelist
  • ProviderError: Transaction gas price supplied is too low
  • ProviderError: Transaction nonce is too low
1 Like

The first two sound like they could be related to a low gas price. You should check what gas price your transactions are using.

I’m not sure what could be causing the third error. Did this show up in the middle of your deployment script or was it when you reran the script after a failed run? Perhaps there was a transaction pending in the mempool that cause the nonce to become out of sync.

1 Like

I’m not specifying gas or gasLimit in hardhat config so they should be ‘auto’ assuming hardhat-upgrades is using those values. Ive also explicitly set those values to ‘auto’. I wonder about the code path for redeploy. Is it correctly looking at last nonce and gas and adjusting appropriately?

Im actaully trying to do a fresh deploy in this case so I dont care to recover some old deploy, so I completely deleted the .openzeppelin directory and was getting the timeouts now again all of a sudden, so Im using npm edit @openzeppelin/upgrades-core to edit src/deployment.ts.

-    const pollTimeout = 60e3;
+    const pollTimeout = 160e3;
-    const pollInterval = 5e3;
+   const pollInterval = 1e3;

And now Ive seen my deploy mostly work.

Others trying it seem to be at 100% without any changes. But I randomly Ill still see it hang forever. I’m known to have pretty bad internet here so I think theres definitely something that could be done around retry logic, timeouts.

1 Like