Truffle, OpenZeppelin SDK, and web3 - bundle size and web3 redundancy

I need clarifications in reqards to bundle size and redundancy with web3 …
I have used truffle and continue to use it . Truffle wraps some of web3 functions for deployed contracts and their artifacts (ABI wrapper). But OZ Network, helpers and most OZ packages also have web3 as a dependency. I have converted my truffle project to use some of OZ sdks but not all. I’m a bit confused the bundle is getting heavy on load. I’m using webpack of course and, unless mistaken, it doesn’t do tree shaking.

Please provide an explanation to clear this up: is OZ a complete replacement of Truffle? What’s the most efficient approach? hint: I just use Truffle for everything except GSN.

1 Like

According to webpack docs deduplication isn’t the default it seems.
Deduplication

If you use some libraries with cool dependency trees, it may occur that some files are identical. Webpack can find these files and deduplicate them. This prevents the inclusion of duplicate code into your bundle and instead applies a copy of the function at runtime. It doesn’t affect semantics. You can enable it with:
--optimize-dedupe resp. new webpack.optimize.DedupePlugin()
This feature adds some overhead to the entry chunk.
This option is redundant in webpack 2.

1 Like

Hi @Steeve. Can you share with us what version you’re using of each of these dependencies? And also, which of these dependencies are being included in the bundle? Most OpenZeppelin packages are not something you would use in a web application. (I’m assuming you’re bundling for a web application.)

Have you analyzed in what proportion each dependency is contributing to the bundle size?

For a web application OpenZeppelin is not a Truffle replacement, because you may for example be using @truffle/contract and there is nothing in OpenZeppelin to replace that.

1 Like

For my Vue SPA thus far, my package.json looks like this:

  "dependencies": {
    "@openzeppelin/contracts-ethereum-package": "^2.4.0",
    "@openzeppelin/gsn-provider": "^0.1.6",
    "@openzeppelin/solidity-loader": "^1.4.6",
    "@openzeppelin/upgrades": "^2.5.3",
    "truffle-contract": "^3.0.3",
    "vue": "^2.5.13",
    "vue-router": "^3.0.1",
    "vuex": "^3.0.1",
    "web3": "^1.2.1"
  },
  "devDependencies": {
    "@openzeppelin/cli": "^2.5.3",
    "@openzeppelin/contracts": "^2.4.0",
    "@openzeppelin/gsn-helpers": "^0.2.3",
    "@vue/cli-plugin-babel": "^3.0.0-alpha.11",
    "@vue/cli-plugin-eslint": "^4.2.3",
    "@vue/cli-plugin-unit-jest": "^4.1.2",
    "@vue/cli-plugin-unit-mocha": "^4.2.2",
    "@vue/cli-service": "^3.0.0-alpha.11",
    "@vue/eslint-config-standard": "^5.1.0",
    "@vue/test-utils": "1.0.0-beta.31",
    "babel-eslint": "^10.0.3",
    "babel-register": "^6.22.0",
    "chai": "^4.1.2",
    "eslint": "^6.7.2",
    "eslint-plugin-import": "^2.20.1",
    "eslint-plugin-node": "^11.0.0",
    "eslint-plugin-promise": "^4.2.1",
    "eslint-plugin-standard": "^4.0.0",
    "eslint-plugin-vue": "^6.1.2",
    "node-sass": "^4.7.2",
    "sass-loader": "^6.0.6",
    "vue-template-compiler": "^2.5.13"
  },

I’m trying to clean up the package to the absolute bare essentials (and solve a persistent error when I use web3 version >1.x ). When I look into node_modules I noticed that two versions of web3 are involved in OZ packages and truffle 1.2.6 and 1.2.1. That may be trivial however and I haven’t yet optimized bundle size /pageload speed until can get web3 working again.

web console:

TypeError: Object(...).version is undefined
["./node_modules/readable-stream/lib/_stream_writable.js"]()
...

["./node_modules/web3/node_modules/web3-providers-http/src/index.js"]()

....
["./node_modules/web3/node_modules/web3-core-requestmanager/src/index.js"]()

...
["./node_modules/web3/node_modules/web3-core/src/index.js"]()
....

["./node_modules/web3/src/index.js"]()

Right. Unfortunately web3 has had issues with preserving backwards compatibility across updates, so we (like many other projects) have resorted to pinning to a specific web3 version known to work. As a result npm can't deduplicate it across multiple dependencies.

Webpack deduplication may definitely help with this, and I encourage you to try that out first. Unfortunately bumping our web3 dependency can potentially be a significant effort that we don't have the bandwidth to take on at the moment.

2 Likes

I listed all my deps using containing “web3” versions “1.2.1” and “1.2.6” with grep
eg. grep 'web3.*1.2.1' node_modules/*/package.json | cat >> tmp.txt
results

node_modules/truffle-provider/package.json:    "web3": "1.2.1"
node_modules/web3-core-helpers/package.json:        "web3-eth-iban": "1.2.1",
node_modules/web3-core-helpers/package.json:        "web3-utils": "1.2.1"
node_modules/web3-core-method/package.json:        "web3-core-helpers": "1.2.1",
node_modules/web3-core-method/package.json:        "web3-core-promievent": "1.2.1",
node_modules/web3-core-method/package.json:        "web3-core-subscriptions": "1.2.1",
node_modules/web3-core-method/package.json:        "web3-utils": "1.2.1"
node_modules/web3-core-requestmanager/package.json:        "web3-core-helpers": "1.2.1",
node_modules/web3-core-requestmanager/package.json:        "web3-providers-http": "1.2.1",
node_modules/web3-core-requestmanager/package.json:        "web3-providers-ipc": "1.2.1",
node_modules/web3-core-requestmanager/package.json:        "web3-providers-ws": "1.2.1"
node_modules/web3-core-subscriptions/package.json:        "web3-core-helpers": "1.2.1"
node_modules/web3-core/package.json:        "web3-core-helpers": "1.2.1",
node_modules/web3-core/package.json:        "web3-core-method": "1.2.1",
node_modules/web3-core/package.json:        "web3-core-requestmanager": "1.2.1",
node_modules/web3-core/package.json:        "web3-utils": "1.2.1"
node_modules/web3-eth-iban/package.json:        "web3-utils": "1.2.1"
node_modules/web3-eth-personal/package.json:        "web3-core": "1.2.1",
node_modules/web3-eth-personal/package.json:        "web3-core-helpers": "1.2.1",
node_modules/web3-eth-personal/package.json:        "web3-core-method": "1.2.1",
node_modules/web3-eth-personal/package.json:        "web3-net": "1.2.1",
node_modules/web3-eth-personal/package.json:        "web3-utils": "1.2.1"
node_modules/web3-net/package.json:        "web3-core": "1.2.1",
node_modules/web3-net/package.json:        "web3-core-method": "1.2.1",
node_modules/web3-net/package.json:        "web3-utils": "1.2.1"
node_modules/web3-providers-http/package.json:        "web3-core-helpers": "1.2.1",
node_modules/web3-providers-ipc/package.json:        "web3-core-helpers": "1.2.1"
node_modules/web3-providers-ws/package.json:        "web3-core-helpers": "1.2.1",
                 
node_modules/eth-crypto/package.json:    "web3": "1.2.6",
node_modules/web3-eth-abi/package.json:        "web3-utils": "1.2.6"
node_modules/web3-shh/package.json:        "web3-core": "1.2.6",
node_modules/web3-shh/package.json:        "web3-core-method": "1.2.6",
node_modules/web3-shh/package.json:        "web3-core-subscriptions": "1.2.6",
node_modules/web3-shh/package.json:        "web3-net": "1.2.6"
node_modules/web3/package.json:        "web3-bzz": "1.2.6",
node_modules/web3/package.json:        "web3-core": "1.2.6",
node_modules/web3/package.json:        "web3-eth": "1.2.6",
node_modules/web3/package.json:        "web3-eth-personal": "1.2.6",
node_modules/web3/package.json:        "web3-net": "1.2.6",
node_modules/web3/package.json:        "web3-shh": "1.2.6",
node_modules/web3/package.json:        "web3-utils": "1.2.6"
 grep  'web3.*1.2.1' node_modules/@openzeppelin/*/package.json                                                                                                    
node_modules/@openzeppelin/gsn-helpers/package.json:    "web3": "^1.2.1"
node_modules/@openzeppelin/gsn-provider/package.json:    "web3": "^1.2.1",
node_modules/@openzeppelin/gsn-provider/package.json:    "web3-eth-abi": "^1.2.1",
node_modules/@openzeppelin/gsn-provider/package.json:    "web3-utils": "^1.2.1"

I want to pin it to 1.2.1

However my truffle-contract is web3 0.20.6 ! so i will upgrade truffle …

1 Like

here is a more succinct representation:

bash # npm list web3
MyDapp
+-- @openzeppelin/cli@2.8.2
| +-- truffle-config@1.1.16
| | `-- truffle-provider@0.1.16
| |   +-- truffle-interface-adapter@0.2.5
| |   | `-- web3@1.2.1 
| |   `-- web3@1.2.1 
| +-- typechain-target-web3-v1@1.0.4
| | `-- web3@1.2.6  deduped
| `-- web3@1.2.2 
+-- @openzeppelin/gsn-helpers@0.2.3
| `-- web3@1.2.6  deduped
+-- @openzeppelin/gsn-provider@0.1.10
| `-- web3@1.2.6  deduped
+-- @openzeppelin/upgrades@2.8.0
| `-- web3@1.2.2 
+-- @truffle/contract@4.2.0
| +-- @truffle/interface-adapter@0.4.6
| | `-- web3@1.2.1 
| +-- ethereum-ens@0.8.0
| | `-- web3@1.2.6  deduped
| `-- web3@1.2.1 
`-- web3@1.2.6 

EDIT: I ‘sucessfully upgraded @truffle/contract (not truffle-contract which doesnt work)’ I note that ether.js uses web3 version 0.20.6 !! there must be a way to align all these critical web3 deps to ONE version ** to reduce bloat **

1 Like

You could consider using Yarn resolutions to force your dependencies to use a particular version of web3. This has the potential to break many things though.

2 Likes