Blazing Fast Contracts Testing

Blazing Fast Contracts Testing

Two things are unavoidable in the life of a software developer – death and writing tests. Certainly, due to advances in technology, it soon might be possible to escape death, but tests are here to stay. While developers recognize the value and importance of tests, not many find it easy to write them.

In the Ethereum world, having a full test suite for smart contracts is an absolute necessity, because a critical bug may lead to funds lost forever.

The famous OpenZeppelin Contracts library has 2273 tests, and it takes 21 minutes to run them. When it takes so long to run a test suite, it becomes a developer experience issue. Even if you want to run only one test in your entire suite, bootstrap time can be as high as 30 seconds, sabotaging your productivity on every run. Forced breaks during development caused by long tests interrupt the flow and decrease productivity.

Not only does a test library need to be fast but the developer experience is crucial for creating and maintaining a comprehensive test suite. If you don’t enjoy writing tests, you’ll inevitably fail to maintain them.

We’ve spent 3 years and countless hours writing tests for the OpenZeppelin Contracts library. We are excited to present the result of what we have learned, our best practices, and the use of the best industry tools – OpenZeppelin Test Environment! This library provides blazing fast smart contracts testing with a developer experience we actually enjoy on a daily basis. We use it to power tests for Contracts and SDK.


Test Environment and Truffle Test Script run side by side with 4x speed

We have managed to drop our Contracts tests suite CI time from 21 minutes to 12 minutes and bootstrap time from 50 seconds to 5 seconds.

Test Environment is a library, not a framework. Unlike other frameworks that manage the entire testing flow, you are free to use any test runner such as Mocha, Jest, or Ava. You decide when and how to run it. You can use either @truffle/contract or web3-eth-contract contract abstraction. We now have a highly configurable and agnostic library to match our needs.

Compile, test, and deploy smart contracts with OpenZeppelin. This is the last piece of the OpenZeppelin development stack. OpenZeppelin Contracts provides the building blocks for writing your contracts, while the CLI allows you to create, upgrade, and interact with them. Test Environment is the last piece of the puzzle, allowing you to test your contracts.

Try it right now!

Install

npm install --save-dev @openzeppelin/test-environment

Usage

Simply require the @openzeppelin/test-environment package in your test files, and it will take care of all Ethereum-related tasks. A local ganache-powered blockchain with unlocked accounts will be spun up, and all tools will be configured to work with it.

For example, integration with Mocha is super easy, and Test Environment also plays nicely with the assertions from the Test Helpers library.

With Mocha

const { accounts, contract, web3 } = require('@openzeppelin/test-environment');
const { BN, balance, ether } = require('@openzeppelin/test-helpers');
const { expect } = require('chai');

const GLDToken = contract.fromArtifact('GLDToken');

describe('GLDToken', function() {
  const [sender, receiver] = accounts;
  const initialSupply = new BN(100);

  beforeEach(async function() {
    this.token = await GLDToken.new(initialSupply);
  });

  describe('total supply', function() {
    it('returns the total amount of tokens', async function() {
      expect(await this.token.totalSupply()).to.be.bignumber.equal(initialSupply);
    });
  });

  it('sends ether successfully between accounts', async function() {
    const tracker = await balance.tracker(receiver, 'ether');
    await web3.eth.sendTransaction({ from: sender, to: receiver, value: ether('10') });
    expect(await tracker.delta()).be.bignumber.equal('10');
  });
});

As you can see, writing tests with Test Environment looks and feels like a familiar Mocha test experience.

What about real production level examples?
We migrated all the tests at OpenZeppelin Contracts and SDK to use Test Environment. Check them out to see the full potential of Test Environment!

8 Likes

Amazing! Is there a way to use this with the ethers.js Contract API (instead of @truffle/contract or web3-eth-contract) ?

3 Likes

Hey @gitpusha! It is definitely possible. test environment expose web3 and provider which can be used to create any contract abstractions. Do you have an open source example of a project which use tests with ethers.js? I would love to check them out.
Regarding test environment supporting ethers.js directly, I’d say create an issue at repo. It is an open source project after all!

3 Likes

@gitpusha Agreed that adding ethers.js-style contract abstraction is the most important missing feature!

4 Likes

I just created an issue for this like @ylv-io suggested.

3 Likes

This looks awesome! Amazing how much you were able to decrease the runtime.

Put me down as another one looking forward to ethers.js abstractions :slight_smile:

I asked this on Twitter, but figured you might want to move the discussion here: Would it be possible for the local ganache blockchain be spun up as a fork of the mainnet? E.g. ganache-cli -d -f https://mainnet.infura.io/v3/yourInfuraProjectId -u "0xBE0eB53F46cd790Cd13851d5EFf43D12404d33E8"

Since we build on top of a lot of other existing contracts, I find that developing/testing on a forked mainnet is by far the easiest approach. Eliminates the need to deploy the various contracts locally and seed them with a realistic state.

4 Likes

This is awesome! Kudos.

Last time I tried to use @openzeppelin/test-helpers, it broke compatibility with bignumber.js (due to some strange reason I am still not aware of).

I suppose this issue shouldn’t exist when used alongside @openzeppelin/test-environment, but is this a fact? Has anyone successfully used these libraries with bignumber.js in their tests?

Also, add me to the list for the integration with ethers.js!

2 Likes

Thanks! We are very happy with the result for test-environment, using it makes testing enjoyable again :slight_smile:

The library is bignumber agnostic, so you shouldn’t run into any issues related to this. The reason why test-helpers includes support for chai-bn and uses BN.js instead of bignumber.js is that web3’s 1.0 release removed support for bignumber, favoring BN instead.

I’m personally not a fan of BN and prefer bignumber (you can read some of the reasons why here), but we think interoperability trumps all in these cases.

I’m curious, why is it that you need to use bignumber? I understanding not wanting to migrate (doing it on the OpenZeppelin Contracts repository was a pain), but are there any other reasons? Is your workflow completely independent of web3?

Forking is not something we currently support, but we do plan on enabling these uses cases soon. Most likely, we’ll let you provide an URL to a node and a list of named unlocked accounts, and then export those back to you (so you’d get both new accounts and the ones you requested).

Would you mind sharing what other contracts and system you find yourself building on top on and testing these integrations?

3 Likes

That would be awesome, since for me the forking feature is pretty essential. It makes things so simple that I’m surprised this approach doesn’t seem to be more popular.

Right now we are essentially a one step fiat to Compound on-ramp (though we’re adding more functionality soon) with Wyre as the payment processor. So when testing the full flow of purchasing Dai from Wyre > sending it to a contract > minting cDAI, forking from the mainnet effectively removed all setup/configuration of Maker and Compound contracts.

4 Likes

Hi @ylv-io - sorry for not creating an issue and thanks @pcowgill for doing your part on that!
Regarding testing with ethers.js there is a whole framework dedicated to that called waffle: https://github.com/EthWorks/Waffle

@ylv-io: you can find ethers.js style tests in there, albeit using waffle chai syntax.

Also buidler supports testing with ethers.

@msolomon4 thanks for raising the --fork question. I for one also really need that one @nventuro.

Sorry for raising all of this in a comment and not opening issues or PRs. Please only take this as gentle feedback, not more not less. You guys have done so much for this space already. Wish I would have more time to contribute directly.

2 Likes

@gitpusha, I am aware of Waffle. I even played with it few months ago. I am interested in projects with substantial tests suite using ethers.js. To answer my own question. I found Universal Login SDK using Waffle/Ethers.js combination for tests. Would appreciate more examples.

2 Likes

ok sorry, I misunderstood. Thank you for providing this example, I will report back should I find one too.

2 Likes

A post was split to a new topic: Does OpenZeppelin Test Environment make tests faster to write?