Incorporating `solidity-docgen` into your project

I would like to share how we at Bancor have incorporated solidity-docgen in order to auto-generate our official documentation.

The full process consists of the following steps:

  1. Using solidity-docgen in order to generate a bunch of MarkDown files from the Natspec documentation in our Solidity contracts
  2. Rearranging the output of the previous step into the structure dictated by GitBook’s content-configuration

It assumes the following:

  1. Your entire project resides in a folder next to file package.json
  2. All contracts reside in a folder named contracts, which resides under your project’s folder

Here are the technical details:


Step 1:
Add this in file package.json:

  "scripts": {
    "docify": "node docify.js"
  },
  "dependencies": {
    "solidity-docgen": "0.3.11"
  }

Step 2:
Assuming that your entire project resides under folder project next to file package.json, add this in file docify.js next to file package.json:

const NODE_DIR     = "node_modules";
const INPUT_DIR    = "project/contracts";
const CONFIG_DIR   = "project/docgen";
const OUTPUT_DIR   = "project/docgen/docs";
const README_FILE  = "project/docgen/README.md";
const SUMMARY_FILE = "project/docgen/SUMMARY.md";
const EXCLUDE_FILE = "project/docgen/exclude.txt";

const fs        = require("fs");
const path      = require("path");
const spawnSync = require("child_process").spawnSync;

const excludeList  = lines(EXCLUDE_FILE).map(line => INPUT_DIR + "/" + line);
const relativePath = path.relative(path.dirname(SUMMARY_FILE), OUTPUT_DIR);

function lines(pathName) {
    return fs.readFileSync(pathName, {encoding: "utf8"}).split("\r").join("").split("\n");
}

function scan(pathName, indentation) {
    if (!excludeList.includes(pathName)) {
        if (fs.lstatSync(pathName).isDirectory()) {
            fs.appendFileSync(SUMMARY_FILE, indentation + "* " + path.basename(pathName) + "\n");
            for (const fileName of fs.readdirSync(pathName))
                scan(pathName + "/" + fileName, indentation + "  ");
        }
        else if (pathName.endsWith(".sol")) {
            const text = path.basename(pathName).slice(0, -4);
            const link = pathName.slice(INPUT_DIR.length, -4);
            fs.appendFileSync(SUMMARY_FILE, indentation + "* [" + text + "](" + relativePath + link + ".md)\n");
        }
    }
}

function fix(pathName) {
    if (fs.lstatSync(pathName).isDirectory()) {
        for (const fileName of fs.readdirSync(pathName))
            fix(pathName + "/" + fileName);
    }
    else if (pathName.endsWith(".md")) {
        fs.writeFileSync(pathName, lines(pathName).filter(line => line.trim().length > 0).join("\n\n") + "\n");
    }
}

fs.writeFileSync (SUMMARY_FILE, "# Summary\n");
fs.writeFileSync (".gitbook.yaml", "root: ./\n");
fs.appendFileSync(".gitbook.yaml", "structure:\n");
fs.appendFileSync(".gitbook.yaml", "  readme: " + README_FILE + "\n");
fs.appendFileSync(".gitbook.yaml", "  summary: " + SUMMARY_FILE + "\n");

scan(INPUT_DIR, "");

const args = [
    NODE_DIR + "/solidity-docgen/dist/cli.js",
    "--input="         + INPUT_DIR,
    "--output="        + OUTPUT_DIR,
    "--templates="     + CONFIG_DIR,
    "--solc-module="   + NODE_DIR + "/truffle/node_modules/solc",
    "--solc-settings=" + JSON.stringify({optimizer: {enabled: true, runs: 200}}),
    "--contract-pages"
];

const result = spawnSync("node", args, {stdio: ["inherit", "inherit", "pipe"]});
if (result.stderr.length > 0)
    throw new Error(result.stderr);

fix(OUTPUT_DIR);

Step 3:
Inside folder project, create folder docgen with files exclude.txt and contract.hbs.
File exclude.txt should contain a simple list of all files and folders which you want to exclude from your documentation. Since all of these files and folders reside under folder contracts, there is no need to specify it as part of the path of each file or folder.
File contract.hbs should contain a set of instructions for the desired structure of your documentation. These instructions should be written using the HandleBars language syntax, for example:

{{{natspec.devdoc}}}

{{#if ownFunctions}}
# Functions:
{{#ownFunctions}}
{{#if (or (eq visibility "public") (eq visibility "external"))}}
- [`{{name}}({{args}})`](#{{anchor}})
{{/if}}
{{/ownFunctions}}
{{/if}}

{{#if ownEvents}}
# Events:
{{#ownEvents}}
- [`{{name}}({{args}})`](#{{anchor}})
{{/ownEvents}}
{{/if}}

{{#ownFunctions}}
{{#if (or (eq visibility "public") (eq visibility "external"))}}
# Function `{{name}}({{args}}){{#if outputs}} → {{outputs}}{{/if}}` {#{{anchor~}} }
{{#if natspec.devdoc}}{{natspec.devdoc}}{{else}}No description{{/if}}
{{#if natspec.params}}
## Parameters:
{{#natspec.params}}
- `{{param}}`: {{description}}
{{/natspec.params}}
{{/if}}
{{#if natspec.returns}}
## Return Values:
{{#natspec.returns}}
- {{param}} {{description}}
{{/natspec.returns}}
{{/if}}
{{/if}}
{{/ownFunctions}}

{{#ownEvents}}
# Event `{{name}}({{args}})` {#{{anchor~}} }
{{#if natspec.devdoc}}{{natspec.devdoc}}{{else}}No description{{/if}}
{{#if natspec.params}}
## Parameters:
{{#natspec.params}}
- `{{param}}`: {{description}}
{{/natspec.params}}
{{/if}}
{{/ownEvents}}

Following that, you (and anyone else seeking to do so) can auto-generate the docs via npm run docify.
Then you can publish your project’s documentation on GitBook simply by registering your repository for this service.
Note that you will still need to add all the auto-generated files (.gitbook.yaml, project/docgen/SUMMARY.md and everything under project/docgen/docs) to your GitHub repository.

Thank you OZ!!!

4 Likes

Hi @barakman,

Thank you for this fantastic guide, and all your testing, feedback and contributions to improve solidity-docgen. It is greatly appreciated. :pray:

@barakman

I’ve made a few changes to your script to generalize it a bit.

Is now

  '--solc-module=' + NODE_DIR + '/solc',
  '--solc-settings=' +
    JSON.stringify({
      remappings: ['@openzeppelin/=./node_modules/@openzeppelin/'],
      optimizer: { enabled: true, runs: 200 },
    }),

I had to use the default solc, because I am just using oz without truffle and the truffle version was 5.0.0 so it didn’t meet the minimum requirements.

The second, I had a compilation error if I didn’t add the openzeppelin contracts remapping. This might have been caused because I was using the truffle solc, and my project wasn’t using truffle to compile.

Great work!

4 Likes

@Emiller88: Thank you, glad to know you’ve found it useful.

At the time of posting this guide, I forgot that my script had been designed for a Truffle-structured project (though this fact is implicitly implied at the top of my post - “All contracts reside in a folder named contracts, which resides under your project’s folder”).

So yes, it does indeed configure --solc-module to the compiler used by Truffle, under the assumption that your project relies on Truffle.

I suppose that the general solution would be to place this path alongside all those paths at the top of the script (which I have placed there in order to make it clear that these are the constants which every user should configure specifically for his/her project).

Since we do not have any “external contract” dependency (such as openzeppelin contracts in your example), I did not face any issue which required adding remappings. Glad you were able to sort this out yourself, because as I recall, solc has some issues with contracts which reside outside of the project’s directory (resulting with a File outside of allowed directories error).

Thanks