Redesigning OpenZeppelin open-source processes

[spalladino]

Hey everyone! One of the things we want to tackle this year is improving our processes around our open source projects, with the goals of boosting transparency, increasing community participation, and overall making our own jobs easier.

To be consistent with the first goal, we decided to begin by having this very discussion here in the open. Feel free to chime in with your thoughts, and share any existing open source projects you like - so far we've been looking at Typescript, Rust, Kubernetes, Rails, Neovim, Crystal, and Nim, among others.

The following is a proposal on how to structure processes from now on.

---

Projects

To make management easier, smaller projects are grouped together by topic. From here on, by project we will refer to one of the following bullet points.

• Contracts
• CLI & Upgrades
• Testing: Test-Env & Test-Helpers
• Front-end: Starter Kits, Network.js, Hot-loader
• GSN: GSN Provider & GSN Helpers

Roles

Roles covered by the OpenZeppelin Development team.

• One owner responsible per project
• A second-in-command per project to assist the owner
• One or more support roles to act as first line of support and community managers

The owner is in charge of leading the project, triaging, building the roadmap, reviewing all contributions, releasing, etc. The 2nd-in-command (we need a fun/shorter name for this one!) is the owner to-go person when they need to discuss anything, is who reviews the owner's contributions by default, and who covers for the owner when they are on vacation.

Support roles answer questions, tag issues, and create new ones (based on community input), working across multiple projects. When on vacation, the owners handle these tasks.

Planning & iterations

Every 6 months, the entire team gathers to define high-level goals for the semester, and compile them into a single public document. As an example, where some items correspond to our current period, and others are made up:

Improve cohesion among our tools

We have many tools today that act as building blocks for creating apps: contracts, CLI, testing libraries, etc. However, it is not immediately clear how to easily combine them all into a single project. We should strive to build a cohesive platform by ensuring all our tools play together nicely with each other, making it easy to adopt a new one while using others.

Build comprehensive platform

Our contracts and CLI projects cover for many use-cases of smart contract developer, but not all. Build the appropriate pieces to ensure that a smart contract project can be fully developed within the OpenZeppelin set of tools, or we have a good project to recommend to our users to cover for those missing needs.

Cater for DeFi projects

DeFi is taking the Ethereum scene by storm. DeFi is possibly the vertical with the largest number of active users today. We should gain a better understanding of the problems that DeFi projects go through, and focus our tools on solving the them.

Windows support

Given that all our core devs work on Linux or Mac, we sometimes fail to properly support Windows users. We should add Windows to our CI process, and ensure that all our tools work in Windows as well as in other platforms.

Every iteration (taking a page from Basecamp) all project owners gather to plan for the upcoming period. Each owner is responsible for writing the public 6-week roadmap for their own project, syncing with the other owners. The roadmap is built and prioritized according to:

• The 6-month goals: Each individual project efforts should be aligned with the overarching goals of the open source platform.
• Bug backlog: While it is never exciting to fix bugs, it's critical to maintain a high quality for our projects. Each project owner should keep their bug count low, and prioritized based on the usage and severity of each issue.
• Internal feedback: We are developers ourselves, and are constantly using tools for developing our projects. Our own experiences should be a driver for the backlog as well. This applies to all technical teams at OpenZeppelin - both development and research.
• Audits feedback: The OpenZeppelin research and security team has a unique view of the needs of the industry in terms of security through their audits.
• Partners feedback & requests: Any feedback or requests that derive from our work with technical or business partners.
• Community general feedback: All comments and general feedback from our community and the Ethereum community at large should be taken into account. Our community manager is the to-go person to supply this information, and we can also leverage information from other projects.
• Github issues: All community requests should be channeled to github issues, where we also track how many users are interested in a particular feature.
• User interviews: We can reach out to both our open source users and our audits clients for help in better understanding their problems in personal interviews. These should help not only prioritize but also design the features in the roadmap.

Once the roadmap is defined, everyone sets out to work. New bugfixes or minor enhancements can be included in the 6-week iteration if needed. Each project is encouraged to set a housekeeping day (eg Fridays). The entire team should also gather on a weekly basis for check-ins. An example roadmap item, for the testing project:

New test-environment project

While it's possible to compile, deploy, upgrade, and interact with our CLI, it is not possible to test. While we recommend our users to work with ganache as a local blockchain, using truffle as test runner causes issues regarding configuration, and we have received internal feedback regarding its long set-up time. Aligned with our comprehensive platform goal, we shall develop a new project for running tests, structured as a library, and easy to integrate with our current offering.

After each 6-week iteration, there is a 2-week cooldown period. This time is used for retrospectives, wrap-up, housekeeping, interviewing, and (most importantly) planning for the next 6-week iteration.

Artifacts

All tasks should be handled via public Github issues, using labels, projects, and milestones. Additionally, we should have:

• One kanban github project per project (forgive the redundancy) for tracking work within that project. The flow should be: backlog, next, in-progress, and done.
• A single set of labels for all projects, including status and kind (see SDK), and project-specific topic labels if needed.
• A set of milestones per project, including the next patch, next minor, and next major (optionally).
• A pinned issue per project covering the 6-week iteration roadmap (see Typescript for an example)
• The global document with 6-month goals, linked from every roadmap (potentially here in the forum, or in a dedicated repository).
• A set of common contributing guides, linked from every repository contributing.md (either in a dedicated repo, here in the forum, or in the docsite).

Triaging community issues

Support members should catch incoming issues from the community on a daily basis (weekly max), tag them, and:

• Answer questions, escalating if needed
• Ask for further info if needed
• Reproduce bugs

They should also create issues themselves based on input via the forum, twitter, telegram, etc.

Owners should review all new issues in their repos once a week (consider using the housekeeping day for this), review their tags, and triage them:

• Add them to the corresponding milestone and project, if the issue should be included in the current iteration or the next one.
• Tag as help-needed if it's an issue they would like to see implemented, but cannot tackle now.
• Tag as low-priority if it's something they're interested in, but cannot allocate time to neither develop nor review it.
• Close if it's not something they're interested in (must feel comfortable saying no)

Handling community contributions

A similar flow applies to new pull requests by the community. These should be reviewed on a weekly basis (2 weeks max) by the owner, who should take the following approach:

• Reply whether the feature applies or not to the project, providing a quick answer to the contributor, and closing if it does not apply
• Reply whether the feature will get reviewed as part of the current milestone, or will be deferred for later - if it applies to the current milestone, try to review it within a week
• Make a first review on architectural and design decisions
• Make follow-up reviews on polishing and code-style

An issue here with Contracts is that it has a much higher barrier for contribution quality than the other projects for contributions. We should set clear community expectations that reviews for contributions not tagged as help needed can be significantly delayed.

[frangio]

Thanks for putting this together! It’s great.

First of all, I think the intent of this post should be stated more explicitly. It’s not entirely clear that it’s a draft of a proposal, given the way it’s written. It can also read as a description of the current state of things or as a definitive plan. I would add a small note after the introduction to make this clearer.

As for the proposal, I generally like that there is structure but not too much of it.

What kind of high-level goals are you thinking of for the semester? Should we expect these goals to stay relatively stable or should we aim to have new goals every semester? I’m also wondering how these goals would be affected by the projects being in a fast-moving industry.

What resources can the owners count on when planning for a 6-week iteration?

The cooldown period is a new idea to me and I really like it.

I generally dislike redundant labels, which I see often in the status category. For example, I’d say a PR should always be assumed to be in to review status, unless there is another label such as on hold. Another example is ready to merge, which IMO should be implied by there being an approving review (or whatever number is configured for the repo). I find that too many labels end up being a burden to maintain more than they’re useful.

What is the role of milestones? I find it particularly confusing that the milestones you proposed are organized around patch/minor/major. I think that a discussion about release practices/cadence should precede this choice of milestones. Perhaps you have something in mind for releases that would help me understand how the milestones would be used.

I really like the low priority tag. I was going to ask whether the low priority also extends to reviewing a potential PR that a contributor makes for such an issue. What do you think? You sort of touched on this at the end regarding delays for stuff that is not tagged as help needed. Like you said, I guess the key here is setting clear expectations. I think a low priority tag may be a good step in that direction, but I wonder if it will be enough or if contributors will implement low priority stuff and we’ll find ourselves in a situation where we will prioritize the review because of the work a contributor has already put in. It’s not really clear where we would set these expectations. Do people read contributing guides in detail?

[spalladino]

Thanks for the feedback Fran! I updated the document adding an initial clarification, examples for both the goals and roadmap, and listing the inputs for deriving the roadmap.

@frangio

What kind of high-level goals are you thinking of for the semester? Should we expect these goals to stay relatively stable or should we aim to have new goals every semester? I’m also wondering how these goals would be affected by the projects being in a fast-moving industry.

I'd expect them to change from semester to semester, though some may extend over more than a single period. What I'd like to avoid is having goals like fewer bugs. In OKRs terminology, I'm aiming for objectives, not health metrics. This reminds me: we can even enhance goals with key-results.

@frangio

I generally dislike redundant labels, which I see often in the status category. For example, I’d say a PR should always be assumed to be in to review status, unless there is another label such as on hold . Another example is ready to merge , which IMO should be implied by there being an approving review (or whatever number is configured for the repo). I find that too many labels end up being a burden to maintain more than they’re useful.

Well, I do find uses for them. A PR can be either pending review by the reviewer, or pending changes by the author. And ready-to-merge is actually a signal to mergify, to avoid having it merge stuff accidentally. But we can discuss them in depth.

@frangio

What is the role of milestones? I find it particularly confusing that the milestones you proposed are organized around patch/minor/major. I think that a discussion about release practices/cadence should precede this choice of milestones. Perhaps you have something in mind for releases that would help me understand how the milestones would be used.

I've found them useful for categorizing issues in terms of immediacy and compatibility. A bug that cannot wait for a new release should go on the next patch, a regular feature should go on the next minor, and something that is breaking compatibility should go into the next major. But I agree that they may not always be that useful, hence I flagged them as optional.

@frangio

I really like the low priority tag. I was going to ask whether the low priority also extends to reviewing a potential PR that a contributor makes for such an issue. What do you think?

Agree, it's a good idea!

@frangio

It’s not really clear where we would set these expectations. Do people read contributing guides in detail?

I'd say not everyone does, unless you direct them. These guides/expectations would then serve as canned responses when these issues arise.

[frangio]

Oh okay, on my first read I had understood that semester goals would be defined per project. Now I understand that they’d be defined by/for the entire development team. This makes much more sense.

[abcoathup]

This is awesome.
I the transparency and structure. This is great for the community.

Do all projects start the 6/2 week cycle on the same day/week? What happens during retreats, Christmas/New Year holidays, Devcon? Is the iteration longer or effort scaled based on resource availability?

For smaller projects grouped together, e.g. Starter Kits, should we look at using mono-repos to keep them together more easily for issues/PRs?

I assume that priorities are set in the high-level goals, which then decides how resources are allocated to projects.

Where do repositories like Solidity Hot Loader and Ethernaut sit in the projects?
Are there any orphan repositories, if so, what happens to them?

In the Support role, what tags are being used?
Can only members of Roles tag?

@spalladino

2nd-in-command

In other contexts I have used 2IC. Struggling for fun names: Co-pilot, wing-person, shotgun.

[spalladino]

@abcoathup

Do all projects start the 6/2 week cycle on the same day/week? What happens during retreats, Christmas/New Year holidays, Devcon? Is the iteration longer or effort scaled based on resource availability?

Yes, I'd try to have all teams in sync with regards to the cycle. Retreats or other major events simply push back the cycle for the entire team.

@abcoathup

For smaller projects grouped together, e.g. Starter Kits, should we look at using mono-repos to keep them together more easily for issues/PRs?

I'd use org-level Projects for those. The decision between mono or multi repo should be affected by the code, not the issues.

@abcoathup

Where do repositories like Solidity Hot Loader and Ethernaut sit in the projects?

I'd place hot loader as part of the Starter Kits suite (ie stuff related to front-end), I'll update the original post now. As for Ethernaut, I'd keep it as a community effort.

@abcoathup

In the Support role, what tags are being used?

Kind and topic tags for sure. Priority or help-needed tags should best be managed by the owner.

@abcoathup

Can only members of Roles tag?

I'd say any OpenZeppelin team member can tag, but the Support and Owner roles have the responsibility to do so.

@abcoathup

Struggling for fun names: Co-pilot, wing-person, shotgun

+1 for co-pilot!

[demibrener]

This is great, thanks for putting it together Palla!

I think we should add “Partners feedback/requests” to the list of inputs for roadmap prioritization. For example, Microsoft introducing OpenZeppelin Contracts into VSCode might have (or did) require some prioritization from the team side.

Also, is there a process or method for prioritizing all the input from all the different sources? Or is it more an “art” to be led by the owner?

[spalladino]

Added Partners feedback & requests to the list, thanks Demi!

@demibrener

Also, is there a process or method for prioritizing all the input from all the different sources? Or is it more an “art” to be led by the owner?

While some items carry more weight than others, I'd definitely leave this to each owner. Nevertheless, the idea is to have a round of validation for all roadmaps among the entire team before starting the cycle, so that should help align our criteria.

[jonathan]

Overall really like the general approach here to set/review high-level goals every 6 months, and then to have 6/2 iterations (though really maybe they are 2/6 iterations since after the 6-month meetup you start with the 2 for planning/shape-up?).

However rather than having the goal of the 6-month to be a published doc that cuts across all projects, I like the idea that each project (owner) publishes one doc every 6-months with the project-specific 6-month goals (like the Typescript example).

So an alternative process would be: every 6 months the steering committee (is this a formal role/group?) gets together, project owners present their proposed 6-month goals, feedback is given and steering committee makes any required high-level decisions, after which owners publish the 6 month high-level goals for each project.

[spalladino]

Interesting idea. For me the rationale for having a single set of semester goals was to 1) ensure all projects were aligned, and also to 2) have more flexibility to allocate resources among the different projects throughout the semester. And also because 3) it's less documents that we need to prepare!

But I'm not sold on this. Why would you rather go with a high-level goal per-project?

@jonathan

the steering committee (is this a formal role/group?) gets together

I'd say this is a group composed by all project owners (and yourself!).

[jonathan]

I was thinking that goals per project might be easier and faster, rather than more work (sometimes making a single doc is harder). For example the project owner can prepare 6 month goals independently prior to steering committee meeting, then all can review and discuss at steering committee, adjustments can be made and then published. Also I thought it might be nice to have the individual project goals within each project repo (although admittedly it’s minor to link) and it would be healthy for project owners to be able to express their own vision and goals in their own “voice” to each project community. Separate project goal docs might scale a little better too (again gets harder to consolidate one doc as projects grow). Definitely I think steering committee will be there to help alignment and discuss time allocations, but I think that can be done even w/ separate docs for project goals.

[spalladino]

I’m on the fence, but willing to give the one-doc-per-project a shot. I think it’d make sense to make a 6-week iteration starting now-ish, and on March start with the 6-month cycles. WDYT?

[ylv-io]

@spalladino

The 2nd-in-command

Let call him or her Sidekick!

I love transparency, the high level goals and that all products have to align with them.
Regarding to review and on hold labels, GitHub now has draft PR so unless PR is ready it should be in a draft state which helps to avoid using labels for that.

[spalladino]

@ylv-io

Let call him or her Sidekick!

Love the name!!

@ylv-io

Regarding to review and on hold labels, GitHub now has draft PR so unless PR is ready it should be in a draft state which helps to avoid using labels for that.

There's a difference though: whether a PR is draft or not is decided by the author. Sometimes the project owner may decide to hold off a PR for various reasons, and in those cases the label comes in handy. As for to review, agree that its usefulness is dubious.

[spalladino]

Sharing here the owners & sidekicks for each project, as agreed on a call:

Group	Projects	Owner	Sidekick
Contracts	contracts	@nventuro	@frangio
CLI	cli, upgrades	@frangio	@ylv-io
Testing	test-env, test-helpers	@ylv-io	@nventuro
Frontend	starter-kits, network-js, hot-loader	@ylv-io	@frangio
GSN	gsn-provider, gsn-helpers	@nventuro	@ylv-io
Documentation	site, solidity-docgen	@frangio	@nventuro

[spalladino]

Also, who do you think should own contract-loader? Right now it’s an artifact used by testing, but I’d like to see it generalized and used as part of the CLI. Should we include it in that group? Or frontend, since it should also be able to load contracts in the context of a dapp?