Best practices to handle Transaction signing on Backend servers & deal properly with Private Keys

Hey guys

We are currently thinking a lot of how to deal with transaction signing on backend applications, that are running on a server.
As an example:

  1. User buys token via Fiat Gateway payment and backend will send out token automatically.
  2. A specific event that occured on the backend is triggering some data-storage functions on a smart contract automatically

We’re thinking about either

  • using HSM
  • or storing private keys in memory (memcached for instance) for a while, that constantely needs to be recreated and signed from Staff within a period
  • different roles and permissions on the smart contract
  • further rules how many transactions a key can sign within the contract

and all these things.

Any experiences out there to share and discuss?

3 Likes

Hi @itinance,

I have asked this question in a few different communities, so will see what response we get.

Tagging @rstormsf, @Dennison_Bertram, @lukaslukac who may have experience with off chain signing (e.g. using the Gas Station Network) and @bbarton (with Peepeth).

2 Likes

Dillon (on CryptoDevs) suggested:

what if you used secret sharing to require 2 servers to sign the tx without needing to do a double verification? just in case one gets compromised

hsm probably a better idea
https://freedom-to-tinker.com/2015/03/08/threshold-signatures-for-bitcoin-wallets-are-finally-here/

I do think HSM is a better idea (maybe 2 hsms with secret sharing if you really wanna be fancy lol)

1 Like

BatsForCats on r/EthDev:

Depends on their hosting environment, but they likely want to store the key in an Hardware Security module, or encrypted at rest key store and create a workflow where as needed the key is pulled into an execution environment where the txn is passed securely through either a queueing system or database lookup.

And the environment is destroyed every time it has to sign something.

1 Like

Clarifying point: I’ve never setup a backend service for ethereum, but I do have experience working with storage of keys in self sovereign identity.

In answering this, I’m going to make a few assumptions here that may not be true. Please correct them if they’re wrong.
1.) the keys used to sign transactions are for the owner of the backend server.
2.) the transaction signing needs to be automated so the backend server can sign at anytime. For example, a person isn’t required every time a transaction needs to be signed.
3.) you’re looking to pass the management of the keys off as long as you know it’s secure.

You could use a library to perform signing for you and store the keys in an encrypted keystore that shares memory with other applications. This is usually a good enough for things like wallets on smart phones, but in this case I wouldn’t recommend it. If the backend server is hacked and the attacker gets access to the file where keys are stored they’ll be able to steal them. I’d suggest going the HSM route if possible.

HSMs are typically the best options when handling an individual key. With that said, you may have problems signing transactions with an HSM because it won’t be able to support ethereum account key formats I suspect. This may not be a problem if you want to rebuild APIs in which case Azure Keyvaults now support Secp256k1 keys and I had heard AWS now has support as well. However, I’m not familiar enough with the web3.js sign API to say off the top of my head what the signature process is. In any case if at all possible it’s recommended that you use an HSM.

Hopefully that’s helpful to get you going at least.

3 Likes

What about Hashicorp’s vault?
https://www.vaultproject.io

You can set the private keys as secrets over there, setup auth policy for your server and have it retrieve the keys.

Limiting the “power” of such keys to certain roles in the smart contracts could be useful to limit the impact in case of compromise, plus you can implement regular key rotation.

Which HSM are you thing about?

3 Likes

s1gmoid on r/EthDev:

Okay so what you need is colloquially called a crypto enclave. The point is to have an environment that is completely separate from the rest of your Intranet, with very strictly controlled I/O toward the rest of the world (and the rest of your network).

In an enterprise situation, I’d say the enclave ideally consists of at least an isolated, hardened server and a firewall server running completely separate from this server. Preferably on separate hardware to ensure that low level hardware vulnerabilities won’t allow key exposure. Both in a physically secure location (locked room separate from the regular server room, safebox, etc.) If your private key is in the cloud, it’s no longer your private key.

As others said, using a hardware security module is preferred. I’d say, using it inside the above-described crypto enclave is what’s preferred. HSM inside physically isolated server inside physically isolated network segment.

Now this may sound like overkill, but if you’re managing a lot of money, thieves will be coming after you, and you might as well prepate for sophisticated ones. Make sure the enclave is duly protected from social engineering attacks. No keys in receptionists’ hands to allow an unscheduled fire safety checkup into the room, etc.


Also, implementing a granular onchain authorization scheme is a good idea. The master key should be offline, on a piece of paper or a good quality hardware wallet, in a safe. Where only the CEO personally (or other high level stakeholder with personal liability) has access.

Then, allow the master key to authorize signing keys at a smart contract level, each with specific limitations according to the role they play (eg. allow to pay out tokens, but with a daily amount cap). If a client wants to buy a billion dollars’ worth of tokens, set up a personal meeting, have the CEO ceremonially retrieve the master key and authorize the transaction, etc. Once you get to the high flying level, it’s okay to not be automated. If you are suddenly flooded with legitimate business beyond your wildest dreams, get the master key and raise the cap. If your signing key is compromised, all you lose is the daily cap at most, and you can run get the master key, de-authorize the compromised key, and set up a new one.

2 Likes

We are facing the same troubles defining security in our backend. I think this is an anti-pattern tbh, as there should be no such thing as a backend in a blockchain environment, but whatever.

Our current solution is headed toward a multisig smart contract with a “ring window”. For the smart contract to operate, there are 1+N keys: 1 master key that is stored in cold, and N keys (let’s say 3 keys, and a window of 2, for simplicity). The contract is operated in turns: lets say key 1 and key 2 validate a transaction, then next transaction must be composed by key 2 and 3, and next by 3 and 1, and so on. In order for the ring to keep going, a nonce is kept, so that if any anomaly occurs, the ring will stop moving as the backend applications will halt. As a last stand, the master key can revoke the authorization on the keys and stop the contract temporarily. Adjust number of keys and the size of the window to your needs.

I 'd say this is the simplest way to go.

2 Likes

Hi @madcake,

Welcome to the community :wave:. Thanks for sharing.

Are you using an existing open source multi-sig or are you developing your own for the “ring window” functionality?