How are transactions handled on the backend?

Support Defender
1

When the SDK hits the relay endpoint, how is it ensured that large volumes of transactions are processed by the single relayer and no transaction fails or drops?

2

Hey @Yashasvi_Chaudhary,

We have a complex system behind relayers, that takes cares about everything from averaging oracles, repricing, cancelling when validUntil is hit, tracking transaction status, etc. This system is also designed to scale accordingly.

how is it ensured that large volumes of transactions are processed by the single relayer?

We can guarantee large amount of transactions going through a Relayer because every execution is isolated. However, recent issues we've had have been problems with some node providers and network availability.

... and no transaction fails or drops?

We indeed have a FAILED status, so a transaction can fail but it's not expected to happen. Here's what can happen:

  • A transaction may hit a validUntil set by you. In this case the transaction gets replaced by a NOOP
  • If you set a validUntil really high, we'll still keep resubmitting the transaction until it hits a certain threshold of retries. This is when we flag it as FAILED because the transaction was simply not accepted by the nodes. Shouldn't happen under normal conditions
  • If we sent a transaction that for some reason a node didn't broadcasted and just dropped, we have a monitoring schedule every minute checking its status, and if needed, we resend it. We also send each transaction to multiple providers.

Curious, are you looking for a specific volume? I can set you in contact with someone from the team to discuss an increase in your account capacity.

Hope this helps!

3

Thanks for the reply @ernestognw.
Yes this is helpful. I have two follow up questions:

  1. I see there is a limit of 10 transactions per second when I spin up a relayer. What happens when we excede this limit? Do the transactions not go through the relayer? Does it autoscale?
  2. When we increase the account capacity is it still only one relayer relaying the high traffic or now there is a multi relayer setup that comes in the picture?
4

No problem @Yashasvi_Chaudhary, happy to help.

  1. The limits set to a relayer are hard limits we set based on your quotas. If you upgrade your plan, we remove those limits and the system can autoscale. However, we'd like to know what is your expected throughput so we can stress test before committing. There are also some limitations to consider such as node provider limitations (eg. we've had problems with Arbitrum in some circumstances where the node providers start to answer incorrectly for small periods of time, filling the tx queue on our end. We're still working on that).
  2. When you increase your plan, it's increased for the whole account, this means that if you have 10,000 transactions per hour, that includes every relayer in your account. It's basically a global counter.

Let me know if you have further questions

5

Thanks @ernestognw
In the starting tier there is a single relayer that we can attach to our application that would relay the transactions, right? If I understand this correctly, you are saying in higher tiers our application can use a multi relayer system? Or do I have to manually set up multiple relayers from the dashboard and attach it to my application?

1 Like
6

In the starting tier there is a single relayer that we can attach to our application that would relay the transactions, right?

There is a limit of 5 relayers and 30 tx/hr on the free tier. You can either setup a single relayer using the 30 tx/hr, or 3 relayers using 10 tx/hr, but I would only recommend splitting the load if your relayer have actual multiple use cases, making multiple relayers just to replace each other is equivalent in terms of how we process transactions on our end.

If I understand this correctly, you are saying in higher tiers our application can use a multi relayer system?

Yep, you can setup multi relayers since the free tier up to 5 relayers, but...

Or do I have to manually set up multiple relayers from the dashboard and attach it to my application?

Also yes, they have to be set up manually.

Just for clarity, we refer to a relayer by the entity you can see on your Defender dashboard. Under the hood, the resubmission and monitoring system is the same for all of them.