ProxyAdmin consistently not found despite existing

Sometimes, the truffle deployment process won't find the ProxyAdmin contract, causing a failure. The ProxyAdmin is then automatically removed from the manifest. Re-attempting the deployment will cause a new ProxyAdmin to be deployed and break upgrades for other contracts.
I've been told this could be caused by an RPC issue, but I've been trying alternate RPCs and tried at different times to no success.

Every time I add the admin part back into the file and try to re-run the migration, i get the same error message, even if i force the migration to be re-run entirely via
truffle deploy -f <migration_number> --compile-none --network <networknamehere>
Just to be clear, I manually compile the contracts ahead of time, because our build process shifts active network IDs around and the default compile behavior is inconsistent on my machine, giving errors about differing compile artifacts.

The error message states there is no contract on the address, even though it exists, we've been using it for 500+ days on this particular network.
I'm completely out of ideas on how to remedy this.

Our codebase is open

The main branch is generally where we deploy from.

The file that has the ProxyAdmin truncated from the error is here

It's this bit at the top

"admin": {
    "address": "0xe9578Cc9c7883FABb6485702b51f5aE5A0105750",
    "txHash": "0x5d56d886af0901fd398154dda8df715b117787ab39a77cd0044cc9416f7160b8"
  },

As you can see it exists on the Binance testnet:

Here's the migration file in question

This previous migration had also failed on one of our mainnets: (the network is stuck being outdated for now)

Our truffle config is here:

The definition of the RPC URL for the network: (i don't have a custom url specified in the env file)
providerOrUrl: process.env.BINANCE_TESTNET_RPC_URL || 'https://data-seed-prebsc-2-s2.binance.org:8545/'

Binance has 6 public testnet RPCs listed here

Of which 5 have been throwing me timeout errors recently, this one is working for me right now:
https://data-seed-prebsc-1-s3.binance.org:8545/
(i can transact just fine, but the deployments give the above error)

This isn't specific to Binance testnet, we've had it happen on another network, and there's 16 other networks that haven't had any such issue.
Never happened on a local Ganache network.

:1234: Code to reproduce

npm run deploy-testnet

:computer: Environment

Running Windows
Truffle v5.2.3 (core: 5.2.3)
Node v13.14.0

Some dependencies i thought most relevant:
"@openzeppelin/contracts": "^3.4.1",
"@openzeppelin/contracts-upgradeable": "^3.4.1",
"@truffle/hdwallet-provider": "^1.5.0",

"@openzeppelin/hardhat-upgrades": "^1.10.0",
"@openzeppelin/test-helpers": "^0.5.11",
"@openzeppelin/truffle-upgrades": "^1.8.0",

Full list of dependencies:

I'd be happy to provide any necessary information to get this working again.
Thanks in advance!

I have investigated this issue further.
I think I've figured it out but I am desperate to have someone confirm the validity of this.

According to this stackoverflow thread,

web3.eth.getTransaction may return null if the node has chosen not to index all transactions beyond a certain point.

I tested this in truffle console, running this
await web3.eth.getTransaction("0x5d56d886af0901fd398154dda8df715b117787ab39a77cd0044cc9416f7160b8")
returned null, but running this
await web3.eth.getTransactionFromBlock(9207968,0)
actually DID return the transaction.

So this leads me to believe, I need to access a node that has this old transaction indexed by the hash, because openzeppelin's deployment script calls eth_getTransactionByHash through the web3 provider.
What's troubling is, this issue seemed to pop up both on BSC testnet and one of our mainnets simultaneously.

It makes me wonder, maybe the manifest file should store the block number and the transaction index, and the script to call it by block number to avoid this situation?

Either way, I've got a lead to research for now, but please, if anyone could provide further info, even just to confirm what I've ascertained is correct, that would be a huge help!

Your assessment seems to be correct. We were assuming that getTransactionByHash would always return the transaction. We've opened an issue to fix this.