Hi @EmkoK,
Welcome to the community 
I have verified the proxy and ProxyAdmin contracts on several public networks. I will go through and verify the remaining public testnets.
I am using the following method using Buidler Etherscan plugin: Verify Upgrades Plugins proxy on Etherscan
@Skyge is right (Thank you 
), the proxy and ProxyAdmin contracts are here: https://github.com/OpenZeppelin/openzeppelin-upgrades/tree/master/packages/core/contracts/proxy
The contracts are precompiled with Solidity 0.6.8 and optimization disabled.
Though it looks like a community member has verified a flattened version (my preference these days is multi-file verification).