Indeed this is caused because of the PUSH0 opcode. It comes from EIP-3855 and was included in the Shanghai upgrade.
Consider that not all chains support PUSH0 at the moment, so compiling using the Shanghai EVM version will error for those chains. The workaround is specifying an older EVM version (eg. Paris) so that the compilation doesn't produce bytecode including the PUSH0 instruction. This can be done in the following menu in Remix (compiler -> advanced configuration):
The option default is usually the latest (Shanghai). Just select Paris and you'd be fine.