The solution
I noticed that while withdrawing liquidity, the most I could withdraw was exactly 1,000,000 tokens. Which happened to be the _maxTxAmount. So the issue was that I could not withdraw more than the maxTxAmount of tokens. I Fixed this by calling my contract's setTxLimit with 10,000,000,000,000,000,000 which sets _maxTxAmount to the max supply (100,000,000). Then I withdrew 100% of the liquidity without any problems. (this requires having ownership of the contract)
Notes: My wallet is both the deployer and the owner, so it should be exempt from limits. Not sure why this caused a problem. My best guess is that pancakeswap router was not exempt from txLimit and so it couldn't send me the full amount of tokens.
Also, I tried doing this once before and it didn't work. For some reason, the setTxLimit function was like this:
function setTxLimit(uint256 amount) external onlyOwner { _maxTxAmount = amount.div(100); }
Idk why it divides amount by 100. I just multiplied amount by 100 to cancel it out, and got a result of 10,000,000,000,000,000,000. Upon passing that to setTxLimit, everything was fixed.