As with many of the -r remove options, you must provide the exact same address matching the one you used to add them to the list.
So if you added them with /aop nick #yourchannel 3
Then you must use /aop -r nick #yourchannel 3
to remove that same address, otherwise it does not find a matching address to remove.