Definitely be careful pulling out! I'm with Jimmy on this. I actually had one wheel snap, so the outer rim broke off, the inner bore was still wrapped around the axle! Never again will I do that. I don't glue the axle in permanently now, just enough to hold it's position, that way if there is a problem I can yank it out with no damage. If you can use axle pliers, meaning your wheel gaps aren't too tight, I'd go that route, just be careful not to scratch anything up! Hold it firm so it won't slide and scratch the axle, but not too firm so it won't gouge. I have never had a problem, but sure it can happen. Don't just pull out, twist back and forth first small increments until it breaks. Then pull it out.
I don't think Ballistic is saying insert a toothpick and leave it inserted. I think he is saying put a dot of glue on the end of the toothpick, insert it so the glue goes into the axle hole, then remove the toothpick once the glue has been transferred. That is actually brilliant!
In cases that the gap is too tight and I absolutely can't get my pliers in and know I may break the wheel, I have removed them with the Derby Worx Pro Axle Guide. I sanded the actual gauge part so it goes from thin at the top, to thick at the bottom...with me so far? So where you insert it into the wheel gap, it's thin. I insert it into the wheel gap, and gently apply pressure. As it gets thicker, it should move the wheel out of the hole with no damage. Be careful though! Don't want to damage the wheel! If you don't get it behind the wheel, but on top of it, you are going to kill the plastic of the wheel around where it touches the wood and destroy it.
I imagine you could use other things to do this with, just need a flat thin piece of metal with a axle slot cut into it shaved thin on one end, wedging up to thick. I used a Derby Worx Pro Axle Guile because honestly, it felt good to actually find a good use for it! I've yet to damage a wheel with it.
Name of the game, is be careful with all these methods.
Play around and find something that works for you! Don't be scared to experiment. You most likely aren't going to mess anything up, and if you do, it is fixable. When I broke the wheel removing it, I simply took a wheel of the same color, prepped it, and put it on the non dominant side where it's not used, just floats. Three good wheels still, even though one wasn't AS good (They were lightly lathed and trued). Worked great. There is always a solution and finding it is half of the fun.