This is a common occurrence with collet-type holders in various machines. Collets are excellent at holding tooling accurately and securely, however due to the collet being forced into the taper of the collet holder, the collet often doesn't easily release the tool easily. This is remedied by, after unscrewing the collet holder, tapping the tool bit back INTO the collet with a soft-face hammer or with a piece of wood held over the front the tool bit so as not to damage it. This shoves the bit and collet AWAY from the taper of the collet holder, releasing the bit.
The best way to easily accomplish this is by NOT inserting the tool bit all the way to the bottom of the collet, thus leaving some room to tap the tool bit back into the collet when removing it.
Planeman