The key here is to press the button on the handle first to let the car know to stop charging and release the locking mechanism. Then release the button so that your J1772 adapter is still locked to the handle. After one second, you can hear or see from the car's charging indicator that the lock is released, now pull the handle without touching the button. You'll easily pull your adapter out together with the handle. Finally don't forget to remove the adapter. All this should work as long as you have the phone key with you. Otherwise you'll need to unlock the car first.
