Learning to Let Go of the Past While Decluttering