Time Travel Isn't Great For Everyone