‘Constant grinding weakness’: Albanese government losing voters