Barnaby Joyce apologises to family, partner and voters