SLanka Muslims, refugees fear attacks backlash