Not too big, Not too small: the hunt for intermediate mass black holes