Retain sockets until after removal from observer

Not retaining them caused kevent() to be called on an invalid fd.