In ve.ce.Surface.prototype.annotationsAtModelSelection we move the cursor one to the right, and use getNodeAndOffset to find the inner annotation, then traverse the DOM to find all surrounding annotations.
The technique fails when the annotation is one character wide, and is generally quite hacky.
We can easily gather annotation models just by looking at the linear data, so there should be a way to do this for the view.