As popup groups can be moved left or right depending on the their position within the bounding window, it's even more difficult to tell which button they belong to (see examples). Every UI ever highlights the triggering button when the popup opens, and while I appreciate trying to do something different, in this case there's a very good reason for it.
Apex: