Right now, when we use "Add Comment" section at the bottom to associate projects, it only lets you to add projects to the task. It'd be nice to let users edit projects (both adding/removing) instead of just adding projects similar to edit task interface.
The "Action" dropdown is only supposed to provide a shortcut for very common tasks.
I don't consider removing tasks that common so the default path to take is "Edit Task" in the upper right corner. That's exactly why the Edit Task interface with way more complexity is seperate from the default task view.