Category members are currently sorted in alphabetical order. A magic word such as {{NUMERICSORT}} should be introduced to make the sorting sensitive to numbers in sort keys.
For example. a page that contains
[[Category:My category|100]]
i.e. with a sort key of "100" should not be displayed after a page with a sort key of "1" and before "2", but in the proper numeric order, i.e. after any number up to and including "99".