Go small and medium for singletrack, large is better for dirt roads or winter trips.
The rule of thumb is to downsize your load on handlebars as much as possible for the best handling. Bigger is not better; choose the smallest bag you can get away with. Medium is the most versatile size.
Small - 10L volume, 14cm (5.5") diameter x 57.5cm max width.
Medium - 14L volume, 17.5cm (7") diameter x 55cm max width.
Large - 18L volume, 20cm (8") diameter x 55cm max width.