The "low-hanging fruit" or washing machine energy usage is water heating. Washing with cold water (as in the delicate mode) is by far the easiest step you can take toward sustainable clothes-washing.
But does it work?
I'm no expert on clothes, but I do wear and wash them and I've found that washing with cold water works fine even for sweaty athletic clothes or slightly dirty hiking clothes. The delicate cycle on most washing machines uses cold water, and it will clean anything you wear around town, to the gym, or to work (unless you wear suits or work with oily and greasy stuff). You can usually get very muddy clothes down to an acceptable level of dirt by allowing the mud to dry and shaking it off or by doing a quick pre-wash by hand in a sink or bathtub. Also, cold water doesn't make dyes bleed the way hot water does, so you can usually mix dark and light-colored clothes. Combining loads when possible is an easy way to be more sustainable.
So yes, it is more sustainable to wash on the delicate cycle and your clothes will still be clean when the machine finishes.