Surprisingly, the answer is no. Fabric conditioner coats the fibres of your towel and actually reduces your towels' absorbency. A build-up of fabric conditioner over time will eventually make your towels feel hard and scratchy - which is the opposite to the effect you want to achieve. It sounds counter-intuitive, but for the softest towels, wash with detergent only.
Tumble drying your towel is a lovely way to open up the cotton fibres so they're wonderfully soft and absorbent.