Flash drives can be written to a (semi-random) limited number of times.
Defragging a flash drive would use up several of those writes.

I find I format any flash drive from time to time, which means I'm starting from
an unfragmented state.