Turbo winding down after spinning flat out, takes a lot longer in a diesel as there is no throttle body to shut and blow off valve to release the pressure. In a diesel, when you back off their is still pressure in the intake piping that needs to make its way through. No mufflers helps you hear it, more boost makes it more noticeable?
That's my guess.