Until I can go for 6 months without a USB memory stick forgetting its capacity or contents, I'll insist on a safe unmount procedure.
A stick that I 'just pull out' of a Linux box will be interrogated on other operating systems when inserted and disk check advised - so the sign of the unsafe mounting event IS carried with the stick.
I have 2x 32G memory sticks dismantled in my junk pile, and an 8G stick that thought it was 4meg, until repaired using HP tools, then reformatted. They all were damaged on removal/insertion (without eject or unmount prep by user), or at some stage in between - never while in operation.
Maybe hardware running Linux is immune? Anyways, rest assured, these portable memory storage media CAN be broken.