I had heard this too, and am interested and agree with what most of the comments on here have said so far. Here are a couple observations from someone who uses my dado set a lot.
1. Riving knives and blade guards go out the door pretty much. (+1 PurpLev)
2. Removing more material in one swipe can be dangerous if done improperly.
3. Heavier blades result in more wear-and-tear on your TS motor, especially if you take deep cuts.
4. My saw (Ridgid R4511) at one point had a recall because dado sets were being spun off the arbor and being thrown around the shop. Mine wasn't an affected serial number, but it certainly gave me pause for a moment the first time I spun a dado after reading that. Not really a problem with dadoes in particular, but rather with the saw itself I guess.
5. The older model dado blades (that oscillate as they spin, don't remember the name) freak me out. Not for any particular reason I can think of, but they scare me a bit.
6. They are harder to have ZCIs for. A while back I batch cut about 10 Zero Clearance Inserts of my TS, and most of those are now cut for different width dadoes. I always have a spare blank or two that I can dedicate to a random width dado if necessary. But if you don't have the time invested in making many ZCIs and having a few blanks around, you might be running a dado with an oversized ZCI, or even worse, no insert at all. Can't even imagine how scared I would be to cut a dado without a TS insert.