Let me be more specific.
Extremes don't matter to a rational, balanced argument. For example, fear of flying on an airplane is silly -- airplanes are proven to be hundreds of times safer than flying in a car. But because airplane crashes are extreme (as in, lots of people die at once), we pay more attention to them than to car crashes, which are much much more lethal overall.
Extreme don't matter because the chances of anyone ever encountering an extreme are slim. Humans being loss-averse, we are prone to avoiding extremes that involve loss (like flying on a plane) even as we engage in extremes that involve gain (like the lottery). Both the avoidance of a safe method of flying and the engagement in a completely fruitless waste of money are out and out crazy, but we do them anyway.
To a sane, rational, logical person however (and I'm not even touting that as an ideal, just stating a fact), arguing from the extreme is irrelevant because extremes don't come into play often enough to make a relevant point. Thus, they don't matter in a very real an literal sense.