And why you would want it...
As I am finding out, drupal has lots of modules and lots of settings. You could probably spend an entire lifetime understanding them all. One of the challenges of the non-wiki format is that posts and comments on drupal.org from people who are providing help become out of date. So instead of a "master" article being corrected an updated (a la wikipedia), you have to scroll through the entire set of posts to see the most recent. Of course, you see the oldest posts first, then the newest at the very bottom. But when you're looking for help, you really want to see the newest posts at the top and the oldest posts at the bottom (like a blog).
In any case, the reason you would want Clean URLs is that clean URLs are SEO-friendly. The reason you want SEO-friendly URLs is that they are more easily indexed by the search engine spiders, and besides, they look nicer to users. An "dirty" URL is of the form www.mysite.com?q=node/4, whereas a "clean" URL is of the form www.mysite.com/node/4. The different is in the ?q=, which can confuse some spiders and annoy humans.
Why doesn't drupal ship with clean URL support on by default? Because... they expect you to be an admin! The real reason is that some web servers aren't configured to support the rewriting necessary to get from the "dirty" format to the "clean" format, resulting in an installation that doesn't work and no way to fix it. This is sometimes the case on shared hosting servers. Kind of reminds me of installing old revs of Linux before Red Hat and SuSE matured the installation process. They use to configure everything even for the 1% case rather than focusing on the 90% case.
So to configure Clean URL support, check
this out. The bottom line is you can just go to Administer | Settings | General Settings then scroll down till you see Clean URL support. Don't see it? I didn't see it the first time either. Try using your web browser's search function. You'll need to run the "clean URL test" (look for the little link) first.
If you look on the drupal web site, you'll find many of the comments are out of date (as I indicated earlier). You'll see that the most recent versions of drupal don't require you to modify the Apache .htaccess configuration files directly - you can simply change the setting via the drupal administration interface.