It's essentially radio-on-demand, where you choose exactly what you want to listen to, rather than being subject to the programming choices made by others.
With MP3 players, you can literally listen to the shows anywhere, anytime.
Podcasts give you something interesting to listen to (and think about!) during those times when your mind is only superficially engaged: driving, travelling on public transport, mowing the lawn, washing the dishes, walking the dog, or any kind of exercise.
If you miss something in the show—due to zoning out, something I seem to be prone to—you can "rewind" (something you can't do with radio).
If a particular episode bores you, you can just skip it.
While you can listen to music instead of podcasts, some people (c'est moi!) often prefer to have something that provides information, entertainment, knowledge, current affairs... all tailored by your choice of what shows you listen to.
If you haven't tried listening to podcasts—a phenomenon that's barely two years old—I highly recommend giving it a go!
(No, I'm not a podcaster, nor in all likelihood ever will be; my talents just don't lie in the direction of entertaining commentary/discussion. I'm enthusiastic about it simply because it's valuable to me, for the reasons I've given.)