I never knew the name of the music Steve closes out the show, but always liked it on the now defunct John From Cincinnati HBO show. I finally looked it up and found it on you-tube and Amazon.
Johnny Appleseed, Joe Strummer & The Mescaleros live on Letterman
http://www.youtube.com/watch?v=mxo410q0xmw
Amazon has short cuts from the $10.97 album (free shipping with a $25 order)
"Johnny Appleseed" Global a Go-Go Joe Strummer & The Mescaleros
http://www.amazon.com/Global-Go-Go-J.../dp/B00005M98E