If you have XSS attack vector on your site, the attacker can do almost anything including cookie access(for all cookies that aren't HttpOnly). Also CSRF protection doesn't protect against clickjacking attacks where the page is iframed set transparent and user is encouraged to click in a specific location(punch the monkey in the face) on the attackers page. Easiest solution is to use javascript to detect if your page is iframed and bust out of it.
For click-jacking, the easiest thing to do is to set the X-Frame-Options header, but I'll get to that. And it doesn't help IE <= 7, so you need to weigh cost/benefit and your user base there.
And we'll get to session hijacking and why your session cookies in particular should always be HttpOnly and preferably secure.
I understand your question, and in many ways you're correct; that said, let's say you have stored XSS in some obscure parameter of your web application.
One way to exploit the Victim via XSS would be to convince them to browse to the compromised section of the site. Sometimes, that's as simple as getting them to click on a malformed URL -- basically, the same vector as getting them to click on your CSRF-ized link, anyway.
Many applications, however, don't have direct URLs to certain areas of the app. It could be hidden by Javascript/AJAX population, or written in a language/framework that doesn't support that kind of direct linking.
Therefore, if some kind of sensitive function is easily owned by CSRF (say, account.php?sendto=Eve), but the XSS is stored someplace obscure that might not be available via public URL (browse to Accounts -> Details -> My Bio, which populates dynamically), the higher threat severity would be the CSRF.
Sorry this turned kind of long, but I deal with a lot of webapp security issues on a day-to-day basis and feel relatively qualified to answer this one in some depth :)
Hopefully you're taking steps to prevent both. But yes, closing the CSRF window and leaving the XSS door open would largely defeat the purpose of CSRF protections.
