It's not even that. SciPy is paying the price of the idiotic (and highly biased) decisions of Python core-dev members who chose MSVC over MinGW for Python on Windows. (And their motivation is derived from Microsoft sponsorship. A bunch of people on core-dev list are straight-up Microsoft employees, who are paid by Microsoft to be on that list, and, on top of that, Microsoft pays for CI servers for CPython project).
This whole problem could've been avoided if Python just didn't use proprietary tools in its toolchain.
I'm not knowledgeable enough about portability of LLVM, but if it's binary compatible with GCC-compiled stuff (or can be made such) on Windows, then, sure.
See, the problem here is that if you want interop, with, eg. Ruby, Erlang, R, Perl, Go and probably a bunch of others (the only other exception I know of is PHP (PECL) that uses MS toolchain), then you have to produce compatible binaries.
Ideally, it shouldn't be about the flavor of compiler, but be some kind of official documented format... that many compilers can easily implement. But, since de facto this format is "just use GCC", then, in practical terms, either use GCC, or pretend you use GCC.
BTW, in the quest to find out how other popular languages deal with this problem, I discovered that things can actually be even worse than Python. Enter JavaScript (Node.js)!
In JavaScript, you cannot distribute pre-built native modules, only source code. And if it compiles, then it compiles, and if it doesn't -- it's the user's fault.
This whole problem could've been avoided if Python just didn't use proprietary tools in its toolchain.