There's often multiple compilation steps, first compiling some application specific representation down to HLSL/GLSL/MSL source code, then compiling that down to DXIL/SPIRV/Metal bytecode, and then handing that off to the driver for compilation to native code. The first two steps at least can be done ahead of time, the bytecode is stable.
Metal and CUDA actually let you AOT compile native binaries since they have relatively few hardware targets to support, with a fallback to compiling bytecode to native at runtime for forwards compatibility.
The bytecode compilation doesn't really matter, it's still gonna jank. Maybe not as badly, but it will. The only way to get rid of jank is to cache not the bytecode and not even the native code, but the PSOs. Realistically, that is only possible on the target machine, unless you have a ginormous farm of machines representing all the permutations of hardware and drivers. That's basically what Steam does with its mass of users.
The alternative is to just not have that many shader permutations, and potentially take a performance hit from that. This seems to be the strategy that Impeller is following.
I have not seen shipping prebuilt PSOs outside of game console platforms. Typically games that care do on device caching if any. Unfortunately, lots of projects just take shader hitches as given.
Metal and CUDA actually let you AOT compile native binaries since they have relatively few hardware targets to support, with a fallback to compiling bytecode to native at runtime for forwards compatibility.