Regarding sprites, it's actually a bit more complicated. We have to differentiate between sprite files and image files. Image files are only loaded once, because they're static. Sprites are loaded for each particle, because each particle uses a different animation instance (the image files referenced by those animations are only loaded once).
However, since WME allows you to pass an image file directly to SetSprite(), for your convenience, so that you don't need to create one-frame sprites, the side effect is that the image file is checked everytime a new particle is created (in this case, the image works as a sprite itself). Confused already?
Usually there's little harm loading sprite files over and over, because they are small (especially when the game is compiled, they're loaded in compressed version). The real slowdown is loading the image files (especially huge ones), and those are cached (unless you reference them directly, as I described above, in that case they're indeed loaded and dropped immediately, this definitely could be improved).
However, unless you're using huge images for particles, IMO the Windows and disk cache optimize any possible slowdowns.
Regarding the sounds. They are streamed from disk if they are longer than the sound buffer (3 seconds by default, I think). Otherwise the entire sound is loaded into buffer and played. After being played, the sound is dropped, because currently the engine doesn't have a way of knowing whether the sound is going to be used again one second later or not. Again, for repeated short sounds the disk cache usually does the dirty work. I didn't study this any deeper, but the real performance hit seems to be the Vorbis initialization, because e.g. if you use WAV files for footstep sounds, the stuttering doesn't occur.
What about speech lines? They are ogg files just like any other sound file but I can't preload them? Will my scene stutter each and every time my actor says anything?
I'm afraid so.
And what about preloading other stuff? For example, if I call Game.SetCursor("Cursor1.sprite"), then I call Game.SetCursor("Cursor2.sprite") and then I call again Game.SetCursor("Cursor1.sprite"). Will Cursor1.sprite be read again from disk or is it kept loaded in memory from my first call to Game.SetCursor("Cursor1.sprite")?
I already answered this above. Only image files are cached. The actual sprite definitions are not, and IMO there's no reason for it.