Normally all of the code that was in RAM is flushed, since the buffers aren't setup in a way that allow you to flush part of it, and doing so would make any other blocks that branch into the modified block invalid. These modifications to code do have to be checked, because it means that the code that has been dynamically recompiled has to be flushed (since it is no longer valid). Sometimes it's reloaded when the game changes significantly (like entering a new area, or going from the title screen to ingame), but usually it stays the same for a long time. Usually this code is just left alone after being loaded, so it's like ROM but faster.
Most games will load a bit of code into RAM to run it quickly.
ROM can't be overwritten, so when code is encountered from ROM it's known that it'll always be the same. gpSP doesn't support executing code from VRAM right now so that's out already). This is the issue: GBA can execute code from one of two places, ROM (the cartridge and the BIOS) and RAM (that includes IWRAM, EWRAM, and rarely, VRAM. This idea was a bit complex and had some serious complications and I basically abandoned it when I thought of something simpler. Some months ago I came up with a new idea to try to improve the situation. I've tried some hacks to get around it, and they've helped, but they were very ugly.
There is an issue with certain GBA games that has plagued me since I first started work on gpSP. (NOTE: this post pertains to any ports of gpSP using dynarec, that means the PSP and GP2X versions)