1. Does "in-game" Broodwar have any open SDlgDialog windows?
I have no idea, and don't know how to check. My best guess is probably yes when you click on the menu button.
2. For Each SDlgDialog window GdiTransparentBlt color converts the same 640x480 from 8bpp to 32bpp each time... Would it be faster to color convert once then GdiTransparentBlt out to all the windows?
Not a graphics programmer, but to check if anything is faster, you would profile the code.
3. Would it be faster to GdiTransparentBlt only the changed portions of the screen instead of the entire screen to every window... Unfortunately the game locks the entire screen so we don't immediately know where changes occur. However, since we know every unchanged scan line will be all 0xFE bytes... it should be possible to do a quick hash of a scan line to know if anything on that line has changed... which would allow us to quickly determine a bounding rect around changes which we could then intersect with the window rects thus allowing us to move much fewer bytes around.
Updating only parts of the screen is what modern video encoding libraries and game developers do nowadays so it should be faster. Maybe it's even faster to just check the first few bytes of a line for 0xFE bytes instead of hashing the whole line, what would go wrong if this assumption is incorrect?
4. The only thing we can do for "ddraw only mode" is switch to uploading to video memory with a real API, not GDI. But that means when switching from "mixed mode gdi/ddraw" to "ddraw only" we need to clear any gdi stuff... can we just paint the main window with a NULL_BRUSH or do we have to destroy and re-create the main window?
Are you suggesting to interpret the DirectDraw calls and use something like OpenGL or DirectX to draw to the screen instead?