Here is one cool thing I ran into along the way. One of the application's executables would start and immediately die with this error:
"The application has failed to start because its side-by-side configuration is incorrect. Please see the application event log for more detail."
In the event log we faced a mostly gibberish error that indicated it couldn't find x86_microsoft.vc90.crt AKA the 32-bit Visual C++ 9.0 runtime AKA Visual C++ 2008.
The solution was to download and install the Visual C++ 2008 SP1 runtime from Microsoft.
If that link happens to rot away, the search keywords to find that package are "Microsoft Visual C++ 2008 SP1 Redistributable Package (x86) Download"
Viola! The application worked!