March 1999
Version: 1.0, 3.0

Spawning external applications

by Kent Reisdorph

The Inprise newsgroups are a great source of information for C++Builder programmers. It seems that barely a day goes by without someone asking the question, "How do I execute an external application from my program?" This article will explain how to spawn an external application from within your program using the ShellExecute, WinExec, and CreateProcess Windows API functions. We'll also show you how to make your application wait until a spawned external program has terminated before continuing on.

Executing an external application

There are many reasons you may want to execute an application from within your program. You may, for example, want to display a text file to your users. Or you may have an application that your program must call in order to perform some specialized processing. Sometimes this type of program is a legacy 16-bit Windows or DOS program for which you don't have the source code. Running the application from your program is the only way to gain the external application's functionality. This situation is common in applications being ported from DOS to Windows. You essentially have three choices of Win32 API functions when executing an external application:
bullet The WinExec function
bullet The ShellExecute function
bullet The CreateProcess function
The following sections will explain each of these functions and how they are used. The functions are presented from easiest to use to most complex.

 
Note: We aren't including the spawn and exec family of functions from the C/C++ runtime library in this article. Those functions, while useable in a C++Builder application, weren't designed for the Win32 environment.

WinExec

The WinExec function provides the easiest way of spawning an external process. It's the simplest in that WinExec takes only two parameters. Here's the function declaration for WinExec (found in WINBASE.H):
UINT WinExec(
  LPCSTR lpCmdLine, UINT uCmdShow);
The first parameter, lpCmdLine, is the command line. The command line can include an executable filename, or the filename and additional parameters to pass to the application. The uCmdShow parameter is the window display command. uCmdShow is usually one of the values shown in Table A.

Table A: Common show commands
CommandDescription
SW_HIDEThe application is run invisibly.
SW_MAXIMIZEThe application is run maximized.
SW_MINIMIZE The application is run minimized.
SW_NORMALThe application is run in the default size and position.

While the show commands in Table A are the most commonly used values for the uCmdShow parameter, they aren't the only possible values. For a complete list of possible values for this parameter, see the ShellExecute topic in the Win32 API Help.
Note: You should only run applications hidden if you know they don't require any user input. If a hidden application requires user input, it--and possibly your application--will hang waiting for input that will never come.

If the call to WinExec succeeds, the return value is an integer greater than 31. If the return value is less than 32, an error occurred and the return value represents an error code. Reasons for an error include the EXE wasn't found, the path wasn't found, the system is out of memory, or a bad executable format was detected. The bad format error will occur if the application being executed isn't a valid Win32 application or is corrupted.

The following example illustrates executing Windows Notepad using WinExec:

WinExec("notepad.exe", SW_NORMAL);
We simply pass the name of the application to execute and the show command value. If the filename doesn't include a path, Windows will attempt to locate the file (see "How Windows locates files"). Here's another example that executes Notepad and loads a file called TEST.TXT:
WinExec("notepad.exe test.txt", SW_NORMAL);
As you can see, using WinExec in an application is trivial. Although WinExec is easy to use, it suffers from some major drawbacks. First, the application can't exercise any control over the spawned application. Most notably, your application can't easily determine when the spawned application has terminated. WinExec starts the application specified in the lpCmdLine parameter and returns immediately. At that point, your application is on its way again, with no consideration given to the state of the spawned application.

Another drawback to WinExec is that it's on the Win32 "endangered species" list. Specifically, the Win32 API Help has this to say about WinExec: "Win32-based applications should use the CreateProcess function rather than this function. The WinExec function exists in Win32 to provide compatibility with earlier versions of Windows." This warning essentially says that WinExec may not be available in future versions of Windows. While it's unlikely that WinExec will be removed from Windows any time soon, you should be aware that applications that use WinExec might have to be modified and recompiled if this function is removed from Windows at some point in the future.

Finally, WinExec can't be used to execute a 16-bit program. If your application must execute a 16-bit program (either DOS or Windows) you need to use CreateProcess rather than WinExec.

ShellExecute

The ShellExecute function can be used to spawn an executable file, but it has a specific feature that makes it much more powerful than WinExec. The key to this feature is the function name itself. ShellExecute can be used to display any document for which the Windows shell has registered a file extension. Windows keeps a list of known file extensions and the application associated with that extension. For example, for a default Windows installation the TXT extension is associated with Windows Notepad, the BMP extension is associated with the Paint program, the WAV extension is associated with the Sound Recorder program, and so on. Other applications you install may also register their file extensions. Examples include Microsoft Word (DOC), Microsoft Excel (XLW), and C++Builder (CPP, BPR, RES, and others).

The Windows shell uses these associations in many ways. One way is in Windows Explorer. If you double-click on a filename in Explorer, Windows looks up the filename's extension in its list of file associations. If it finds an association for that file, the associated application is executed and the document that was double-clicked is loaded into that application. If, for example, you double-click on a file with a TXT extension, Windows will start Notepad and will load the text file (assuming the TXT extension hasn't been changed to be associated with a program other than Notepad).

Another way Windows uses file associations is when you call the ShellExecute function. If the filename passed to ShellExecute is a document file, Windows searches the association database just as it does when you double-click a file in Explorer. We'll talk more about that later, but first let's take a look at the ShellExecute declaration in SHELLAPI.H:

HINSTANCE ShellExecute(HWND hwnd, 
  LPCSTR lpOperation, LPCSTR lpFile, 
  LPCSTR lpParameters, LPCSTR lpDirectory, 
  INT nShowCmd);
Obviously ShellExecute is a bit more complex than WinExec. In some cases, you'll only use a couple of the parameters and the rest can be 0. In other cases, you'll use all of the parameters. First, we'll describe the individual parameters and then we'll look at some examples. The hwnd parameter is the handle of the window that will act as the parent for the spawned application. The lpOperation parameter is used to specify the action that the spawned application should perform on the document when it's opened. Operations include open, print, and explore. If the operation is open, then Windows executes the associated application and opens the document specified in the lpFile parameter. If the operation is print, then Windows will print the document. This assumes, of course, that the application associated with the given filename is capable of printing. If operation is explore, then Windows will open the folder specified in lpFile in a shell browser window.

As you've probably surmised, the lpFile parameter is used to specify the document to open or print, an executable file to run, or a folder name. The lpParameters parameter is used to specify any additional command line arguments you wish to pass to the application. If no command line arguments are needed, set this parameter to 0. The lpDirectory parameter is used to specify the default directory for the application. If this parameter is 0, the current directory is used. Finally, the nShowCmd parameter is used to specify the show command. The possible values for this parameter are listed in
Table A.

If successful, the return value of ShellExecute will be greater than 31 and will be the instance handle of the application that was executed. If the return value is 31 or less, the return value is an error code. Possible errors include those listed earlier for the WinExec function, plus a few that are specific to ShellExecute (such as the no such file association error code). We won't list all of the possible error codes here, but you can find them listed under the ShellExecute topic in the Win32 API Help.

After that relatively dry explanation of ShellExecute's parameters, a few examples are in order. Let's take first the situation where you want to use ShellExecute to simply run an external application. In that case, the code might look like this:

ShellExecute(Handle, 0, "notepad.exe", 0, 0, SW_NORMAL);
Because we're simply using ShellExecute to spawn an external application, we can supply 0's for most of the parameters. Now, let's look at an example that opens a particular document file. Look at these two lines of code:
ShellExecute(Handle, "open", "test.txt", 0, 0, SW_NORMAL);
ShellExecute(Handle, 0, "notepad.exe", "test.txt", 0, SW_NORMAL);
In the first example, we simply pass a command of open and the filename to open. Windows uses file association to open the application associated with the TXT extension (Notepad in most cases). In the second example, we pass the name of the executable in the lpFile parameter and the name of the file to open in the lpParameters parameter. Use the latter method if you prefer not to rely on Windows file associations to run an application. These two lines result in exactly the same behavior if Windows Notepad is associated with the TXT extension (the default Windows setup). ShellExecute can be used as a poor man's text file printer. Using ShellExecute you can execute Notepad invisibly and print the contents of a text file. Here's an example:
ShellExecute(Handle, "print", "test.txt", 0, 0, SW_HIDE);
Note that here we are using a command of print and that the nShowCmd parameter is SW_HIDE. ShellExecute can be used in a wide variety of circumstances. It can be used to display a Web page in a Web browser, to invoke the user's default email client, or to execute batch files (BAT). The following examples show how you might use ShellExecute in these situations:
// Execute a batch file.
ShellExecute(Handle, 0, "go.bat", 0, 0, SW_NORMAL);
// Display a Web page.
ShellExecute(Handle, 0, "http://www.turbopower.com", 0, 0, SW_NORMAL);
// Begin an e-mail message.
ShellExecute(Handle, 0, "mailto:info@turbopower.com", 0, 0, SW_NORMAL);
As you can see, ShellExecute is a very versatile function. It does, however, suffer from one of the limitations that we mentioned in the section on WinExec. Namely, it's incapable of executing a 16-bit program. For that task you'll have to use CreateProcess. ShellExecute has a companion function called ShellExecuteEx. ShellExecuteEx provides everything that ShellExecute does, plus the ability to get a handle to the spawned application's process. This is important if you want your application to pause execution until the spawned application has finished. See the sidebar "Waiting on a spawned process," for a bit more discussion on this process.

CreateProcess

CreateProcess is the preferred way of executing external applications in a Win32 program. The biggest problem with CreateProcess is that it's a bit cumbersome to use. Looking at the Win32 API Help for CreateProcess can easily leave you with the impression that it's just too difficult to use. The truth is that the whole of CreateProcess is horribly complex. The good news, however, is that you don't have to understand CreateProcess in its entirety, but only a small subset of it. Once you know how to use CreateProcess to execute an application, however, you'll find that it's not as difficult to implement as you had feared. We won't attempt to explain all of the functionality of CreateProcess, but rather will show you only as much as you need to know to run an external application. First, let's look at the declaration for CreateProcess (also in WINBASE.H):
BOOL CreateProcess(
  LPCSTR lpApplicationName,
  LPSTR lpCommandLine,
  LPSECURITY_ATTRIBUTES 
  lpProcessAttributes,
  LPSECURITY_ATTRIBUTES lpThreadAttributes,
  BOOL bInheritHandles,
  DWORD dwCreationFlags,
  LPVOID lpEnvironment,
  LPCSTR lpCurrentDirectory,
  LPSTARTUPINFOA lpStartupInfo,
  LPPROCESS_INFORMATION 
  lpProcessInformation );
Granted, this declaration looks a bit intimidating, but many of the parameters can be ignored. This is particularly true of the parameters dealing with security attributes. A basic call to CreateProcess looks like this:
STARTUPINFO si;
PROCESS_INFORMATION pi;
memset(&pi, 0, sizeof(pi));
memset(&si, 0, sizeof(si));
si.cb = sizeof(si);
int res = CreateProcess("c:\\windows\\notepad.exe", 0, 0, 0, 0, 0, 0, 0, &si, &pi);
First, we declare two structures and zero them out to insure they don't contain random data. Next, we set the cb member to the size of the STARTUPINFO structure. Finally, we call CreateProcess to spawn the application. As you can see, the only required parameters are the lpApplicationName, lpStartupInfo, and lpProcessInformation parameters. If CreateProcess succeeds the return value is non-zero. If CreateProcess returns 0, you should call GetLastError to get an error code indicating what went wrong.

 
Note: If the file to be executed isn't in the current directory, you must fully qualify the path and filename of the application. This is only true of filenames passed in the lpApplicationName parameter, not for filenames passed in the lpCommandLine parameter.

There's a quirk of CreateProcess that you should be aware of if you plan on running a 16-bit application from your program. Take this code for example:

CreateProcess("MyDosApp.exe", 0, 0, 0, 0, 0, 0, 0, &si, &pi);
Note that we passed the filename of the file to execute in the first parameter, lpApplicationName. Under Windows 95 and Windows 98 this code will fail if the filename passed in lpApplicationName is a 16-bit DOS application (it will work on Windows NT, however). In order to execute a 16-bit DOS app in Win95 and 98 you must pass the filename in the lpCommandLine parameter rather than in the lpApplicationName parameter. The following code shows the correct way of calling CreateProcess to spawn a 16-bit application:
CreateProcess(0, "MyDosApp.exe", 0, 0, 0, 0, 0, 0, &si, &pi);
This code will work properly in all versions of Windows (32-bit) so you should make a habit of passing the filename to execute in the second parameter in your call to CreateProcess rather than in the first parameter. The STARTUPINFO structure passed to CreateProcess can contain additional startup information if desired. In the previous examples, we wanted to run the spawned application normally so we didn't provide any startup information. If you wanted to run the application hidden, you'd need to do this:
si.dwFlags = STARTF_USESHOWWINDOW;
si.wShowWindow = SW_HIDE;
int res = CreateProcess(0, "myapp.exe", 0, 0, 0, 0, 0, 0, &si, &pi);
See the STARTUPINFO topic in the Win32 API Help for full details on control of the startup process.

Conclusion

Spawning external applications in Windows can be a frustrating exercise the first time you attempt it. Knowing the pitfalls, as explained in this article, may save you hours of time. With WinExec, ShellExecute, ShellExecuteEx, and CreateProcess, you have everything you need to effectively spawn external applications from your programs.

Kent Reisdorph is a editor of the C++Builder Developer's Journal as well as director of systems and services at TurboPower Software Company, and a member of TeamB, Borland's volunteer online support group. He's the author of Teach Yourself C++Builder in 21 Days and Teach Yourself C++Builder in 14 Days. You can contact Kent at editor@bridgespublishing.com.