Debugging the RAD Studio IDE

By | December 15, 2018

When I work on my IDE plugins I constantly need to debug the IDE. To make my live a lot easier I wrote an IDE plugin that adds another symbol resolver to the Delphi Debugger, so the debugger can use the *.jdbg files and show the actual function names (including line numbers) in the call stack. Even the CPU view uses more colors to show call, jmp, ret, nop, try/finally/except and more in different colors and resolves call and jump target addresses to their names if available.

In order to identify the code that causes performance issues I use “poor man’s profiling”. I pause the IDE multiple times while the task that takes up the time is running and look at the call stack. If a function comes up in all call stacks, it is is either the culprit or a function that is called very often (or you are just unlucky).

If the task takes really long, pausing the IDE withing the debugger IDE via the pause button is doable, but if the task takes only 2 seconds switching to the other IDE or moving the mouse to the pause button can be too slow, so I use the F12 debug hotkey that Windows reacts to and notifies the debugger to pause the process. Unfortunately Microsoft disabled that feature with Windows Vista. So I had to emulate that functionally with my own IDE plugin that reacts to the F12 hotkey while the debugger is running.

Tools used:

Installing the IDE plugins

Both tools come without an installer, so you have to install them by hand. After extracting the 7z files you have to create two registry entries (Delphi 10.3 Rio):

  1. Start regedit.exe
  2. Navigate to HKEY_CURRENT_USER\Software\Embarcadero\BDS\20.0\Experts
  3. If the “Experts” key doesn’t exist, you have to create it by right clicking on the “20.0” key and select “New/Key”
  4. Add a new String value by right clicking in the listview and select “New/String Value”.
    Name: DebuggerCallStackResolverR103
    Value: C:\The\Path\To\DebuggerCallStackResolverR103.dll
  5. Add another String value by right clicking in the listview and select “New/String Value”.
    Name: DelphiF12HotKeySupport
    Value: C:\The\Path\To\DelphiF12HotKeySupport.dll
  6. Close regedit.exe

Starting the debugger IDE

Start the RAD Studio IDE. There is nothing special here.

Starting the IDE that we want to debug

For IDE plugins and component packages

If you are writing an IDE plugin or want to debug a component, you open your project and in the “Run/Parameters…” dialog set the “Host application” to “$(BDS)\bin\bds.exe” (without the quotes). If you want to use a different registry key for the debugged IDE then you can also set the “Parameters” to “-rMyRegistryKey“. That way the IDE starts with a new registry key having a clean IDE, so it doesn’t have any additional library paths, components or IDE plugins installed that.

Compile and Start your project.

I use the -r command line parameter for my IDE plugins because the debugged IDE has to load the just compiled versions of the DLLs instead of the release versions that are installed into the debugger IDE.

Just for call stacks

If you only want to identify an IDE bug or a performance issue you can start the second IDE without opening any project by using the “Run/Load process…” dialog. Select the “Embarcadero Windows 32bit-Debugger” in the Debugger combo box and set the “Host application” to “$(BDS)\bin\bds.exe” (without the quotes). Set the “After load” radio button to “Run” and press the “Load” button.

Skipping the usual startup exception

The IDE throws some exception during the startup, that are handled by the IDE but now that a debugger is attached to the IDE’s process you will see them.

The first exception you may encounter is an EFOpenFile exception for the “%USERPROFILE%\sanct.log” file. That has something to do with the product licensing and now that two IDEs are running the first holds the file exclusively open causing the second IDE to fail to open the file. You can ignore this exception (you may add the EFOpenFile exception to the ignore list if you start the debugged IDE multiple times.

The next exception is ESanctSocketException “Cannot start instance counter. Port is already in use (10048).”. Again caused by the product licensing. This time it wants to listen to a port but the debugger IDE already opened that port. Add that exception type to the ignore list because we don’t care about it and the exception type is a special type not like EFOpenFile)

The EFOpenFile exception for the sanct.log file is thrown again. Skip it again.

And finally we reach the EAccessViolation from the welcome page (something the IDEFixPack for Rio will fix). Skip that one.

If you are in the editor you may see an EParseError exception. You can add that exception to the ignore list.

Identifying the performance issue

Now you can start the task that you think is taking a lot of time and while it is causing the CPU or I/O usage to be high, you can pause the debugged IDE by pressing the pause button in the first IDE or by pressing the F12 debug break hotkey in the debugged IDE. This brings you usually to the CPU view and you can have a look at the call stack of all the threads by double clicking on a specific thread in the Threads list what shows the thread’s call stack in the Call Stack window.