When I work on my IDE plugins I constantly need to debug the IDE. To make my live a lot easier I wrote an IDE plugin that adds another symbol resolver to the Delphi Debugger, so the debugger can use the *.jdbg files and show the actual function names (including line numbers) in the call stack. Even the CPU view uses more colors to show call, jmp, ret, nop, try/finally/except and more in different colors and resolves call and jump target addresses to their names if available.

In order to identify the code that causes performance issues I use “poor man’s profiling”. I pause the IDE multiple times while the task that takes up the time is running and look at the call stack. If a function comes up in all call stacks, it is is either the culprit or a function that is called very often (or you are just unlucky).

If the task takes really long, pausing the IDE withing the debugger IDE via the pause button is doable, but if the task takes only 2 seconds switching to the other IDE or moving the mouse to the pause button can be too slow, so I use the F12 debug hotkey that Windows reacts to and notifies the debugger to pause the process. Unfortunately Microsoft disabled that feature with Windows Vista. So I had to emulate that functionally with my own IDE plugin that reacts to the F12 hotkey while the debugger is running.

Tools used:

• Delphi F12 Debug Hotkey support
• Delphi Debugger Callstack Resolver

Installing the IDE plugins

Both tools come without an installer, so you have to install them by hand. After extracting the 7z files you have to create two registry entries (Delphi 10.3 Rio):

1. Start regedit.exe
2. Navigate to HKEY_CURRENT_USER\Software\Embarcadero\BDS\20.0\Experts
3. If the “Experts” key doesn’t exist, you have to create it by right clicking on the “20.0” key and select “New/Key”
4. Add a new String value by right clicking in the listview and select “New/String Value”.
   Name: DebuggerCallStackResolverR103
   Value: C:\The\Path\To\DebuggerCallStackResolverR103.dll
5. Add another String value by right clicking in the listview and select “New/String Value”.
   Name: DelphiF12HotKeySupport
   Value: C:\The\Path\To\DelphiF12HotKeySupport.dll
6. Close regedit.exe

Starting the debugger IDE

Start the RAD Studio IDE. There is nothing special here.

Starting the IDE that we want to debug

For IDE plugins and component packages

If you are writing an IDE plugin or want to debug a component, you open your project and in the “Run/Parameters…” dialog set the “Host application” to “$(BDS)\bin\bds.exe” (without the quotes). If you want to use a different registry key for the debugged IDE then you can also set the “Parameters” to “-rMyRegistryKey“. That way the IDE starts with a new registry key having a clean IDE, so it doesn’t have any additional library paths, components or IDE plugins installed that.

Compile and Start your project.

I use the -r command line parameter for my IDE plugins because the debugged IDE has to load the just compiled versions of the DLLs instead of the release versions that are installed into the debugger IDE.

Just for call stacks

If you only want to identify an IDE bug or a performance issue you can start the second IDE without opening any project by using the “Run/Load process…” dialog. Select the “Embarcadero Windows 32bit-Debugger” in the Debugger combo box and set the “Host application” to “$(BDS)\bin\bds.exe” (without the quotes). Set the “After load” radio button to “Run” and press the “Load” button.

Skipping the usual startup exception

The IDE throws some exception during the startup, that are handled by the IDE but now that a debugger is attached to the IDE’s process you will see them.

The first exception you may encounter is an EFOpenFile exception for the “%USERPROFILE%\sanct.log” file. That has something to do with the product licensing and now that two IDEs are running the first holds the file exclusively open causing the second IDE to fail to open the file. You can ignore this exception (you may add the EFOpenFile exception to the ignore list if you start the debugged IDE multiple times.

The next exception is ESanctSocketException “Cannot start instance counter. Port is already in use (10048).”. Again caused by the product licensing. This time it wants to listen to a port but the debugger IDE already opened that port. Add that exception type to the ignore list because we don’t care about it and the exception type is a special type not like EFOpenFile)

The EFOpenFile exception for the sanct.log file is thrown again. Skip it again.

And finally we reach the EAccessViolation from the welcome page (something the IDEFixPack for Rio will fix). Skip that one.

If you are in the editor you may see an EParseError exception. You can add that exception to the ignore list.

Identifying the performance issue

Now you can start the task that you think is taking a lot of time and while it is causing the CPU or I/O usage to be high, you can pause the debugged IDE by pressing the pause button in the first IDE or by pressing the F12 debug break hotkey in the debugged IDE. This brings you usually to the CPU view and you can have a look at the call stack of all the threads by double clicking on a specific thread in the Threads list what shows the thread’s call stack in the Call Stack window.

The DDevExtensions 2.85 and the DFMCheck 1.6 are now available for Delphi 10.3 Rio.

DDevExtensions Changelog:

• Added: Support for Delphi 10.3 Rio
• Added: Use Unit dialog option “Every unit on a single line”
• Improved: UnitSelector Dialog in Delphi 2009 opens much faster
• Fixed: Structure-View search dropdown had a max height of 2 items

DDevExtensions

DFMCheck

There was an off-by-one bug in the Win64 Compiler Speed Pack for XE2 since the release of version 6.0. This caused the XE2 dcc64 compiler to generate defective code that caused crashes in the created executable file.

The off-by-one bug only affected XE2. When I began writing the patches I used XE4 and all Win64 compilers worked almost out of the box with the same patches. But XE2 is different as it was the first Win64 compiler. One of the enumerations got extended by one element in XE3 and the patch didn’t have checks for that case. So I never noticed that my enumeration type had an additional element in the middle if it was compiled for the XE2 compiler. That change offsets all following elements by one an caused bugs in the patches.

There is now a special release 6.3.2 for XE2 that addresses this IDE Fix Pack bug.

Download

The new IDE Fix Pack 6.3 adds some IDE fixes and makes DFM reading a little bit faster due to the usage of buffers on the stack instead of the TBytes heap allocations with unnecessary FillChar calls that were introduced with XE3.

The compiler option extension -x-fpr also got a new feature. If a function has stack variables that exceed 4096 bytes, the function prolog that is generated will be up to 3 times faster than the original compiler generated code (RSP-19826).

Furthermore the compiler option extensions are now combined into -x-On options what makes it easier to specify the options because you don’t need to remember all the option names anymore.

-x-O1     Enable options -x-fvs   -x-fpr
-x-O2     Enable options -x-fvs   -x-fpr -x-orc
-x-O3     Enable options -x-fvs=2 -x-fpr -x-orc
-x-Ox     Enable ABI changing optimizations: -x-fvs=2 -x-fpr -x-orc=2 -x-ff

All options are listed in the Readme.txt and can be specified under Project Options/Delphi Compiler/Compiling/Other options/Additional options to pass to the compiler. (Screenshot)

Changelog

• Added: Patch to remove IDE flickering when WM_SETTINGCHANGE is broadcasted
• Added: Fix for RSP-20700: Tooltip Help Insight is blinking if Structure View is scrolled
• Added: Undo XE3+ TFiler/TReader/TParser/TStream TBytes usage, replace SetLength with SetLengthUninit for special cases
• Added: -x-fpr generates 3 times faster stack memory page probing code (RSP-19826)
• Added: Options -x-O1, -x-O2, -x-O3, -x-Ox that enable other optimization options

Download:

Download (fastdcc for the command line compiler):

Delphi 10.2 Tokyo changed how the files for units that are explicitly specified in the project file (dpr/dpk) are found. The IDE Fix Pack’s directory file search cache still made assumptions that were correct in Delphi 2009-10.1 Seattle but aren’t in Tokyo. This could result in an “program or unit xxx recursively uses itself” error messages if you had a relative path in the filename that is specified in the project file.

This release also adds another option to the compiler codegen optimizations. The new -x-orc / -x-orc=n option allows you to eliminate temporary record copies for functions like “begin Result := FRedirect.GetRecord; end;”. This optimization allows the compiler to skip the try/finally block with InitializeRecord/CopyRecord/FinalizeRecord calls for the temporary record variable that is then copied to the actual result record.

• Fixed: Directory search cache failed if project units had “..\” in it (Delphi 10.2 only)
• Fixed: Some VirtualProtect calls specified nil as last argument what Windows 10 1709 doesn’t like if a debugger is attached.
• Added: Option -x-orc and -x-orc=n to remove temporary record variables for function results (n=1: only if the assignment is the last statement, n=2 for all)

Download:

Download (fastdcc for the command line compiler):

A small update for IDE Fix Pack that adds support for the just released Delphi 10.2.2.

• Added: Support for Delphi 10.2 Update 2
• Added: Fix for unnecessary temporary variable if an empty open array argument is part of a function call (Delphi 2009-10.1)

Download:

Download (fastdcc for the command line compiler):

While debugging the String4D code to hunt down a bug in the CompilerSpeedPack, I saw a lot of CopyRecord/FinalizeRecord calls with a try/finally that the compiler generated.

If you have a record with managed fields (string, interface, …) and use it as a function return type the Delphi compiler will change the function into a procedure with an extra var-parameter. So you would think that the compiler treats the result-parameter like a normal var-parameter, but that isn’t the case. The compiler will generate code that guarantees that the result record isn’t changed if the called function throws an exception. For this is adds a temporary record that is used for the function result and then copies it to the target record.

type
  TMyRec = record
    Value: string;
  end;

function InternalGetRec: TMyRec;
begin
  Result.Value := 'Hello';
end;

function GetRec: TMyRec;
begin
  Result := InternalGetRec;
end;

procedure Test;
var
  R: TMyRec;
begin
  R := GetRec;
end;

The compiler rewrites the “Test” function to:

procedure Test;
var
  R, TempR: TMyRec;
begin
  try
    GetRec(TempR);
    CopyRecord(TempR, R, TypeInfo(TMyRec));
  finally
    FinalizeArray([TempR, R], TypeInfo(TMyRec));
  end;
end;

The same happens if you assign another function’s record result value to your own record result value. The compiler rewrites the “GetRec” function’s code to:

function GetRec: TMyRec;
var
  TempResult: TMyRec;
begin
  try
    InternalGetRec(TempResult);
    CopyRecord(TempResult, Result, TypeInfo(TMyRec));
  finally
    FinalizeRecord(TempRecord, TypeInfo(TMyRec));
  end;
end;

Because the compiler assumes that you may want to use “Result” in the function after the call, it has to guarantee that it is unchanged if an exception is thrown. But if it is the last statement in the function and not secured by an explicit try/finally/except where “Result” is used again, an optimization could be to omit the temporary record, making the code a lot faster.

With the release of IDE Fix Pack 6.1, the Compiler Speed Pack not only makes the compiler compile faster but it can now also change the generated code, something that IDE Fix Pack has never done before. For this, new command line compiler options are introduced. They all start with “-x” (eXtension) followed by the Compiler Speed Pack option (-x-ff -x-fdi -x-fvs -x-fpr) and if you want to use them from the command line compiler you need to use fastdcc32/fastdcc64.
You can specify these options in the “Project/Options…” dialog under “Delphi Compiler/Compiling/Additional options to pass to the compiler”. You may need to rebuild the project to see an effect as only then the compiler will generate new code.

Windows 10 Creators Update 1703 caused issues with all Delphi programs, libraries and packages because it changed how Windows loads imported DLLs in such a way that it causes performance issues and can crash the debugger. Delphi 10.2 Tokyo Update 2 fixed this by not producing multiple dll import sections for one DLL anymore. IDE Fix Pack 6.1 implements that “feature” for all previous Delphi versions (2009-10.1 Berlin) and extends it to not only eliminate duplicate dll imports but also duplicate delay dll imports.
This patch changes the generated binary, the Win32 and Win64 compiler outputs, and it can be disabled by using the new “-x-fdi-“ option.

The next patch that changes the Win32 code generator is the “fast floating point” option that C++Builder users may know from the “bcc32.exe -ff” option. It removes all “fwait” instructions that the compiler usually emits after floating point operations. Removing “fwait” may cause FPU exceptions to be thrown at the wrong source code line.
This option is disabled by default and can be enabled by specifying the new option “-x-ff”.

When calling virtual methods through an interface the Win32 Delphi compiler has to route that call through a helper function that translates the interface reference into an object reference and calls the virtual method. For this helper the compiler uses the XCHG instruction that has an implicit CPU LOCK.
The new “-x-fvs” / “-x-fvs=1” option replaces the XCHG instruction with an alternative code and if the called virtual function doesn’t use the ECX register for the 3rd parameter, it generates a direct jump into the virtual method.
The “-x-fvs=2” option replaces the XCHG and uses ECX if available but if ECX is not available it keeps the CPU’s “return stack cache” valid by replacing the RET with a JMP instructions. For this it uses stack memory below ESP.

For some functions that meet special conditions the Win32 compiler emits stack frame code that fills the stack with zeros to clear variables with managed types. If there are too many of those the compiler uses a loop and the XCHG instruction to restore the ECX register that is used as the loop counter.
The option “-x-fpr” replaces the XCHG with an alternative code.

Changelog:

• Added: Option -x-ff to enable “fast floating point” (like Borland C++’s -ff command line option)
• Added: Option -x-fvs and -x-fvs=n to enable fast interface virtual stub (n=1: replace XCHG, n=2: keep the CPU’s return stack buffer in order)
• Added: Option -x-fpr to remove XCHG from the function prolog code.
• Added: DLL import table section folding and duplicate name/ordinal elimination, also for delay dll imports
• Changed: Split “Compiler64.X86″ patch into multiple smaller patches and removed the “Compiler64.X86” patch name
• Changed: EditorFocusFix now skips the SetActiveWindow call if the mainform (undocked) is not the active window

Download:

Download (fastdcc):

The new IDE Fix Pack version 6.0 is available. It supports Delphi 10.2 RTM and 10.2 Update 1. And after over a year of being in BETA testing without any bug reports, I also included all the Win64 compiler performance optimizations. Thus the jump to version 6.0 can finally be done as they make the Win64 compiler up to 50% faster.

Changelog:

• Added: Win64 compile speed optimizations
• Added: Delphi 10.2 Update 1 support
• Added: Editor Block Completion UTF8 fix (Delphi 2009 only)

Download:

Download (fastdcc):

Debugger Callstack Resolver is a Delphi IDE plugin that I wrote in 2011 to make the IDE’s CPU View more readable. It colors different instructions, resolves absolute and memory address jump and call targets and shows their function name if available. It also uses the *.jdbg to show more information (the dcc32 compiler’s jdbg files are for the debug build of the compiler and haven’t match the deployed version since jdbg files exist)

This plugin supports Delphi 2009 – 10.1 Berlin.

[Download]