Directory cache for the compiler – aka IDE Fix Pack 4.3 Beta

By | June 26, 2011

Now that IDE Fix Pack 4.2 is out, I can finally give you an experimental “directory cache” for the Delphi compiler (Delphi 2009/2010/XE). Let’s see how fast that makes the compilation.

UPDATE: The directory cache is built by using FindFirstFile/FindNextFile before the compiler starts compiling and is discarded after the compiler has finished its work. The cache is not written to the harddisk.

The 2007 version doesn’t contain the directory cache, it only fixes QC #40945: Right ALT-key causes exception under Vista.

Name IDE Version File Size Downloads Added
DelphiSpeedUp 3.1-untested RC for Delphi 6 6 DelphiSpeedUpV31D6.7z 74.59 KB 1220 times 2011-04-22
DelphiSpeedUp 3.1-untested RC for Delphi 2006 2006 DelphiSpeedUpV31D2006.7z 106.03 KB 1309 times 2011-04-22
DDevExtensions 2.4.2 7/2007 speed 7, 2007 DDevExtensions242Setup7_2007.zip 535.43 KB 3548 times 2011-08-02

32 thoughts on “Directory cache for the compiler – aka IDE Fix Pack 4.3 Beta

  1. Thomas Speck

    Hi Andreas,

    for me, this is an unbelievable Boost. Together with my OCZ RevoDrive X2, I compile now on the Fly, the Programs start without any Delay. Another unbelievable Work that you have done. You are an genius and I always appreciate your Work you do for the Delphi Community, you are my Delphi Hero !

  2. chaosben

    Where can i configure the directory for the cache?

  3. Sharken

    I made some early tests on some of our projects. Harddisk is SSD and Delphi is running within Vmware.
    Interestingly the slowest part of the compiling is now related to Generics and Serialization, where the DeHL.Serialization unit is used.

    Results

    				v4.3		v4.2
    ProjectGroup #1			23s		35s
    Single project in group #1	14s		19s
    Project #2			16s		20s
    Project #3			4s		6s
    Project #4			12s		14s
    Project #5			2s		4s
    
    1. Mason Wheeler

      Yeah, DeHL.Serialization is a big mess. (Even Alex is willing to freely admit this!) He said he was working on a lighter, better serialization concept for Collections, but I’m not sure how much progress has been made on that.

  4. gabr

    Results from the project I’m currently working on (‘build all’ of 5 projects in the project group, 4 M lines total). Files are on a SSD drive.

    Delphi XE: 66 sec
    Delphi XE + DDevExtensions : 43 sec
    Delphi XE + DDevExtensions + IDEFixPack 4.2: 41 sec
    Delphi XE + DDevExtensions + IDEFixPack 4.3 beta 1: 33 sec

    Whooping 100% improvement over barebones XE!

  5. nedko

    my timings:

    XE w/o IdeFix: 41 sec
    XE w/ IdeFix: 19 sec

    so compile time is ~50% reduced. Great!

  6. David Robb

    Great work Andreas! Building on an OCZ RevoDrive here, under Win7 64 with 6GB RAM; the difference was similar to what has been reported by others.

    All times are averages of three builds. It seems that with IDE Fix Pack 4.3 build times were very consistent. The biggest differences in build times were seen with XE standalone. XE frequently spent a huge amount of time thinking about something *before* it started compiling.

    XE = XE standalone without DDevExtensions or IDE Fix Pack.

    Project 1. Application built without runtime packages, 2.1 MLOC.

    XE – 1m 38s.
    IDE Fix Pack 4.2 + DDevExtensions – 1m 22s.
    IDE Fix Pack 4.3 + DDevExtensions – 53s.

    Project 2. Application built with runtime packages. 575 KLOC.

    XE – 39s.
    IDE Fix Pack 4.2 + DDevExtensions – 20s.
    IDE Fix Pack 4.3 + DDevExtensions – 16s.

    Project 3. Project group containing about 70 packages. 1.67 MLOC.

    XE – 1m 15s.
    IDE Fix Pack 4.2 + DDevExtensions – 1m 15s.
    IDE Fix Pack 4.3 + DDevExtensions – 1m 13s.

    In the last project group, it seems that I could see improved compilation performance at the beginning of the build but it seemed to slow down roughly linearly as it went farther down the group. I believe this happens because the packages at the bottom of the project group have many more inter-package dependencies than the packages at the top of the project group. I see the difference even if I build the last three packages by themselves.

    Thanks a lot, Andreas! This makes day-to-day work much more enjoyable.

  7. Eric

    Excellent! Working on a projects with source organized in lots of directories, and the speed up is very noticeable!

  8. David Robb

    Andreas,

    I’m sorry to report that source breakpoints are not available with IDE Fix Pack 4.3 when compiling with runtime packages enabled.

    To troubleshoot this, I created a console app with “Build with runtime packages” enabled, with the standard package list, added a single Writeln statement to it (between the try/except), set a breakpoint on the Writeln, and ran the program.

    With IDE Fix Pack 4.2, the IDE goes to the breakpoint line. With IDE Fix Pack 4.3, it opens the CPU window. This behavior was the same across multiple applications.

    I tried using SysInternals’ ProcMon to see if there was any pattern to how BDS accesses files, but I couldn’t see any differences.

    Sorry,
    Dave

      1. FreeMan

        Andreas,
        Than you very much for your tools,
        This error fixed it. But showing still “Beta 1” on delphi splash screen
        best regards

  9. Julio Furquim

    Hi Andreas,

    Under version 4.3 just doing File->New->VCL Forms Application -Delphi and run it, I get a “File not found: Project1.res” error. This does not happens under version 4.2. Tested on Delphi 2010.

    Thanks for your excellent tools.
    Julio Furquim

    1. Jen Smith

      Getting this issue as well (XE), although it’s pretty minor, only seems to happen with a new unsaved project. The speed improvements are amazing however.

  10. Natalie Vincent

    Hey Andreas,

    The download for Beta 2 still contains the Beta 1 files as far as I can see.

    Regards,

    N@

  11. Roman Ganz

    I sometimes get an AV at Address 0000 and sometimes a Invalid Pointer Operation when I try to compile a project. The Error appears directly after “Checking project dependencies…” Message in the Build Log.

    1. Roman Ganz

      There’s another Issue: when I try to compile a empty new VCL Project, I get a “[DCC Error] E1026 File not found: ‘Project1.res'”. When I save the project, it compiles perfectly

  12. Markus Imhof

    Thank you for this amazing tool!

    Can you give us (me) some more specific informations about the creation of the cache? Is it really only to call the FindFirst/FindNext functions to force Windows caching the specified directorys?

    1. Andreas Hausladen Post author

      Before the compilation starts IDE Fix Pack 4.3 Beta reads all directories, that the compiler needs to access to find files, into a hash table. And when the compiler wants to know where a file is, the hash table is consulted instead of probing all library paths.

  13. Markus Imhof

    Thanks for the information.

    The tool is really excellent, but sometimes I wish back the good old days of the 80s, when I could tell to my boss, I must make a break for half an hour , I just compile my code 😉

  14. fabio

    Hi Andreas, discovered a little problem in 4.3 beta 2 for Delphi XE

    Steps to reproduce:
    1. Tools, Options…

    Get the following exception:

    compiled with : Delphi XE
    version : 15.0.3953.35171
    madExcept version : 3.0m
    callstack crc : $f7f75d96, $8fd69e4d, $97ef7702
    exception number : 1
    exception class : EAccessViolation
    exception message : Access violation at address 5003C49A in module ‘rtl150.bpl’. Write of address 53909850.

    but then it continues fine opening the option form.

      1. fabio

        Yes Sir I have JCL VersionInfo expert installed!
        Do I need to uninstall it?
        After all I think I can live without it, but not without your IDE Fix Pack!

        1. Andreas Hausladen Post author

          The problem is that only the JCL VersionInfo expert causes this bug with IDE Fix Pack. But I can’t reproduce it here.

          1. fabio

            I’m looking how to remove JCL VersionInfo expert from my IDE ?!?!?!? 😉

          2. fabio

            OK SOLVED!
            I’ve found the offending bpl’s:

            1. menu Components, Install Packages…
            2. uncheck the following bpl’s:
            – JCL Debug IDE extension
            – JCL Stack Trace Viewer

            After having disabled the above bpl’s, no more IDE exceptions when opening Tools, Options…

            Maybe in my specific case there is a collision or interference between MadExcept and JCL Debug.

            I prefer to stay with MadExcept.

          3. fabio

            I was wrong: after restarting Delphi Xe the exception still shows when opening Tool, Options…

            What is strange is that if I:

            1. menu Components, Install Packages…
            2. check again the following bpl’s:
            – JCL Debug IDE extension
            – JCL Stack Trace Viewer
            3. save

            then:

            1. menu Components, Install Packages…
            2. uncheck the following bpl’s:
            – JCL Debug IDE extension
            – JCL Stack Trace Viewer
            3. save

            I got no more exceptions when opening Tools, Options… but this is until I close the delphi session.

            When I restart Delphi, I have the exception again.

            So, since as I stated in my first post the exception is not a blocking one, I can live with it.

            Anyway, thank you very muck Andreas for this as for all others plugins that you kindly donated to the community!

            fabio

  15. FreeMan

    Hello Andreas,
    This is my ask and / or suggestion.
    On my test Delphi XE has a IDE bug on “Project” menu “language”, “Update localized projects”.
    When change in project option “Output directory” .exe file compiled to there. and save this directory to project of “.dproj” like this:

    e:\EXEs\

    Language site has a problem,when compile or build no problem, but if set output directory, then “Update localized projects” IDE add this directory to language project(s) “.bdsproj” for example “myproject_ENU.bdsproj”E:\EXEs\MyProject.ENU this is maybe normal. but problem is when close all and reopen project group and then compile or build main project, exe file created to added aoutput directory, but IDE when try to build language output file name(.enu or etc.) add lanuage project source path and language project ‘s(.enu) files “targetname” to output file(in ENU.bdsproj). for language output file path is been “C:\MyProject\Lang_exam\ENU\E:\EXEs\MyProject.ENU” so language file cannot create, compile or build breaking with fatal error.

    delete main project output directory, click “Update localized projects” and then reopen project group build or compile ono error, just output file same on .dpr file

    Is it possible fix this error ?

    Best regards

Comments are closed.