JSON parser JsonDataObjects is now on GitHub

By | February 15, 2015
30 Comments

It all started with a new DDevExtensions feature I’m working on, or was working on before my JSON parser project took over. For this new feature I needed to write and read some information to and from a file. XML was out of question, so I came up with a “one line per entry” format. As the feature grew I “suddenly” needed to write more structured data. After trying to do that in a “one line per entry” format the I/O code became such a mess that I deleted it and came up with the idea to use JSON for the file.

After browsing the web and looking into the different JSON parsers for Delphi, I decided that I didn’t need such complicated and feature rich parsers and it would be fun to write my own parser. So I started, not knowing that my class names would be the same of Delphi’s JSON parser, because I never looked into it before I started benchmarking my parser against it.

And here we are. The DDevExtensions feature is on halt and my brand new JSON parser is available on GitHub. It uses a lot of low level tricks to get as much performance out of the different Delphi compilers and RTL and beats Delphi’s JSON parser in performance and memory usage, by far.

Features

  • Fast dual JSON parser for parsing UTF8 and UTF16 without conversion
  • Automatic creation of arrays and objects
  • Easy access mode with implicit operators
  • Compact and formatted output modes
  • Win32, Win64 and ARM Android support (MacOS and iOS may work)
  • Supports Delphi 2009-XE7

Some internal tricks:

  • Elimination of unnecessary _UStrAsg and _UStrClr calls, reduces CPU locks
  • ARC: Faster ARC handling by doing it by hand without the virtual method calls (RSP-9712)
  • Storing string literals where possible instead of creating a heap string for them
  • Caching the last string literal pointer for faster repeated name access. Broken in XE7 (RSP-10015)
  • Specialized own StringBuilder implementation that doesn’t need to copy the buffer to the final string
  • Fast String to Int64 conversion implemented in assembler for x86 and x64
  • Fast String to Double conversion

30 thoughts on “JSON parser JsonDataObjects is now on GitHub”

  4. Alexandr

    Greetings!
    Why just dont use the SuperObject JSON parser? It alreay do theese things, even more, it can serialize/deserialize the RTTI context.

    Your parser more perfect than their?

    1. Andreas Hausladen Post author

      It is fun (for me) to write my own JSON parser. I needed something that works in Delphi 2009 and Android. So no RichRTTI. And generics are a no-go because Delphi 2009’s generics are unusable.

      Why do I use Delphi? C++ can do what Delphi does and even more. 😉

  5. Pingback: Flotsam and Jetsam #95 | Delphi-losophy

  6. RjK

    Wow nice fast parser.

    But I think I found a bug.
    Json.S[‘Name’] := ”;
    This will result in : {“Name”:}
    Missing the double quotes after the colon.

  8. karim kouni

    Hi Andreas,

    Nice work as always. One thing though. It seems that the parser does not recognize object fields. I made a comparison between your parser and TJSON using an object with dynamic arrays and double field types. TJSON returned the correct JSON string while your parser returned just { }. Any idea on what I am doing wrong here.

    Thanks and keep up the good work.

    K.

    1. Andreas Hausladen Post author

      Did you use ToSimpleObject/FromSimpleObject? Those functions only work with the old TypeInfo and as a result don’t support array and dynamic arrays.

        1. Andreas Hausladen Post author

          JsonDataObjects supports Delphi 2009 and I haven’t had the time, and it is low priority to me, to implement Delphi 2010+ RichRTTI functions.

      2. karim kouni

        The object has also double and integer fields but nothing is returned only {}.

  9. Marcelo

    Andreas,

    Do you think it will work with C++Builder?

    I’m been lazy to ask you before work with it, but my idea is to report this test if you agree with it.

    Marcelo.

    1. Marcelo

      Hi,

      Sorry about my first question.

      It is working with C++Builder:


      TJsonObject* Obj;
      try
      {
      Obj = dynamic_cast(
      TJsonObject::Parse("{ \"foo\": \"bar\", \"array\": [ 10, 20 ] }") );
      ShowMessage(Obj->Values["foo"]);
      ShowMessage(IntToStr(Obj->Values["array"].Count));
      ShowMessage(IntToStr( int(Obj->Values["array"].Items[0]) ));
      ShowMessage(IntToStr( int(Obj->Values["array"].Items[1]) ));
      }
      __finally
      {
      delete (Obj);
      }

    2. Marcelo

      And…


      TJsonObject* Obj;
      TJsonObject* ChildObj;

      Obj = new TJsonObject();
      try
      {
      // easy access
      Obj->S["foo"] = "bar";
      // normal (and faster) access
      Obj->S["bar"] = "foo";
      // automatic array creation, Obj is the owner of 'array'
      Obj->A["array"]->Add(10);
      Obj->A["array"]->Add(20);
      // automatic object creation, 'array' is the owner of ChildObj
      ChildObj = Obj->A["array"]->AddObject();
      ChildObj->F["value"] = 12.3;
      // automatic array creation, ChildObj is the owner of 'subarray'
      ChildObj->A["subarray"]->Add(100);
      ChildObj->A["subarray"]->Add(200);

      ShowMessage(Obj->ToJSON(/*Compact:=*/false));
      }
      __finally
      {
      delete (Obj);
      }

  10. 872036709

    // I found the following code have bug.
    procedure TStackStringBuffer.AppendChar(Value: Char);
    var
    P: PChar;
    begin
    P := FBufPos;
    if P = @FBuf[MaxBuf] then
    Done;
    P^ := Value;
    Inc(P);
    FBufPos := P;
    end;

    procedure TStackStringBuffer.Append2Char(Value1, Value2: Char);
    var
    P: PChar;
    begin
    P := FBufPos;
    if P + 1 >= @FBuf[MaxBuf] then
    Done;
    P^ := Value1;
    Inc(P);
    P^ := Value2;
    Inc(P);
    FBufPos := P;
    end;

    // following is I modify the code.
    procedure TStackStringBuffer.AppendChar(Value: Char);
    begin
    if FBufPos >= @FBuf[MaxBuf] then
    Done;
    FBufPos^ := Value;
    Inc(FBufPos);
    end;

    procedure TStackStringBuffer.Append2Char(Value1, Value2: Char);
    begin
    if FBufPos + 1 >= @FBuf[MaxBuf] then
    Done;

    FBufPos^ := Value1;
    Inc(FBufPos);
    FBufPos^ := Value2;
    Inc(FBufPos);
    end;

  11. RjK

    Found a memory leak when compact is false

    .ToJSON(False)

    An unexpected memory leak has occurred. The unexpected small block leaks are:
    13 – 20 bytes: Unknown x 3
    21 – 28 bytes: Unknown x 1

    1. 872036709

      function TJsonOutputWriter.Done: string;
      var
      i : Integer;
      begin
      if not FCompact then
      begin
      FlushLastLine;
      for I := 0 to FIndentsLen – 1 do
      FIndents[i] := ”;
      FreeMem(FIndents);
      FLastLine.Done;
      end;

      if FLines = nil then
      FStringBuffer.DoneConvertToString(Result);
      end;

  12. Pingback: Blazing Fast JSON Library For Delphi XE7 Firemonkey On Android And Windows | Delphi XE5 XE6 XE7 Firemonkey, Delphi Android, Delphi IOS

  13. Vojslav

    Hi,

    it’s possible to encode array of array, eg something like:
    {“offer”:{“Points”:[[353,977],[680,977],[680,1407],[353,1407]]}

    Thanks Vojslav

  14. Larry

    XE7 build gives some errors:

    [dcc32 Error] JsonDataObjects.pas(1965): E2036 Variable required
    [dcc32 Error] JsonDataObjects.pas(3755): E2036 Variable required
    [dcc32 Error] JsonDataObjects.pas(3760): E2036 Variable required
    [dcc32 Fatal Error] BCEditor.Highlighter.JSONImporter.pas(16): F2063 Could not compile used unit ‘JsonDataObjects.pas’

Comments are closed.