I've got an idea for how you can do it with parse_many(json,window) and truncated_bytes() but it would be easier if there were just an example out there I could look at.

See pages like [0] and [1] that describe it should be possible but I am just not seeing (or yet able to produce myself) working code.

[0] https://github.com/simdjson/simdjson/issues/188

[1] https://github.com/simdjson/simdjson/blob/master/doc/iterate...

Simple - memory-map the whole file with `mmap`. We also suggest using `madvice`, to inform the kernel that you will be accessing the data just once, strictly in the sequential order. And then simply give it to SIMDJSON. With `ondemand` parser, your memory usage will be proportional to the depth of the deepest tree, not their number.

Harder way, would be to parse file in chunks, locating continuous sequences that form a single document.

There may be more options, but I'd have to check the docs :)

https://github.com/dolthub/jsplit

If you have any code examples of either of those options I'd love to see them!

Edit: Also, I'm curious: what are the downsides to doing the mmap route?

You can mitigate this somewhat with explicit MAP_POPULATE and mlock.

They're... kinda free (in terms of capacity). A PTE or PDE is 64 bits per page. Even with 4kB pages, this is 8/4096 = 0.2% overhead. Churning the page tables and TLB as you walk the file is expensive, or at least historically was expensive (used to be that removing an entry required flushing the entire TLB; I don't think that's true anymore).

If you can use MAP_HUGE_2MB, this drops off to 0.00038% and the TLB impact goes way down.

> no way of handling errors.

Yeah, this aspect is really significant. Thanks for mentioning it.

When you source the data, can you instead output the overall doc as "part" files that are aligned along subdocument boundaries?

THen you don't have to solve that "find the subdocument terminator" on stream-back.

You are PROBABLY building the json doc via some DB dump, so just change the code that dumps the data.

If you need to reconstruct it as a single json doc for some reason, it is about as trivial as it gets to reconstruct the data as a single document.

Or programattically, a "file" object that just does the reconstruction on the fly from the source part files as it is read, that should be relatively trivial in lots of languages.

If there is any interest, I can ask if they can to open source it.

Edit: Oh wow, I see you used it on those exact files. How about that.

And yep, it was more or less they way you did with ijson. I found ijson just a day after I finished the prototype. Rapidjson would probably be faster. Especially after enabling SIMD. But the indexing was a one time thing.

We have open sourced the codebase. Here's the link: https://github.com/multiversal-ventures/json-buffet . Since this was a quick and dirty prototype, comments were sparse. I have updated the Readme, and added a sample json-fetcher. Hope this is more useful for you.

Another unwritten TODO was to nudge the data providers towards a more streaming friendly compression formats - and then just create an index to fetch the data directly from their compressed archives. That would have saved everyone a LOT of $$$.

We needed it for streaming large json from async server sockets.

Code link: https://github.com/Edgio/is2/blob/master/include/is2/support...

You just had to implement the interfaces like Peek/Take/Tell/etc. It worked really well for us.

Probably not as fast as simdjson, but they used some simd tricks I think for skipping whitespace:

https://rapidjson.org/md_doc_internals.html#SkipwhitespaceWi...

If so, it’s easy in Python to do the following:

  with open(“file.json”) as src:
      for line in src:
          json.loads(line)

[0] https://github.com/simdjson/simdjson/issues/188#issuecomment...

Whatever remained at the end of the buffer (few hundred bytes of incomplete json) we would copy just before the boost-unzip block so that there was a continuation of json data. We also reused the parser and string buffer for performance.

Also try to parse the json fields in the right order for fastest performance, if possible.

What do you mean with this precisely, not sure I understand? What is the right order?

AFAIK, JSON attributes/keys are not ordered, so there is no "right" order, or order at all.

The docs on simdjson mention that if you parse the fields in the order as they appear on file, then simdjson can do it in one iteration. If you get fields in random order, simdjson will have to loop the data a few times.

If your json on disk is

  { "field1": "val1", "field2": "val2" }

  simdjson::document_reference elem;
  std::string_view tmp;
 
  elem["field1"].get_string().get(tmp);
  elem["field2"]...

See also [1] under "Extracting Values: You can cast a JSON element to a native type..."

[1] https://github.com/simdjson/simdjson/blob/master/doc/basics....

Since simdjson is where a lot of the performance benefits come from here, would you need to compile the underlying C/C++ code on the machine that you're deploying on to make sure that simdjson is using the correct instruction set? Like what if the processor I'm compiling on has AVX-512 support, but the target machine for deployment doesn't? Does it just generate the machine code for all instruction sets and then can automatically choose to use the optimal instructios at runtime? Or does it just have to compile every time you pip install the package? I could see this being awkward if you're building a docker container perhaps, but maybe I'm just woefully uninformed.

When compiling C/C++, with some compilers you are able to specify multiple architectures and provide a fat binary. It's been a few years now since I've worked on this sort of thing, but the Intel compiler for e.g. used to allow you to do something like this at the compilation stage

-march=avx,avx2,avx512

And at runtime, your processor would use the most recent instruction set. In practice, you don't always want to do this, since you produce fat binaries - i.e. every function call that's got vector instructions will have multiple versions, and the file size goes up, library is slower to load, etc... I can't remember if there was an overhead too to each function call. So instead of doing this, what is also common is to compile N versions of the library (one per instruction set), and then load the appropriate version at runtime. This you can do with any compiler, and is indeed what Intel do themselves with the MKL library - if you look into the package for it, you can see that you end up with multiple shared libraries, each with a suffix saying which instruction set it supports (e.g. libmkl_avx.so, libmkl_avx2.so, libmkl_avx512.so). I'm not familiar with simdjson so not sure which approach it takes.

Interestingly, if you use ARM processors, the SVE instruction set is designed so that you don't need to re-compile if a new processor family comes along with longer vector processing units.

[1]: https://gms.tf/riscv-vector.html

https://github.com/simdjson/simdjson

The question is about a performance benchmark. From a performance standpoint, rpc and rest are the same (http requests that execute code on the server).

The "semantics" you assign to one or the other don't change the technical details about implementation and performance.

> it's all just computers

> we're all just going to die anyway

What is the difference between remote.perform('some-action', { payload }) and remote.POST('some-action', payload) ?

Calling an URL with an action in the path name (rather than a resource name) is technically not done in REST -- the action is expressed through your HTTP verb.

Using GET/POST/DELETE directly does not mean you're doing REST - there's a whole set of rules and assumptions that come with it.

1 - https://github.com/squeaky-pl/japronto

To make it as fast as possible we don't use PyBind, NanoBind, SWIG, or any high-level tooling. Our Python bindings are a pure CPython integration. There is just no way to beat that combo, not that I know.

https://github.com/unum-cloud/ujrpc/blob/main/src/python.c

So, you optimized your C IO loop and it's really fast when there is not much Python...but soon as you add any Python, you'll be bottlenecking in Python and all that fastness wont matter...eg if you do some ORM SQL or any other thing that's more complex than `return data`

One thing people do - use two protocols. That is the case with Apache Arrow Flight RPC = gRPC for tasks, Arrow for data. It is a viable path, but compiling gRPC is a nightmare, and we don't want to integrate it into our other libraries, as we generally compile everything from sources. Seemingly, UJRPC can replace gRPC, and for the payload we can continue using Arrow. We will see :)

[1]: https://github.com/unum-cloud/ukv/blob/main/src/modality_doc...

I'm just an interested hobbyist when it comes to performant RPC frameworks but had some fun benchmarking capnproto for a small gamedev-related project and it was pretty awesome.

[1] https://capnproto.org/

[1]: https://arrow.apache.org/

anyone who disagrees I’d be very interested to hear your thoughts on alternatives.

Are data types useful for data to/fro web/mobile clients? Encode type into the column header?

Many parquet supporting libs will load Parquet files into an Arrow structure in memory for example.

The benefits of CSV are:

- human readable

- does not need to be typed (sometimes, data in the raw such as date-formatted data is not amenable to typing without introducing a pre-processing layer that gets you further from the original data)

- accessible to anyone: you don't need to be a data person to dbl-click and open in Excel or similar

The main drawback is that if your data is already typed, CSV does not communicate what the type is. You can alleviate this through various approaches such as is described at https://github.com/liquidaty/zsv/blob/main/docs/csv_json_sql..., though I wouldn't disagree that if you can be assured that your starting data conforms to non-text data types, there are probably better formats than CSV.

The main benefit of Arrow, IMHO, is less as a format for transmitting / communicating but rather as a format for data at rest, that would benefit from having higher performance column-based read and compression

See also: 'Why isn’t there a decent file format for tabular data?' https://news.ycombinator.com/item?id=31220841

It is true there will be exceptions-- such as if you know you only want to read the second half the file only. In that case CSV with quoting does not give you a direct way to find that halfway point without parsing the first half.

I suppose whether this is worth the other pros/cons will be situation-dependent. For my use cases, which are daily, CSV parsing speed, when using something like xsv or zsv, has just, by itself, never been a material concern/impact on performance.

Where I think the CSV parsing downside is much greater than the fact that it must be serial (but which as described above does not prevent parallelized processing), is in type conversion not just of numbers but in particular of dates-- it can be expensive to convert the text "March 6, 2023" to a date variable. However, if you have control over the format, you could just as easily printed that as an integer such as 44991 and reduces the problem to one of integer conversion. Which is still always going to be slower than a binary format, but isn't so bad performance wise.

In fact, the single-thread parser approach (with multi-thread processing) might even be better, because it is not trying to access your hard disk in 4 places at the same time. Then again, if your threads are doing some non-trivial task with each row, then IO will not be your bottleneck either way.

Obviously starts to break down if you aren't reading the whole file and you wanted to start some meaningful portion of the way in and never process what comes before it. The point is, the benefit of being able to, effectively, implicitly shard a file without saving as separate files-- might not be as impactful in practice as in theory

My mistake, I misread your answer!

FastAPI relies on (not so fast) pydantic, which is one of the slowest libraries in that category.

Don't expect to find such benchmarks on the pydantic documentation itself, but the competing libraries will have them.

[0] https://ltworf.github.io/typedload/

https://docs.pydantic.dev/blog/pydantic-v2/

Anyway we will see when it comes.

This week at work I was just appreciating that typedload just works with static type checkers without having to install plugins (and all the type errors of pydantic won't be reported unless you do install the plugin).

[0] https://github.com/ibireme/yyjson

[1]: https://github.com/jcrist/msgspec

[2]: https://github.com/TkTech/json_benchmark

Is this supposed to be just a blank page, which is what I see?

I'm picking a protocol for my project. I was looking at protobuf, but this post made me think otherwise.

I know about fast_float (which is by the same author as simdjson), however my believe is that binary format is impossible to beat here.

However, your use case is likely doesn't need to process multi-gigabit traffic in JSON requests. Therefore, protobuf, or more specifically, gRPC will be just fine.

> Performance wise, simdjson-go runs on average at about 40% to 60% of the speed of simdjson. Compared to Golang's standard package encoding/json, simdjson-go is about 10x faster.

I haven't tried it yet but I don't really need that speed.

https://github.com/minio/simdjson-go