Related to that, last year Uber's engineering blog mentioned very interesting results with their internal log service [1].
I wonder if there's anything as good in the open-source world. The closest thing I can think of is Clickhouse's "new" JSON type, which is backed by columnar storage with dynamic columns [2].
The design described there is what Uber should be logging in the first place. Instead they are logging the fully resolved message and then compressing back into the templated form.
However, the compression back into the templated form is a good idea if you have third party logs that you want to store where you can not rewrite the logging to generate the correct form in the first place.
Neat! The only downside of this approach is having to force the developers to use the library, which can work in some companies. On the other hand, other approaches discussed previously like Uber's don't require any change in the application code, which should make adoption way simpler.
