I can only second this. The placeholder in the search bar is “ex: tacos, breakfast, …” so I searched for tacos - zero results. Having a bit more content would likely leave a better first impression.
So, we were trying to display the recipe creation software but I realized now that we should upload some of the couple hundred recipes we have. I have just been mainly sharing this with engineering friends to get input on the software, not the recipes.
But, I'm going to begin uploading them all over the next few days. I am also getting involved with people who have more recipes to upload some of their own.
You would help us in designing, building, improving and operating services that process several trillion event per - coming with an "interesting technical challenges included" guarantee.
At remerge we put culture and people first:
- we are fully transparent (from salaries to company financials)
- clear role design and a growth framework - so you know where you stand, what's next and how to get there
- our renumeration includes share options, an education budget
and a percentage of the company profit as end of year bonus
- unlimited vacation days (with a minimum you have to take) + "fully covered one month work from any office" program
- work from home/remote support (setup + internet reimbursement)
We used Apache Drill a few years back and it could do exactly that - run locally and provide a SQL interface to query parquet files. Still great to see more and better options!
Drill is a fair choice (it supports tons of formats), but for me it was a bit heavy -- it required Java and plus it returned results in a form that wasn't easily manipulated (the result is a table in the console drawn with text-chars [1]). If you had a really long result set, you couldn't do a buffered scroll due to text UI limitations -- everything gets printed out and you have to either do manual pagination using LIMIT clauses or just have a really long scrollback buffer. This kind of breaks the continuity of many data analysis workflows.
With DuckDB, the return result is a Python data structure (list of tuples), which is amenable to further manipulation without exporting to another format.
Many GUI db tools often return results in a tabular form which can be copy-pasted into other GUI apps (or copy-pasted as CSV).
That looks great, especially some of the unique optimisations. I gave it a nonscientific test run with a set of 1k different 2-4kb JSON encoded messages that we saw in our day to day traffic using the default Go benchmark library. Compared to easyjson (generated parser) goccy/go-json is unfortunately 20-25% slower and allocated four times the number of bytes.
Any chance that you will implement code generation like easyjson?
Are you unmarshaling to interface{}?
Have you tried commenting out the easyjson json.Un/Marshaler implementations? By default they wrap the easyjson code and add reflection overhead.
I had the same results when testing with a project that is already using easyjson, but after commenting out easyjson's json.Un/Marshaler impls I am seeing much improved performance using goccy/go-json.
Indeed. Thanks for the hint! While the allocations are still 2.5x, not by accident calling into easyjson makes goccy/go-json perform faster than easyjson with generated code by ~10%. That is some impressive result.
Agree! Both reports are a pretty good and reflect the status quo in the industry. There is an overcollection and oversharing of data without proper consent and that has to stop. For Europe - forcing the ad-tech industry to adhere to the GDPR is the correct next step as self regulation did not work.
Having said that the report paints a pretty dark and one-sided picture. Let's see how far the authorities will follow their argumentation / conclusion.
We wrote our bidder (in app advertising) in Go. It is globally distributed (close to the exchanges) and handles 1.5-2M requests/s (OpenRTB,~50-90k/s per instance) with a p99 of 10-20ms (without network latency). Really happy with Go, especially the GC improvements done by the Go team in the last few releases. For a previous similar project we used Ruby which was quite a bit slower.
Like every engineering decision it is a risk/reward spectrum. But unless you have profiled to know that net/http is your bottleneck, no, you should almost certainly not use fasthttp.
