I took a look at the diff linked in the article with code that "we are all running". The top of the diff certainly looks interesting. They remove the bounds check in dict_put() and add a safe version dict_put_safe().
This kind of change is difficult to make without mistakes because it silently changes the assumptions made when code calling dict_put() was originally written. ALL call sites would need to be audited to ensure they are not overflowing the dictionary size.
if you can get one call of dict_put somewhere to pass the limit, all later calls of dict_put_safe will happily overwrite memory and not actually be safe.
Agree, nice catch. Also, there are many other opportunities in this patch to hide memory safety bugs.
This is the kind of optimization I might have done in C 10 years ago. But coming back from Rust, I wouldn't consider it any more. Rust, despite its focus on performance, will simply not allow it (without major acrobatics). And you can usually find a way to make the compiler optimize it out of the critical path.
I think at this point is clear that everybody has to assume that XZ is completely rotten and can no longer be trusted. Is it XZ easy to replace with some other compression tool? Or has it been so widely adopted that is going to take huge effort moving out of it?
There is no reason to assume that. Even if you assume every commit since Jia became a maintainer is malicious, the version from 3 years ago is perfectly fine.
Zstd has a number of benefits over Xz that may warrant its use as a replacement of the latter, and this will likely be a motivating factor to do so. But calling it entirely rotten is going way too far IMO
There is an interesting argument to be made that pre-JT xz code is probably pretty secure due to the fact that the threat actors would have already audited the code for existing exploits prior to exerting effort to subvert it.
I always use "zstd --long=31 -T0 -19" to compress disk images, since that is a usecase where it generally offers vastly superior compression to xz, deduplicating across bigger distances.
XZ offers slightly better compression on average, but decompression is far slower than Zstd.
IIRC memory consumption is generally worse for Zstd at comparable levels of compression. Which, these days, is generally fine, but my point is you can't thoughtlessly substitute the two.
It's not enough to write a law on principles alone. It must be clear and practical to comply and clear how it will be enforced. The EU should not have created a situation where the most practical solution for 1000's of companies is a cookie banner.
I would assume that the Debian _maintainer_ cares about what is being shipped with Debian. And for each dependency (transitive or not), the Debian maintainer has to make sure that it is shipped by Debian, and therefore has to not only know about the dependency, but also to make a package out of it (if it is not already being shipped by Debian).
Whereas the developer of a popular Rust library most likely just added some dependencies that were convenient, and doesn't know the full story of the transitive dependencies.
To me it is much more likely that somebody at Debian _saw_ the dependency being shipped than an arbitrary Rust developer, who probably does not even know how many dependencies are being shipped with their program.
because it often takes so long to do any updates that people just give up on updating in time, weather that is for bug fixes or for injecting supply chain attacks /s/j
Debian developers go through a long step-by-step process to get in that position. Volunteering as a package maintainer first, getting Debian dev to advocate for you, cross-checking people passports, doing interview/exams to become a DD.
DDs review the libraries they package. All uploads are personally signed to keep people accountable.
Instead opening a github account and developing a library can be done anonymously and there's been supply chain attacks done this way.
I'm not sure how one would define "adaptation" (presumably meaning "adoption", ie: how widely used something is) in a way that could have debian have a higher adoption than rust: rust ships in programs within many linux distributions and other operating systems (mac os and windows). rust is used in widely used programs/kernels (Firefox, Chrome, Linux itself, Windows) that span further than debian itself.
All this is to say: a large amount of debian users likely are using rust, and an even larger amount of non-debian users is likely to be using rust.
> Kubernetes has done a good job of replacing OTP.
I feel like this statement alone demonstrates a wild misunderstanding of what OTP does and is. Kubernetes is at best a replacement for VM-level scheduling, but without a slew of additional functionality and application-level interaction with the kubernetes runtime, individual servers are basically running blind in a container, totally unable to do anything that OTP provides to gen_servers for free.
I personally don't find this binding once behaviour problematic in any way, but Elixir specifically addresses this (it creates another variable with the same name, though)
I will post the same answer from before: I like the explicitness of pattern
matching. In Elixir, as soon as I see ^foo, I know I am matching. If there is
no ^, I know the previous value regardless if it there is one or not, will be
discarded. If pattern matching is not explicit, I always need to know if a
variable was previously defined or not to know what is going to happen. To me
this behaviour is non negotiable. In my experience, it is more likely to run
into accidental matches than into accidental rebindings.
Another possible limitation to the suggestion above can be related to macros.
Let's suppose you have a macro that stores a value in a hygienic variable:
defmacro do_something(a, b) do
quote do
var!(hello, Hygienic) = calculate_something(unquote(a), unquote(b))
end
end
Someone may call this macro multiple times but we always care about the last
value
of hello. If we make a distinction in between being assigned once and then
multiple times, the macro simply won't work. Or the macro developer would need to
work around this behaviour by inspecting the environment or we would need to
provide some sort of functionality that does it for you. Maybe this behaviour
could be built-in in var! or maybe we'd need to introduce something like:
defmacro do_something(a, b) do
quote do
set!(hello, Hygienic) = calculate_something(unquote(a), unquote(b))
end
end
We can see how this is getting complex:
1. Use = to define variables
2. Use ^ for pattern matching
3. Use := for rebinding
4. Use set! in macros when you don't care if a variable was previously defined or not
It is also interesting to point out that the := operator won't work for rebinding
inside clauses. For example, how would you make the rebinding below explicit?
x = true
case false do
x -> :ok
end
Of course, there are interesting consequences for making a distinction in between
defining and rebinding a variable by introducing something like the := operator.
We could have better control of the scope to provide better warnings or even make
it easier to implement imperative for loops:
x = 0
for i <- 1..5 do
x := x + i
end
x #=> 15
But the fact Elixir have different ways for variables to be introduced in the
scope, adding more rules can make the overall system very complex.
As the engineering manager you should implement an automated code formatter in your CI checking. I think clang-format will do this for C. Doing this simply removes this whole class of conflict from your team.
This is a program that you run which will deterministically re-format your whole codebase. The CI checks on a PR should not pass until the PR diff matches what the formatting program says is the correct formatting.
Plyable | Graduate Research Engineer, Full Stack Engineer | Oxford, UK | Full-time | ONSITE
At Plyable we are automating to the composites manufacturing process. We have started by automatically pricing and designing molds from a part design. We make parts which go on superyachts, electric cars and aeroplanes.
We have found a good product/market fit and have the ambition to take our automation to the next level.
Mouldbox are adding automation into the composites manufacturing process. Our technology helps to make moulds for parts which go on superyachts, electric cars and aeroplanes. Founded in 2018 Mouldbox have raised venture capital funding in 2019 and has received government grants. We are a team of 6 and expect to double our headcount in the next year.
Feel free to reach out directly to me if you have any questions (I'm the CTO), a.lofts@mouldbox.com
This kind of change is difficult to make without mistakes because it silently changes the assumptions made when code calling dict_put() was originally written. ALL call sites would need to be audited to ensure they are not overflowing the dictionary size.
The diff I am referring to is here:
https://git.tukaani.org/?p=xz.git;a=commitdiff;h=de5c5e41764...