What I think is funny is that circa 2008 I had a manager who used to work at Amazon who told me that "a surprising amount of Amazon artificial intelligence is artificial artificial intelligence, low paid workers".
I heard this was behind mechanical turk. Sounds like the playbook remained the same.
> As a setuid program meant for elevating privileges, all code that is compiled into sudo-rs has the potential to accidentally (or intentionally) give access to system resources to people who should not have that access. The setuid context additionally puts some constraints on how code is executed, and dependencies might not have accounted for that context. We could not expect any of our dependencies to take into account such a context either.
This is the real problem. I've come to the conclusion that setuid programs basically shouldn't be using most libraries. The setuid environment is just fundamentally different. A normal library can have a debug output file who's location is controlled by an environment variable without that being a security risk. But the instant that program becomes setuid, that's an arbitrary file overwrite security bug. Most libraries aren't built with that in mind. They shouldn't have to be. Setuid is poorly designed.
I used it and even took a class taught by salt stack employees.
I really was not a fan. I think one of the core problems I ran into was that the tool attempted to layer on rpc behavior to what was fundamentally a broadcast.
So you'd apply something to everything with a web role. The tool would attempt to make it appear as if you executed apply on n nodes and return after n responses. But since this was all just a broadcast what really happened is the tool dropped a message on the bus and then guessed how many responses it would get. If it guessed wrong the cli invocation would just hang.
I'm not sure if I follow but it does occur to me that being thrifty and changing from a aerospace engineering company to a financial engineering company can be two different things that might end up categorized as "accountants are in control".
This is a very sharp tool and I find it's really rare to need it.
I do alot of profiling and performance optimization. Especially with go. Allocation and gc is often a bottleneck.
Usually when that happens you look for ways to avoid allocation such as reusing an object or pre allocating one large chunk or slice to amortize the cost of smaller objects. The easiest way to reuse a resource is something like creating one instance at the start of a loop and clearing it out between iterations.
The compiler and gc can be smart enough to do alot of this work on their own, but they don't always see (a common example is that when you pass a byte slice to a io.Reader go has to heap allocate it because it doesn't know if a reader implementation will store a reference and if it's stack allocated that's bad.
If you can't have a clean "process requests in a loop and reuse this value" lifecycle, it's common to use explicit pools.
I've never really had to do more than that. But one observation people make is that alot of allocations are request scoped and it's easier to bulk clean them up. Except that requires nothing else stores pointers to them and go doesn't help you enforce that.
Also this implementation in particular might not actually work because there are alignment restrictions on values.
I love and hate this. On a visceral level I don't like dealing with issues via malicious compliance.
OTOH I absolutely agree this is a good call if your goal is for more users to be able to run more software on your platform, even if printing is broken.
I do like this about go. And on Linux it arguably makes sense (I say arguable because DNS without CGO is still a common cause of issues and incompatibility).
But Linux has, to the best of my understanding said "yes, we are ok with users using syscalls". Linux doesn't think that glibc is the only project allowed to interface with the kernel.
But for other platforms like OpenBSD and windows they are quite simply relying on implementation details that the vendors consider to be a private and unsupported interface.
This whole thing is also separate from "is making libc the only caller of the syscalls instruction" a good and meaningful security improvement.
> I say arguable because DNS without CGO is still a common cause of issues and incompatibility
DNS without CGO works perfectly. The vendor specific ad hoc mechanisms for extending DNS in a site local context are not well supported. If they were implemented more sensibly, then Go, or any other language, would have no problem taking advantage of them even without the "C Library Resolver."
Speaking of which, that "C Library Resolver," in my opinion, has one of the worst library interfaces in all of unix. It's not at all a hill worth new projects dying on.
> The vendor specific ad hoc mechanisms for extending DNS in a site local context are not well supported
DNS is one of those things that OS vendors think should be extendable and configurable. It allows VPN apps to redirect DNS only for certain subdomains, for example, which enables proper split-horizon DNS. I think this is totally reasonable behavior, and it’s undeniably useful. If a particular programming language reimplements DNS on its own, you lose guarantees that the OS is striving to provide to the user.
You can make the case that OS’s shouldn’t make these guarantees, and we’re free to disagree on that, but from a practical standpoint it is a very useful feature and it sucks that pure Go apps don’t work with it.
Or you make the case that the OS should ship a DNS server. It can handle DNS forwarding and iterative queries anyway (iterative queries as you often need them for dnssec) and the logic for serving DNS responses is ok'ish (DNS name dedup and length restrictions being the biggest complexity issue).
I am not super happy with systemd-resolved but it solves this particular issues. No requirement to use libc, but same (os configurable) behavior for all users.
The difficulty and overhead of running those queries from everybody's phones is one of the reasons (a small reason, but one of them) DNSSEC isn't deployed.
It does not. I know this because it impacts my daily work and the work of others. Honestly if you could make my day and go figure out exactly what's going wrong with the pure go DNS implementation it would make my life alot simpler and I wouldn't have to maintain shell scripts that update etc/hosts to hard code in ipv4 addresses for the APIs I access with terraform.
It seems like the explanation might be right there in that issue. The server is occasionally not successfully sending the A record and you're getting bad fallback behavior due to the way the dial call is imprecisely constructed and this is masking the underlying problem. The code, as written, is doing exactly what you would expect in this scenario.
Should it be your DNS resolver library that takes stock of your OS environment and only make calls for A records and not AAAA records when it "detects" some configuration?
Shouldn't your application itself have an environment variable or command line option that allows you to specify that your dials should only be done using tcp4? Wouldn't this be immensely useful to have outside of "auto detection" in some library somewhere?
Why should every application be aware of whether it is running on a v4 or v6 network? If the application merely wants to connect to an external service, that is firmly in the OS's job to decide.
As a user, if ipv6 is flaky today, I want one central place to configure for ipv4 only, I don't want to go and change every application'settings only to revert that tomorrow.
Oh.. right.
So we need another configuration for each and every application for something almost always system wide and can change dynamically (like wifi reconnecting) ?
Most unixes allow for static binaries. And most unixes allow for you to run a static binary compiled for an older OS on a newer OS. In order for that to work, the syscall interface needs to be at least mostly stable.
Yes, if you're writing netstat or lsof or ps or something, you need tight coupling with the binary and the kernel, and you can argue Linux does that better, but most people aren't writing netstat or lsof or ps.
On the BSDs they sure don't, /proc (for its role referred to) and /sys are peculiar to Linux, /proc is not even mounted by default on FreeBSD, is deprecated and doesn't exist on OpenBSD. On OpenBSD/FreeBSD most of the info comes out with sysctl(3) or reading kernel memory directly.
Emphasis on most of the sundry information for the live kernel now comes from sysctl, I note the (root only) mem/kmem interface for completeness and rare utilities (eg btsockstat) use it.
I do not know what the previous poster was talking about, but, while unrestricted access to kernel memory is of course unacceptable, a perfectly secure and useful means for the kernel to provide information to a user process is to map a page as read-only in the user address space, from where the user process could get the information without the overhead of a system call.
For information of general interest such a special kernel page could be mapped as read-only in the address space of all user processes.
Much of the information that is provided in the special file systems /proc and /sys could have been provided in some appropriate data structures in such read-only shared memory, for a faster access, by avoiding the overhead of file system calls and of file text parsing.
I've never done load testing before, but would it be hard to write a script in pure ruby (maybe with a few libraries) that makes a lot of concurrent requests to whatever endpoints and using whatever params you like?
Don't write your own load testing tool other than as a fun little exercise. At least not without understanding coordinated omission and thinking about workload modeling (open? closed? hybrid? all of the above?) [1]. Get this wrong and the results produced by your tool will be worthless.
Once you've got that out of the way, don't forget that you'll want a distribution story. It does not matter how efficient your tool might be on a single machine - you'll want to distribute your tests across multiple clients for real-world testing.
"Sure it's easy" you might say, "I know UNIX. Give me pssh and a few VMs on EC2". Well, now you've got 2 problems: aggregating metrics from multiple hosts and merging them accurately (especially those pesky percentiles. Your tool IS reporting percentiles rather than averages already, right?!), and a developer experience problem - no one wants to wrangle infra just to run a load test, how are you going to make it easier?
And, this developer experience problem is much bigger than just sorting out infra... you'll probably want to send the metrics produced by your tool to external observability systems. So now you've got some plugins to write (along with a plugin API). The list goes on.
I think mentioning you are a founder in the original comment might have prevented that particular reply from happening. I think one of the things I, personally, am becoming less okay with as a reader here is seeing recommendations without properly disclosing a connection.
Not saying everyone has nefarious reasons for doing it, but, it's just... everywhere.
I also play guitar, and there is a popular store in Europe with a pretty dang popular YouTube channel that I sometimes watch when the topic seems interesting. There was a whole kerfluffle a few months ago because one of the brand names that was getting a lot of air time on their YouTube channel was one that was financially backed by the owner of the store and a host of the channel. It took a ton of research of another YouTube to uncover this, and after it was found out, the owner of the store and host of the channel, finally disclosed his relationship with the brand he was promoting.
I feel like this was my more eye opening moment that tons of people out here on all variety of services are recommending their products but not disclosing their relationship clearly.
Now, you are saying so in your profile, but how many people are going to click into your profile?
I'm not saying you _have_ to do this, just suspecting that there are more and more people who are giving every recommendation the side eye these days because lack of disclosure. Disclosure isn't a bad thing, it just puts the bias in the open and people can gauge the recommendation more easily with that bias in mind.
None of this is probably new to you, but, trying to add something to the conversation rather than just call someone out, which is the easy and far more violent thing to do.
There's a lot of subtleties. It's really easy to accidentally load test the wrong part of your web application due to differences in compression, cache hit ratios, http settings, etc.
Shameless self promotion but I wrote up a bunch of these issues in a post describing all the mistakes I have made so you can learn from them: https://shane.ai/posts/load-testing-tips/
This is a great blog post! just taking the opportunity here to comment on this:
> Finally for full scale high fidelity load tests there are relatively few tools out there for browser based load testing.
It exists as of a few months ago and it's fully open source: https://github.com/artilleryio/artillery (I'm the lead dev). You write a Playwright script, then run it in your own AWS account on serverless Fargate and scale it out horizontally as you see fit. Artillery takes care of spinning up and down all of the infra. It will also automatically grab and report Core Web Vitals for you from all those browser sessions, and we just released support for tracing so you can dig into the details of each session if you want to (OpenTelemetry based so works with most vendors- Datadago APM, New Relic etc)
Load testing can be harder on the client side than the server side at high loads, which might be surprising, but consider how many servers need to handle large numbers of clients connecting and how few clients need to connect to large number of servers --- if you do it yourself, you're going to have to relearn all the techniques to make large number of connections possible.
If your desired load for testing is small, it's not a big deal, of course.
Depends on needs and familiarity. I used Bash and Curl, then PHP, then settled on Jmeter for a tricky bug due to a memory leak from a race under load. All three could reproduce the problem. Jmeter had quite a learning curve, and preparing data for it was a pain. Ultimately though it was more reliable, can do distributed testing, and has lots of nice built in features and analytics.
It is simple enough to do a script doing that in Ruby, but there also enough off the shelf tools specifically for load testing that encodes a lot of experience and avoids a lot of obvious and not so obvious mistakes, so it's usually not worth writing your own.
Android has a BLE scanner app that can passively read info. I don't know enough about the protocol to know if that is the same thing you are referring to.
Android allows you to initiate scanning and receive advertisements without user interaction (assuming the app has the permissions to run in the background and use Bluetooth). Advertisements are "broadcasts" or "beacons"
I heard this was behind mechanical turk. Sounds like the playbook remained the same.