What a terrible article. When you have a section titled "The Problem With Hardcoded Filters", it's entire contents should be about how the only way they have to prevent their bot from emitting outrageously libelous claims about people is to shut it down completely. So the other 8 billion people on earth who are not in that 6 name blacklist will continue to be defamed without consequence.
They once reported on some science result and for the life of me I could not figure out who actually published the result and had to search for the original paper. I emailed the AT author and asked why this was omitted from their article and they responded with something like "I went to that same university, so I felt like it would be biased if I mentioned it".
From the arguments I've read against this, I think not enough emphasis is being placed on the strongest one:
In the modern world online access is as necessary as water, power and phone service. No one would suggest forcing the power company to cut service to a customer over trivial civil law matters (which is what copyright is) that are completely unrelated to the company or the service it provides. No one should suggest cutting internet access either.
I guess ISPs in the US don't want to use that argument due to the regulatory implications (the common carrier classification thing)? But someone should be making that argument to the court.
The strongest argument comes from Viacom v Youtube. If Viacom itself is unable to identify which videos are infringement or not how is a Youtube supposed to be able to?
Or to put into different terms. If a copyright holder historically has asked an ISP (Google eventually become one) to un-takedown content as it wasn't actually infringing; why should the ISP be liable for not-proven-in-court activity by customers?
Yep, internet is a utility, like a toll road (most highways in many countries).
If you shoplift, should you lose your highway sticker?
Somehow the intrnet is this 'magical place' where real world analogies don't work for many people,... surveillance related stuff being the worst offender.
Of course public transportation in some parts of the US is so bad a car is almost necessary, despite being a privilege. I’m not saying it shouldn’t remain a privilege, but for many losing their license or not having a functional vehicle would mean almost certain financial ruin.
Not driving licence, the toll-road sticker. Vignette in many countries. It's usually tied to the vehicle, and it indicates that the yearly highway toll has been paid, and you're allowed to use the paid-highways.
If you personally did something, then you personally are responsible for that something that you did. Not your families car.
The only thing similar I’ve heard of in the US is vehicle registration fees and stickers (that usually go on the license plate), but these vary state by state and are not tied to any toll road or the like.
Every toll road I’ve ever been on in the US you have to pay each time you use it. These days most states use the same transponder system where your vehicle is detected by that device when you enter the toll road or by license plate readers.
I’ve never heard of a general “yearly highway toll” anywhere in the US.
Over here (central europe), many countries have yearly (monthly, weekly) tolls. You used to get a sticker to put on the windshield (now it's mostly digital, tied to the licence plate), and you can drive on any highway within the country for a week/month/year. No slowing down or stopping to pay, on the other hand, you either have to buy it in advance or stop at the border or the first gas station in the country and buy it there.
We had a huge reducation in traffic deaths due to that (because people use the highway more than before, even for just "one exit", since you don't have to stop and wait in line to pay anymore).
Keep in mind that you can still drive with a revoked license.
Trivially, there's the you can just do it illegally. But also pretty much every state allows you to get a "Hardship" license [1] which basically means you're not responsible enough to drive but also you can't live without driving so we're letting you drive to work/store.
In your specific context, the following doesn't directly apply, but the statement that driving is a privilege is frequently made here, often as a whole truth and not merely legal truism. It is a prevarication at best, to argue this beyond a purely legal premise, however.
To think that the entire nation would immediately collapse irreparably if this trivial "privilege" were removed, kinda suggests a problem with this factoid as a general view.
I am aware that for some individuals driving is entirely unnecessary. Some individuals don't have homes. I hope that for however anti-automobile one might rightfully be, the reality of this is still clear.
There might be a First Amendment argument for internet access. Congress shouldn’t have the power to force or coerce an ISP to disconnect someone from the internet. (Doesn’t take too much imagination to come up with a partisan copyright troll.)
Yeah, was thinking that. Do we cut off power to a home if it’s repeatedly used for growing weed? If no, then we definitely shouldn’t cut off internet over something even more trivial.
It's becoming less and less possible to live without every year. More and more government services over here can only be accessed online. And the private industry has been happily gutting offline or in-person services for a long time. If you still want access to everything, you better be ready to pay huge premiums just to be able to do things the old-fashioned way of having humans perform tasks for you. So if you're poor, you basically have no choice but to be online.
I got stuck in the spiral slide on the same level. I got the impression framerate glitches are affecting the collision detection (common physics implementation pitfall). I could be wrong though.
Still, very cool. Too cool to waste on marketing in fact :)
Only the OP would know for sure, but it might be the case that this never would've come into existence were it not for the project to land the messages about the company.
Yeah, sometimes the ball does some crazy things due to the way collision detection works. We tried to optimize and avoid most of the issues but it can happen.
Cookie header parsing is a shitshow. The "standards" don't represent what actually exists in the wild, each back-end server and/or library and/or framework accepts something different, and browsers do something else yet.
If you are in complete control of front-end and back-end it's not a big problem, but as soon as you have to get different stuff to interoperate it gets very stupid very fast.
Well we're about to find out now that CDPR have announced Cyberpunk 2077 will get a native Metal port. I for one am extremely curious with the result. Apple have made very lofty claims about their GPU performance, but without any high-end games running natively, it's been hard to evaluate those claims.
That said, expectations should be kept at a realistic level. Even if the M4 has the fastest embedded GPU (it probably does), it's still an embedded GPU. They aren't going to be topping any absolute performance charts.
I wouldn't mind JS as much if its type system wasn't an utter disaster.
Basically everything in that WAT talk should be an error. If your code is throwing type errors, then either it's shitty code or you're trusting non-validated input, in which case you should be able to catch the type error and return an error back to whoever sent you the data.
I know the article is just using the noise thing as an excuse to deep dive into zfs and proxmox, which is cool, but if what you really care about is reducing noise I thought I'd leave some practical advice here:
1. Most hard drive noise is caused by mechanical vibrations being transmitted to the chassis the drive is mounted on.
2. Consequently the most effective way to reduce noise is to reduce the mechanical coupling in the drive mounting mechanism. Having the drives in a noise-isolating case is helpful too, but only as a secondary improvement. Optimizing the drive mounting should really be the first priority.
3. If space isn't a concern the optimal thing is to have a large case (like an ATX or larger) with a large number of HDD bays. The mounting should use soft rubber or silicon grommets. Some mounting systems can work with just the grommets, but systems that use screws are ok too as long as the screw couples to the grommet not the chassis. In a good case like this any number of hard drives can be made essentially inaudible.
4. If space is a concern, a special purpose "NAS like" case (example: the Jonsbo N line of cases) can approach the size of consumer NAS boxes. The lack of space makes optimal accoustics difficult, but it will still be a 10x improvement over typical consumers NASes.
5. Lastly what you shouldn't ever do is get one of those consumers NAS boxes. They are made with no concern for noise at all, and manufacturing cheapness constraints tend to make them literally pessimal at it. I had a QNAP I got rid of that couldn't have been more effective at amplifying drive noise if it had been designed for that on purpose.
To add to this, I mounted an HDD to a case using motherboard standoffs in a place that was obviously intended for SSDs. Not only was it very loud, the resonance between the disk and the case also broke the disk after six months.
>Lastly what you shouldn't ever do is get one of those consumers NAS boxes. They are made with no concern for noise at all, and manufacturing cheapness constraints tend to make them literally pessimal at it. I had a QNAP I got rid of that couldn't have been more effective at amplifying drive noise if it had been designed for that on purpose.
Is there any solution that lets me mix and match drive sizes as well as upgrade? I'm slowly getting more and more into self hosting as much of digital life as possible, so I don't want to be dependent on Synology, but they offered a product that let me go from a bunch of single drives with no redundancy to being able to repurpose them into a solution where I can swap out drives and most importantly grow. As far as I can tell theres no open source equivalent. As soon as I've set up a file system with the drives I already have the only solution is to buy the same amount of drives with more space once I run out.
And I've never used a QNAP, but I'm on my second Synology and their drive carriages all use rubber/silicone grommets to isolate drive vibration from the case. It's not silent - five drives of spinning rust will make some noise regardless - but it sits in a closet under my stairs that backs up to my media cabinet and you have to be within a few feet to hear it even in the closet over background noise in the house.
I don't use any of their "personal cloud" stuff that relies on them. It's just a Linux box with some really good features for drive management and package updates. You can set up and maintain any other services you want without using their manager.
The ease with which I could set it up as a destination for Time Machine backups has absolutely saved my bacon on at least one occasion. My iMac drive fell to some strange data corruption and would not boot. I booted to recovery, pointed it at the Synology, and aside from the restore time, I only lost about thirty minutes' work. The drive checked out fine and is still going strong. Eventually it will die, and when it does I'll buy a new Mac and tell it to restore from the Synology. I have double-disk redundancy, so I can lose any two of five drives with no loss of data so long as I can get new drives to my house and striped in before a third fails. That would take about a week, so while it's possible, it's unlikely.
If I were really paranoid about that, I'd put together a group buy for hard drives from different manufacturers, different runs, different retailers, etc., and then swap them around so none of us were using drives that were all from the same manufacturer, factory, and date. But I'm not that paranoid. If I have a drive go bad, and it's one that I have more than one of the same (exact) model, I'll buy enough to replace them all, immediately replace the known-bad one, and then sell/give away the same-series.
It’s an 8Bay Synology 1821+. Cost about $1300 for the machine, 32GB of ECC memory, and the 10gbe network card.
I have 4 8Tb drives in a btrfs volume with 1 drive redundancy giving me 21TB of space.
All the important stuff gets also backed up to another 8TB drive periodically and sent to glacier.
The way synology’s shr1 setup works seems to be like RAID5 + a bit more flexibility so I can add more drives to the array but as long as they are 8TB or larger.
The docker manager seems to work pretty well. I run a few services there and mount certain volumes into them. A few DNS records and some entries into the reverse proxy in the control panel of it and you can run whatever you want.
Most critically power draw is very low and it’s very quiet which was an important consideration to me.
I might be misunderstanding your needs but my home server uses just LVM. When I run out of disk space, I buy a new drive, use `pvcreate` followed by `vgextend` and `lvextend`.
I've been running LVM and Linux software RAID for like 20 years now.
The only limits (for me at least) are:
smallest device in a raid determines size of that array. But that's fine since I then LVM them together anyhow. It does let you mix+match and upgrade though really I always just buy two drives but it helped when starting and I experimented with just LVM without RAID too.
I have to know RAID and LVM instead of trusting some vendor UI. That's a good thing. I can fix stuff in case it were to break.
I found as drives went to Terabytes it was better to have multiple smaller partitions as the raid devices even when on the same physical drive. Faster rebuild in case of a random read error. I use raid1. YMMV
I still have the same LVM partitions / data that I had 20 years ago but also not. All the hardware underneath has changed multiple times, especially drives. I still use HDDs and used to have root on RAID+LVM too but have switched for a single SSD. I reinstalled the OS for that part but the LVM+RAID setup and its data stayed intact. If anything ever happens to the SSD with the OS, I don't care. I'll buy a new one, install an OS and I'm good to go.
> Is there any solution that lets me mix and match drive sizes as well as upgrade?
Probably more than one, but on my non-Synology box I use SnapRAID, which can take any number/size of drives. Downside is that it isn’t realtime, you have to schedule a process to sync your parity: http://www.snapraid.it/
> As soon as I've set up a file system with the drives I already have the only solution is to buy the same amount of drives with more space once I run out.
Recent versions of zfs support raidz expansion [1], which let you add extra disks to a raidz1/2/3 pool. It has a number of limitations, for example you cannot change the type of pool (mirror to raidz1, raidz1 to raidz2 etc.) but if you plan to expand your pool one disk at a time it can be useful. Just remember that 1) old data will not take advantage of the extra disk until you copy it around and 2) the size of the pool is limited by the size of the smallest disk in the pool.
I started thinking about this a year ago. Unraid works great for me. Just bought another 32tb to extend to 104tb usable space, 8gb to 20gb drives. It’s a JBOD with dual parity setup, next upgrade path requires a disk shelf but hopefully that won’t be for a couple of years.
I recently upgraded my home-NAS from a Fractal Define R4 to a Define 7 XL. The R4 had the rubber grommets, but hot-swappable trays that were just held in by spring force. As such they rattled a lot.
The Define 7 has the same grommet system, but the trays can be fastened by screws to the support rails.
The difference in noise was significant. Even though I went from 6 to 10 disks it's much more quiet now.
I went down this rabbit hole of sound reduction, but then I bought a house and put all my NAS and NAS-like things in my basement storage room where I can’t hear them from anywhere else in the house.
I find NASes to be a waste of money except for the “no one ever got fired for….” aspect in an enterprise environment. $600 for a NAS with a Celeron and 8GB of RAM is absurd.
Value of your time and effort maintaining it is not zero.
I used to play with stuff like this. It was fun when I was single and had lots of free time. I don't play with it anymore. If I pay someone $500 over nominal value to provide me with 8-9 years of support for security updates, etc., and I just install their packages... that's worth it to me. My first Syno was a DS412+ and my second was a DS1621+. Nine years between introduction of the two. The 412+ is still running just fine at a friend's house. I gave it to him with ~12 TB total drive space, said just help me next time I need something done with car audio (he's a DJ and knows cars) and we're square.
He's happy, I'm happy. I go set up his network, he installs my head unit. We both win by doing what we're good at and letting someone else use their expertise instead of learning a lot of stuff we will almost never use again.
Yeah, the article title seemed kind of weird to me. I have a ZFS NAS, it's just a bunch of drives in an ATX case with (what I'd considered to nowadays be) the standard rubber grommets.
I mean, you can hear it, but it's mostly just the fans and drives spinning, it's not loud at all.
The recommendations seem reasonable but for noise? If it's noisy probably something is wrong I think.
I totally understand the article title, I have a ZFS NAS that makes the same kind of noise as described there. Roughly every five seconds the drives make sound that is different from the background hum of a running computer. In a calm environment this is very distracting. I even had a guest sleeping in an adjacent room complain about it once.
vfs.zfs.txg.timeout defaults to 5 seconds, but it can be set (much) higher if you wish.
I don't care if I lose up to a minute or two of work instead of <=5 seconds in the face of an unplanned failure, so I set it to a couple of minutes on my desktop rig years ago and never looked back.
AFAIK there's also no harm in setting it both dynamically and randomly. I haven't tried it, but periodically setting vfs.zfs.txg.timeout to a random value between [say] 60 and 240 seconds should go a long ways towards making it easier to ignore by breaking up the regularity.
(Or: Quieter disks. Some of mine are very loud; some are very quiet. Same box, same pool, just different models.
Or: Put the disks somewhere else, away from the user and the sleeping guests.)
This is likely a different problem than the article describes. Most newer hard drives will move the actuator arm back and forth every few seconds when the drive is inactive. It has to do with evenly distributing the lubrication on the arm to increase the life of the drive.
No. The most effective way to remove HDD noise is to remove HDDs and add SSDs. I don't have any HDDs since 2016.
P.S. I also talked to a customer in the past who stored their backups in an SSD-only Ceph cluster. They were citing higher reliability of SSDs and higher density, which was important because they had very limited physical space in the datacenter. In other words, traditional 3.5" HDDs would not have allowed them to store that much data in that many rack units.
SSDs are great. Quieter, can be denser, faster, available in small sizes for small money, more reliable, etc.
But they're not great for low cost bulk storage. If you're putting together a home NAS, you probably want to do well on $/TB and don't care so much about transfer speeds.
But if you've found 10TB+ ssds for under $200, let us know where to find them.
The 4TB M.2 SSDs are getting to a price point where one might consider them. The problem is that it's not trivial to connect a whole bunch of them in a homebrew NAS without spending tons of money.
Best I've found so far is cards like this[1] that allow for 8 U.2 drives, and then some M.2 to U.2 adapters like this[2] or this[3].
In a 2x RAID-Z1 or single RAID-Z2 setup that would give 24TB of redundant flash storage for a tad more than a single 16TB enterprise SSD.
On AM5 you can do 6 M.2 drives without much difficulty, and with considerably better perf. Your motherboard will need to support x4/x4/x4/x4 bifurcation on the x16 slot, but you put 4 there [0], and then use the two on board x4 slots, one will use the CPU lanes and the other will be connected via the chipset.
You can do without bifurcation if you use a PCIe switch such as [1]. This is more expensive but also can achieve more speed, and will work in machines without bifurcation. Downside is it uses more W.
Right, and whilst 3.0 switches are semi-affordable, 4.0 or 5.0 costs significantly more, though how much that matters obviously depends on your workload.
True. I think a switch which could do for example PCIe 5.0 on the host side and 3.0 on the device side would be sufficient for many cases, as one lane of 5.0 can serve all four lanes of a 3.0 NMVe.
But I realize we probably won't see that.
Perhaps it will be realized with higher PCIe versions, given how tight signalling margins will get. But the big guys have money to throw at this so yeah...
