> "Less expensive SSDs typically have write speeds significantly lower than their read speeds. Higher performing SSDs have similar
> read and write speeds."
> So it sounds like this isn't as much of an issue as the article claims.
I'm going to guess that the article is taking about read/write throughput. If you are comparing throughput that can be a true statement (thought it's usually only true in devices where the limiting factor is the bandwidth of the connection to the host computer...).
Luiz was referring to the latency of individual operations. Flash is very asymmetric w.r.t. the latencies of reads and writes. The asymmetry is only getting worse with the smaller lithographies used for newer chips.
> > With flash you can read 4KB of data in 100 microseconds or so, if your read is stuck behind an erase you may
> > have wait 10s of milliseconds. That’s a 100x increase in latency variance for that particular variance that
> > used to be very fast.
> I had never heard this claimed before -- is this true? Wikipedia says:
The claim is very true.
At the level of a single flash chip, you can only do one operation at a time. So at any one time, a single chip can be doing exactly one of: erasing a block, writing a page, or reading a page.
Erasing a block
- Latency ~5-20ms depending on the generation of flash.
- On modern chips, a block will have between 64 & 256 4KiB or 8KiB pages.
Write a page
- Latency 400us - 4ms
- On SLC chips, the latency will be consistently in the 400us range
- On MLC, half of the pages in a block will see 400us latency, the other half will be in the 3ms-4ms range
- Writes can only be performed on erased pages.
Read a page
- Latency ~80-100us
SSDs achieve performance by putting together a lot of flash chips and trying to keep them all busy at the same time. The more the operations an SSD can do in parallel, the better off it will be. A (very) simplified way to think about it is to imagine that the SSD is a RAID0 controller that is talking to flash chips rather than disks. For simplicity, imagine the data is striped uniformly across the chip, so a read/write for logical address A will always end up on chip (A / page_size) % number_of_chips.
Imagine you have two apps:
The first app only reads from the SSD. As long as nobody else is writing to the device it will see very read latency. If the reads are perfectly distributed across the address range, it will see 100us latency consistently. In the real world, some reads will be targeted to the same chip, and will therefore have to wait for the preceding read to complete.
The first app is seeing:
Read - 100us latency
Read - 100us latency
Read - 200us latency (got stuck behind a previous read)
Read - 100us latency
Read - 100us latency
Everyone is happy.
Now imagine that another process wakes up and starts writing to the SSD.
The total latency for a write can vary greatly. If there is an erased page ready to go, the write operation will take ~400us. If the SSD needs to erase a page, it can be much higher. If there is an empty page that just needs to be erased the write operation will take (20ms + 400us). If the SSD needs to do garbage collection (i.e. copy data from one block to another to free up a block) the time could be much higher, possibly 50ms or higher.
App Operation Latency
A Read chip 0 100us
B Write chip 0 400us
A Read chip 0 500us+
... or ..
A Read chip 0 100us
B Write chip 0 40ms
A Read chip 0 40ms
So app A would see a 40ms/100us = 400x variance in read latency.
I'm going to guess that the article is taking about read/write throughput. If you are comparing throughput that can be a true statement (thought it's usually only true in devices where the limiting factor is the bandwidth of the connection to the host computer...).
Luiz was referring to the latency of individual operations. Flash is very asymmetric w.r.t. the latencies of reads and writes. The asymmetry is only getting worse with the smaller lithographies used for newer chips.
> > With flash you can read 4KB of data in 100 microseconds or so, if your read is stuck behind an erase you may > > have wait 10s of milliseconds. That’s a 100x increase in latency variance for that particular variance that > > used to be very fast. > I had never heard this claimed before -- is this true? Wikipedia says:
The claim is very true.
At the level of a single flash chip, you can only do one operation at a time. So at any one time, a single chip can be doing exactly one of: erasing a block, writing a page, or reading a page.
Erasing a block - Latency ~5-20ms depending on the generation of flash. - On modern chips, a block will have between 64 & 256 4KiB or 8KiB pages.
Write a page - Latency 400us - 4ms - On SLC chips, the latency will be consistently in the 400us range - On MLC, half of the pages in a block will see 400us latency, the other half will be in the 3ms-4ms range - Writes can only be performed on erased pages.
Read a page - Latency ~80-100us
SSDs achieve performance by putting together a lot of flash chips and trying to keep them all busy at the same time. The more the operations an SSD can do in parallel, the better off it will be. A (very) simplified way to think about it is to imagine that the SSD is a RAID0 controller that is talking to flash chips rather than disks. For simplicity, imagine the data is striped uniformly across the chip, so a read/write for logical address A will always end up on chip (A / page_size) % number_of_chips.
Imagine you have two apps:
The first app only reads from the SSD. As long as nobody else is writing to the device it will see very read latency. If the reads are perfectly distributed across the address range, it will see 100us latency consistently. In the real world, some reads will be targeted to the same chip, and will therefore have to wait for the preceding read to complete.
The first app is seeing: Read - 100us latency Read - 100us latency Read - 200us latency (got stuck behind a previous read) Read - 100us latency Read - 100us latency Everyone is happy.
Now imagine that another process wakes up and starts writing to the SSD. The total latency for a write can vary greatly. If there is an erased page ready to go, the write operation will take ~400us. If the SSD needs to erase a page, it can be much higher. If there is an empty page that just needs to be erased the write operation will take (20ms + 400us). If the SSD needs to do garbage collection (i.e. copy data from one block to another to free up a block) the time could be much higher, possibly 50ms or higher.
App Operation Latency A Read chip 0 100us B Write chip 0 400us A Read chip 0 500us+ ... or .. A Read chip 0 100us B Write chip 0 40ms A Read chip 0 40ms
So app A would see a 40ms/100us = 400x variance in read latency.