The rest of the world has partially switched, but as long as people continue to use hours, days, weeks, months, years, leap years, and not kiloseconds and megaseconds, they don't seem to have switched completely either, but stopped at some point.
Even if you want to argue days and years are based on physical measurables, hours, weeks, and months are arbitrary and not based on 10.
Seconds (and thus all time units derived from them, e.g. hours, nanoseconds) are officially part of the metric system.
But yes, it is still worthwhile to keep in mind that despite nearly all countries officially adopting SI, a much lower percentage of humanity uses SI exclusively in their day-to-day life.
Seconds are indeed a fundamental unit of the metric system.
You can then use the standard multipliers and get things like kiloseconds and megaseconds, milliseconds and nanoseconds, some which are widely used, while some are not.
You the can also use legacy minutes and hours, which are based off the fundamental seconds unit, but that doesn't make them "metric", nor technically a SI unit (although it's accepted to be used alongside SI units. "Although not an SI unit, the minute is accepted for use with SI units" Source https://www.bipm.org/en/publications/si-brochure/)
Even if you want to argue days and years are based on physical measurables, hours, weeks, and months are arbitrary and not based on 10.