According to the chart, the current homicide rate is about 7/100,000. I found a spreadsheet showing homicides going back to 1849. The rate between 1968 (happy birthday to me) and 1996 is significantly higher than now, bouncing around between 10 and 20. Between 1920 and 1967, it bounces around between a low of 2 (1938) and 9. The 1870s were another relatively high period.
Interestingly, San Francisco's population declined from 775,000 in 1950 to 679,000 in 1980.
Possibly an all-time low. It's hard to compare numbers >40 years old, because they very possibly are not measuring the same things.
As to ease when you "start extremely high" -- SF is now safer, when it comes to violent crime, than most major cities in the US. What it does have is an extremely high rate of quality of life crime.