Believe it or not, the boosters are just built out of stainless steel so that they can survive the heat. Not toxic.
Starship itself uses a silicon-based ceramic for insulation. You wouldn't want to be hit in the head by one of those tiles, but it's unlikely to poison you.
Just stumbled upon this - it does seem that the particulate matter scattered for miles is being investigated as being a major health risk to nearby residents and endangered species.
Breathing in non-toxic particles can be bad for you due to the effects of mechanical injury. The classic example is asbestos, completely non-toxic, but the result of how your body attempts to repair damaged cells in your lungs can cause an increased chance of cancer.
This is one of the top concerns of nanotechnology. It could potentially create a lot of grit that we really don't want to breathe...
You remind me that I heard brake pad particulate was horrible for our health, and highly present in the air in LA as well - another thing to lose sleep to!
Starship itself uses a silicon-based ceramic for insulation. You wouldn't want to be hit in the head by one of those tiles, but it's unlikely to poison you.