“Sorry” is a phatic expression in several anglosphere cultures. You don’t say it because you’re sorry, you say it because that’s what you say when you… do anything. Sorry, would you like some breakfast? Here it is, sorry about that.
It drives her acutely nuts. I really try not to, and I frequently end up chewing my tongue while the word struggles to escape my lips like a mad cat in a bag, and then it slips out two minutes later anyway.