Yes. If you're storing the objects on your server, why would you use S3?

Caching. We request the same objects quite frequently and bandwidth is killing us.

Well, if the consumer is the server, rather than the server then serving the S3 data to a client, then I could see the benefit in the server caching retrievals locally.

S3 for persistence, the server for caching? Perhaps you have multiple servers that carry no state?