> Outerport is a caching system for model weights, allowing read-only models to be cached in pinned RAM for fast loading into GPU. Outerport is also hierarchical, maintaining a cache across S3 to local SSD to RAM to GPU memory, optimizing for reduced data transfer costs and load balancing.
This is really cool. Are the costs to run this mainly storage or how much compute is actually tied up in it?
The time/cost to download models on a gpu cloud instance really add up when you are paying per second.
Thanks! If you mean the costs for users of Outerport, it'll be a subscription model for our hosted registry (with a limit on storage / S3 egress) and a license model for self-hosting the registry. So mainly storage, since the idea is to also minimize egress costs which are associated with the compute tied up in it!
This is really cool. Are the costs to run this mainly storage or how much compute is actually tied up in it?
The time/cost to download models on a gpu cloud instance really add up when you are paying per second.