@matt @nobodyknows that's how #Pleroma does it. They don't store remote assets.
On one side big instances would suffer as being CDN for all smaller instances. The same way, a smaller instance with popular accounts would suffer.
It could also be if those servers were slow serving you or your users would have a degraded experience. That's one of the reasons, I think, Mastodon chose the architecture of storing everything locally.
Just guessing.