Now that "scale is all we need" has predictably faltered...
1. Small AI models often perform better in context.
2. Obsession w bigness has bad consequences, from climate, to power concentration, to research capture.
Me +
@GaelVaroquaux
@sashaluccioni
arxiv.org/abs/2409.14160
@Mer__edith @GaelVaroquaux …to be fair, further scaling may falter, but current large models can do amazing things, will continue to do so and probably will get more efficient and cheaper
@ErikJonker @Mer__edith @GaelVaroquaux
But will they get cheap enough to be profitable on their own without a surveillance Big Tech corp behind them?
What good is a technological "revolution" if it can only be operated, at great financial cost, by, like 5 entities?