Sources and leaks from Amazon, Adobe, Atlassian, Citi, and more show what is really happening with AI right now: companies are trying to rein in AI use as costs spiral out of control.
It’s not like that. Tokens are an inherent computational property of how a model calculates the probabilities and such to generate text.
Having said that, what a token means in terms of computation varies wildly between models and is not directly comparable. So attributing a money value to tokens in general, independently of the model, is weird by nature.
And even within a model, the number of tokens needed to generate a response is very variable too, depending of the model itself and the parameters with which it has been configured (thinking mode, temperature, etc.).
So yeah, companies can pretty much set any price they want and there’s not much anyone can do about it.
It does make sense for the provider as those for a specific model provide a good measure for computational effort, for that doecific model. That doesn’t mean that token rate comparison between models give you a good picture.
It’s not like that. Tokens are an inherent computational property of how a model calculates the probabilities and such to generate text.
Having said that, what a token means in terms of computation varies wildly between models and is not directly comparable. So attributing a money value to tokens in general, independently of the model, is weird by nature.
And even within a model, the number of tokens needed to generate a response is very variable too, depending of the model itself and the parameters with which it has been configured (thinking mode, temperature, etc.).
So yeah, companies can pretty much set any price they want and there’s not much anyone can do about it.
It does make sense for the provider as those for a specific model provide a good measure for computational effort, for that doecific model. That doesn’t mean that token rate comparison between models give you a good picture.