minus-squarehubobes@sh.itjust.workstoTechnology@lemmy.world•Companies Are Throttling Employees’ AI Use Because It’s Too ExpensivelinkfedilinkEnglisharrow-up7·8 hours agoThat doesn’t make much sense. When Anthropic moved to Sonnet 5 they introduced a new tokenizer which increased token use up to 35%. If these would be unrelated kinds of tokens why would the usage go up when the process of tokenization changes? linkfedilink
That doesn’t make much sense. When Anthropic moved to Sonnet 5 they introduced a new tokenizer which increased token use up to 35%. If these would be unrelated kinds of tokens why would the usage go up when the process of tokenization changes?