The article is not clear on what a “distillation attack” is… what exactly is Alibaba supposed to be getting away with here? The article mentions using many different connections through obfuscation networks and proxies… so that would get them around rate limiting, and maybe enable them to submit many queries on free accounts… just spin up a new account whenever you hit the token limit of an unpaid account. So basically it’s a terms of service violation?
I don’t see why it’s necessarily a huge leg up for a competitor… they are just using the outputs of another model as training data. They still need to train their model, which is the expensive and energy intensive part.
It sounds to me like Anthropic just wants the US Government to help enforce its TOS internationally and force Alibaba to pay for those precious tokens? Because apart from that piece, the “attack” just seems like normal use of the service. If Anthropic’s service has an inherent vulnerability, that’s their problem.
Of course all the other comments about how they stole all their training data in the first place are spot on.
Distillation allows you to make a smaller model that can produce the same outputs as a larger model. Basically they’re pirating all of the hard work anthropic did pirating the entire internet.
Alibaba gets a model that produces basically the same output for a tiny fraction of the cost to operate the model once it’s finished training. Distillation training also uses basically all of its data from the big model (afaik it’s all of it sourced from the parent model)
It’s like if you took a lump of metal and showed it Porsche 911s until it turned into a 911 shaped chunk of metal that had 95% of the performance, but it only cost you $3000 for the ingot, and also cost ⅕ the amount in fuel and maintenance.
The article is not clear on what a “distillation attack” is… what exactly is Alibaba supposed to be getting away with here? The article mentions using many different connections through obfuscation networks and proxies… so that would get them around rate limiting, and maybe enable them to submit many queries on free accounts… just spin up a new account whenever you hit the token limit of an unpaid account. So basically it’s a terms of service violation?
I don’t see why it’s necessarily a huge leg up for a competitor… they are just using the outputs of another model as training data. They still need to train their model, which is the expensive and energy intensive part.
It sounds to me like Anthropic just wants the US Government to help enforce its TOS internationally and force Alibaba to pay for those precious tokens? Because apart from that piece, the “attack” just seems like normal use of the service. If Anthropic’s service has an inherent vulnerability, that’s their problem.
Of course all the other comments about how they stole all their training data in the first place are spot on.
Distillation allows you to make a smaller model that can produce the same outputs as a larger model. Basically they’re pirating all of the hard work anthropic did pirating the entire internet.
Alibaba gets a model that produces basically the same output for a tiny fraction of the cost to operate the model once it’s finished training. Distillation training also uses basically all of its data from the big model (afaik it’s all of it sourced from the parent model)
It’s like if you took a lump of metal and showed it Porsche 911s until it turned into a 911 shaped chunk of metal that had 95% of the performance, but it only cost you $3000 for the ingot, and also cost ⅕ the amount in fuel and maintenance.
Ok, thanks for the detailed explanation. I guess if your goal is to make your model sound like another model that makes perfect sense.