There was just a subquadratic model (SubQ)released that performs about as good as the last round of frontier models, with a 12 million token window, but with 3 orders of magnitude less resource usage. Google also has their compressed models which greatly reduce processing. I think at some point the stuff we use frontier models for today will be running locally on our PC’s and all this buildout will have been for naught or for people who have such good business cases that paying top dollar for a better model makes sense.
My question is this model is it open source or is it open weight because there is a difference. Open source gives you access to the training data that was used for the model. It gives you the code used to make the model. It gives you checkpoints during training of the model and methodologies that were used to train the model, etc.
Open weight models, however, don’t give you access to the training data or snapshots during training. Mainly, they just give the methodology and what they weighted the data as.
The only fully open source model I am aware of, personally, is OLMO from AI2.
Sorry I am mixing up my news. SubQ is not actually open source. I was confusing with the open source GLM5.2 which is also near frontier level but not lightweight. I edited my original post to reflect that.
There was just a subquadratic model (SubQ)released that performs about as good as the last round of frontier models, with a 12 million token window, but with 3 orders of magnitude less resource usage. Google also has their compressed models which greatly reduce processing. I think at some point the stuff we use frontier models for today will be running locally on our PC’s and all this buildout will have been for naught or for people who have such good business cases that paying top dollar for a better model makes sense.
My question is this model is it open source or is it open weight because there is a difference. Open source gives you access to the training data that was used for the model. It gives you the code used to make the model. It gives you checkpoints during training of the model and methodologies that were used to train the model, etc.
Open weight models, however, don’t give you access to the training data or snapshots during training. Mainly, they just give the methodology and what they weighted the data as.
The only fully open source model I am aware of, personally, is OLMO from AI2.
Sorry I am mixing up my news. SubQ is not actually open source. I was confusing with the open source GLM5.2 which is also near frontier level but not lightweight. I edited my original post to reflect that.