This is screaming into the the ether, but I’ll point out that this is not how GPT, or transformer architectures in general, work. You are describing a basic ass bag of words vectorization model, which is certainly in the lineage of transformers but is absolutely not the end of it. What made transformers such an inflection point in NLP is the development of the attention mechanism. It is still generating an iterative and sequential probabilistic series of tokens, but those probabilities are informed by context and attention along a huge scaffold of parameters that encode a lot of semantics. This isn’t exactly how humans reason, but it’s distressingly not that far off either.

That is all to say, this is still a terrible idea but, no, models are not just “hurr durr weighted averages hurr durr” and it’s been over a decade since that was even remotely at the forefront of NLP.