It takes effort to be an informed citizen. Artificial intelligence tools offer an alluring shortcut — but they’re not without risk.
It takes effort to be an informed citizen. Artificial intelligence tools offer an alluring shortcut — but they’re not without risk.
Models don’t “care” about “truths”, they are about the average likelyhood that the next word is correct based on the previous x words based on the data they were trained on, which was mostly internet posting.
Their output can also be manipulated by adding “system prompts” so that the user’s question isn’t the only thing being input.
ChatGPT doesn’t think, it doesn’t “reason”, it doesn’t know what is true, it doesn’t “feel” “integrity”. It regurgitates Wikipedia articles against the biases of 4chan and Reddit posts and tries to sell that diarrhea as bowls of soup.
Personally I’m to the point where I understand gulags now. Like, why would they need that? Why would they do something that seems so intentionally cruel and forceful and then read shit like this and I go “oh, oh yeah. I get it now. Yeah some people belong there.”
I very much doubt the kind of people that most need to be in things like gulags end up in them…
This is screaming into the the ether, but I’ll point out that this is not how GPT, or transformer architectures in general, work. You are describing a basic ass bag of words vectorization model, which is certainly in the lineage of transformers but is absolutely not the end of it. What made transformers such an inflection point in NLP is the development of the attention mechanism. It is still generating an iterative and sequential probabilistic series of tokens, but those probabilities are informed by context and attention along a huge scaffold of parameters that encode a lot of semantics. This isn’t exactly how humans reason, but it’s distressingly not that far off either.
That is all to say, this is still a terrible idea but, no, models are not just “hurr durr weighted averages hurr durr” and it’s been over a decade since that was even remotely at the forefront of NLP.
These datasets, along with the rules governing them, are a bit more complicated than weighing 4chan and wiki equally. They absolutely weigh peer-review journals and white-listed encyclopedia sources higher. Once they establish a baseline of credibility and identifying a certain grade-level of communication, along with identifying critical-thinking patterns (fallacies, formal logic, etc.)., then you can quite easily filter out the noise.
Every single time I’ve used a GPT to analyze a conversation with a Trump supporter and myself, it has identified their fallacies I had already identified, and rightfully pointing out fallacies of my own at times – but almost always concludes their argument is far, far worse.