The main issue with Open / Local LLMs is the lack of broad support. It’s not hard for me to run Local LLMs, but apparently a decade of informal python experience and using linux mint is intimidating to some people. Apparently people want to click button and talk to phone god? Mostly speaks of a failing education system if you ask me.
I’m not even that good with AI, I just lurked a bit on !LocalLLaMA@sh.itjust.works and got mine running at like 30 tokens/second on my laptop.
I will have to defend users here and blame AI scientist/developers together with the whole docker culture. The way they package and delivery software is just terrible. And even though pytourch has support for all GPU apis the devs seems to have a hard time adding this support. Granted, part of that is also on pytourch that for some reason Don’t seem to be able to make a unified release that supports all GPUs and then you need different versions and by default IIRC they default package only supprt CUDA and CPU.
Most of us here think streaming is useful, despite the fact that streaming services are based on the idea that they own and control all access to media with you renting it from them, raising prices, shoving ads in your face, and removing media from the collection you’re renting from them. Our solution in this community is investing in the open source community around Jellyfin, the Arr stack, etc. so we can still enjoy the benefits of streaming services while owning our own content and not having to financially support companies we don’t agree with. I’d say a lot of us here who are happily using Jellyfin might otherwise have a streaming account if it didn’t exist. Admittedly, I used Netflix until I realized I had better options.
I think the same is true with local LLMs. Not all of us agree that LLMs are useful, but most of us here agree that a few tech giants tightly controlling LLMs and renting them to everyone is not going to be a good thing. Without self-hosted LLMs, many people who do find value using them will go ahead and financially support the rent-seekers who are hell bent on destroying the world for their own financial gain, as well as support them by sharing data that can be used to train their models. Even when you use the free tier of Chat GPT, for example, you’re supporting OpenAI by giving them your prompts that they can use to make their models better.
The ecosystem around running open weight models is rapidly evolving. I’m already running the Qwen 3.6 MoE model with the desktop beta of OpenCode on my gaming laptop and it’s pretty decent. Personally, I’ve found ways to use LLMs where they are actually useful and not just slop generators, though I initially thought they were useless before I spent a lot of time working with them. I’m all for supporting and contributing to this ecosystem so that people can use LLMs without giving their money and data to shithead psychopaths.
Mmm… better that all AI fail.
Perhaps in Candyland, or maybe even the land of chocolate. But I think in the real world it’s here to stay.
yeah, we only created the garbage, how could we ever turn the garbage off?
I mean, with shit takes like that, geeze, guess we’re really fucked
You know, just like actual garbage, not all AI is equal. Some ai you probably do want to throw away. Some can be repurposed into something great. locally hosted AI is a wonderful tool that empowers the individual. The same as any FOSS software, it’s a threat to the establishment.
To conflate the mathematics and technology of LLMs with billionaire bullshit is tempting, but also an error.
oh and btw, FUCK YOU for equating AI SLOP to FOSS software. That’s a shit take of mammoth proportions and you should shove it up your ass sideways, it’s absolute garbage.
poor baby can’t even formulate a comeback lol
I just wish we could invest the time/money/resources into compressing AI and making it smaller and more efficient. I’d so much rather have a somewhat capable AI that can be run locally and offline, to outsource menial tasks to like alphabetizing spreadsheets and so basic image modification, than to have to upgrade my hardware constantly or use cloud based SaaS and/or have newer models that are more accurate in their predictions.
Of course that assumes a lot of things, like the intent to help people and not make money. Maybe someone in the Linux-sphere will make something.
I would like to see one integrated into a gnu os like linux where its only capability is to understand the os and guide you through it. No generation and no expertise outside the os exosystem. Maybe allow for it to be given the privelege to search the web. I would have it have capability to use other ais to perform other tasks so modules or whatnot could be added to give it more capability as a general computer butler type. Basically an os that acted like a start trek computer.
I want clippy but actually useful with all software, just giving tips when needed, ai can be useful sometimes, idk like im bad at math always have been, I need to sort some curves by index recentlly and it helped with the math logic a lot, otherwise I was using a repeat node and it was a lot slower than the way it showed me. Downside ofc was the ai way isn’t fully accurate or implementable as they say, has to be modified, it makes up nodes that don’t exist, but there are similar ones.
Increasingly, people ask me questions, send me screen shots, I copy-paste that into gpt, gpt’s answers are helpful and correct… they have access to the same (free to use) gpt themselves…
When I ask a person a question im generally trying to get their perspective. Im likely asking a few people or will over time. Its honestly just a part of socializing and interacting as humans.
I get questions like: why can’t I access this server, I followed the wiki page (first clue, they didn’t follow the wiki page). That’s not asking for insight, that’s asking for where they failed to follow a set of 5 step directions by doing things like: changing the default filename of their new ssh key to something they invented.
GPT explained, far more patiently than I would have, how indeed to do 4 more steps and rename your ssh key to anything you want, but I did offer the insight: if you just leave the name as the default value, you can skip all of this extra work.
why are people asking you this. Do you mean at work?
Yes, do you answer questions for money outside of work? Outside of work if somebody is asking me a question I assume they want my answer and I’ll give them that instead of looking something up, although sometimes I punt with an “I don’t know but I bet Google does…” Inside work I attempt to answer questions as correctly and efficiently as possible - the GPT tools are great at that.
The word “intelligence” is doing a lot of heavy lifting here. LLMs lack any mechanism for true logical reasoning, and they always will by nature. This is why they fail at simple questions like “the car wash test”. It’s also why agents are expensive; They just flail around in token hungry “reasoning loops” until they happen to come across a correct solution. And it’s why Claude Opus 4.8 (High) only scores 1.5% on the ARC-AGI-3 benchmark at a cost of $10,000.
This kind of panic is just part of the hype. Wake me up when real intelligence arrives.
Computers are not actually intelligent. What are they but big calculators? They are removing jobs and barely doing anything better! They take up a whole room! No one needs that!
The internet is hardly worth anything. It’s nothing but scams and porn! You can’t believe anything you read on it! It’s overhyped nonsense! No one is going to use that!
Meanwhile in 2026: LLMs have no logical reasoning and that will never change! They can’t answer simple questions! They just flail around and produce slop!
You guys don’t see it? Really?
I’m confident enough about this that I’ve registered a prediction on Long Bets.
“No LLM-based AI will surpass 70% on the ARC-AGI-3 leaderboard, with a cost of $1000 or less, before June 2028.” - https://longbets.org/973/
I’m curious if you’d really disagree with the premise, and would you (or anyone here on Lemmy) be willing to put money down to challenge the bet? (Long Bets always donates any winnings to a registered non-profit of the winner’s choice, though it’s a $200 minimum).
Are you saying that LLMs can currently reason? How do you explain their low score on ARC-AGI-3? Do you think Transformer LLM architectures will be capable of reasoning within the next two years without some new breakthrough? What mechanism in the architecture allows them to reason?
It doesn’t need to be our new cyber god for it to matter. All the software developers I know use AI as a tool. One person I know who heads a department at a publicly traded company you’ve probably heard of said they’re not hiring more junior developers, and is worried about his long term career outlook.
That’s concerning right? Capitalists who have a choice between hiring ten people or hiring one person who supervises an AI doing the work of nine are going to choose the cheaper option, and pocket the difference.
To say that LLMs are not useful to me indicates a lack of familiarity. I’m not taking about bullshit “write this email for me” type stuff, I mean like “write this web app for me” “find this type of document in this massive trove of documents” “troubleshoot this technical issue”. And it can do it.
LLMs are not replacing humans, but they are reducing labor required for many tasks. Who receives the benefit from that reduced labor? Right now, owners. That’s why I support open source AI or preferably democratic-entity-only open source AI.
Companies are only shooting themselves in the foot in the long term if they stop hiring junior engineers, and most of that work is not being replaced, it’s being shifted to the senior engineers who now have to babysit AIs that can’t actually do the job for any extended period of time. If you’re accepting AI code into a codebase without thorough review, then you’re also shooting yourself in the foot in the long term, because even the senior engineers won’t know the codebase after a while. If you’re doing thorough reviews in order to catch the AI bugs, well then you’re probably better off coding it yourself correctly in the first place, unless you’ve already allowed your skills to atrophy.
Do you really think AIs are reasoning when you ask them to troubleshoot technical issues? You may be lucky if the issue is already in their training data, but anything even slightly novel, and the AI is just going to bullshit an answer, and I guess you’re going to follow it blindly, since you don’t know enough to come up with an answer yourself.
Besides all that, how is open source AI going to stop junior developers from losing their jobs?
Did anybody click the link to read the paragraph-long website text? Not only is it packed with assumptions about AI, it’s simply unhinged.
The ability to… run intelligence systems without asking permission is of existential importance.
Existential?! No it’s not.
This mirrors the delusion Sam Altman demonstrated when he insisted nobody could raise a child without AI.
It is now. If you want it not to be, go ahead and stop all this modern bullshit happening around me.
You can’t yell at people on the ocean that they don’t really need boats, and then attack boats. I mean you can, but it’s foolish and unproductive.
AI is not saving you from anything existential.
If you believe you’re drowning without its salvation, you’re the foolish one.
I think giving language a fixed agency in human society 4,000 years ago by permanently structuring it, binding it to physicality, and giving it violent control over people was a bad idea. But gurl, unless you are going back to pre written language, all this living in the modern world and yelling about “certain parts” of technology, is one of the most ignorant stances a thinking being can take.
Unless you are taking us back back, you are a posturing, uneducated, little baby who has drank so much of the Kool aid she can’t see the forest she is in at all.
unless you are going back to pre written language, all this living in the modern world and yelling about “certain parts” of technology, is one of the most ignorant stances a thinking being can take.
Ahh I get it, thank you. The machine is hurting you specifically now and so you’re lashing out.
“The other stuff, that was cool, but daddy said we were good and everything was chill while we watched the degenerates and the environment burn! but that was a lie! We’re not chill! And now Daddy wants my flower too!”
They’re going to do to your AC what they did to the buffalo🤣
What
Datals took mer jerb! Der turk mur jerb!
OpensourceAI MustWinLoseNo matter how much your distaste for it is, the cat is out the bag. You can refuse it but other people will keep using it
NFTs are the future
I would say AI has a lot more usecases compared to a monkey picture URL on the blockchain
And yet they haven’t gone away, and as far as I can tell, they’ve generated more revenue for the people selling them than an AI ever has.
No matter how much your distaste for [leaded gasoline] is, the cat is out the bag. You can refuse it but other people will keep using it
This comparison is really lacking, just like the NFT one of the other commenter.
