I know AI/LLM hate is strong here, so this is going to get some blow back. But there’s a lot of Linux folk on here, so let me frame it this way…
My understand of the Linux/unix design philosophy is building small, efficient programs that do a limited set of tasks very well and that can be strung together with other programs that do other tasks very well. This is in opposition to the " be everything" program concept of Windows and Microsoft Office Suite. At least this is how would describe the difference to non technical friends: Nothing you think of as your OS in windows is actually what Linux is replacing. You’re getting the Linux kernel packaged up in a distro that combines a bunch of smaller pieces (file explorer, window manager, etc) that you can still customize from there.
When I look at the approach to AI, I see the same thing. I’ve dabbled enough in ML/LLMs to know that LLMs are effectively very fancy next word predictors or for the case of image/video GenAI, next pixel predictors. As others have said countless times, there’s no consciousness or understanding of the context, but you can ask it things in natural language and it will try to produce whatever you asked for in the same app regardless of context.
From a science project standpoint, this is cool, but it doesn’t seem scalable or consistently reproducable and the energy use and easily found blunders seem to support that thought.
So, my question is why is no one building AI with a Linux philosophy? Small purpose built ML models with a language processing/triage model on top? Oh this person has a question about history, send them to the history module. This person wants to edit a photo, send them to the photo editing module. Then let those modules dig deeper from there. That’s how we do customer service with real people after all. With this way we could refine each specialization individually instead of having a giant model that consumes tons of resources and is error prone.
Realistically, it’s just media visibility. People were doing ML research for ages before LLMs became ‘the next big thing.’ The things developed by that research are often incredible but also incredibly narrow. ML based protein folding systems were big news among nerds several years ago. The amount you have to explain to even have the groundwork to explain how important that development is could never fit in a clickbait headline or article. You know what does fit in a clickbait title? ‘Celebrity ignoramus talks to chatbot, decides it’s God.’ ‘AI company insider says their product is the best thing since sliced bread and everyone who doesn’t pay them money will commit suicide.’ and ‘You are doomed because of AI. Click here to find out why.’ No one outside of the field those narrow ML systems are built for can understand their output. Anyone can understand ‘You’re absolutely right!’
Accessibility to profit.
ML is more difficult, even if not impossible, to package in a way that is easily consumable by those with the capacity to pay for services.
An LLM could never have written that so perfectly
I definitely see that. Like android or iOS, getting people in your walled garden for one thing creates dependency and the payees are “saving” by only having to pay for 1 tool. It just feels like it would be more useful and less energy intensive as individual tools than an everything machine
I view “ai” (which I will always think of as a misnomer) as a bastardization of machine learning research that is positively contributing to the world and the literature as opposed to whatever the fuck the “AI” companies are doing.
This already exists as ‘Mixture Of Experts’ (MOE) models. It’s how Deepseek is able to remain competitive with a fraction of the budget.
It’s far more than that. ChatGPT uses MoE for sure and has been since GPT4. As such, it’s highly likely that Claude and Grok do too.
That’s the Unix philosophy you are describing, not Linux. Linux does not follow that, it’s just a common practice among programmers. See linux is a monolithic kernel containing all drivers for all hardware, newer example is systemd
First thing is that the market is pretty young. Imagine you are now using IBM mainframe PC to run chatgpt and not even windows1.0 on your IBM pc.
There are small AI models. Small agentic capable AI models that can do tool calling. GenAI and what you described. One thing is some solutions are mathematically impossible to gain certain quality. Other thing is they’re just not owned by billionaires and not used by corporations so press don’t cover them and nobody cares except few enthusiasts and people that need such capabilities and know limitations. Press only covers what they’re paid for or what can gain some traction.
The hate on AI as you described doesn’t help AI as a science domain. It’s not really related to science but related to ineffective spending of money by billionaires.
Well, mainly because I don’t need an LLM to tell me about history in a way that I would need to go verify. But ML is useful in, e.g. Home Assistant, with many speech-to-text models, as well as text-to-speech models, to choose from. You can hook those up to an LLM, so your voice assistant can understand vague or complex intents. There’s genAI image enhancement and painting tools available in Krita. There’s object recognition in Frigate, and Immich. Et cetera.
In short, the Unix philosophy in AI is what the FOSS community is already doing. It supports the things we want to do, in the background, by doing one thing and doing it well. It just doesn’t look like Copilot, with its single text input which takes over everything. (Which is what we hate because it’s bad.)
There are. You just need to know more about ai.
I mean people are. All the time. They just don’t get the attention things like lmk have.
For example, SAM 3 exactly what I think you are asking, but for images.
But there is another point in here about how"actually, just bigger model better" and that’s the thing with transformers. Them basically becoming chat bots through clever training and massive size and training datasets wasnt expected. You don’t get that behavior from much smaller transformers. And so there was an apparently emergent phenomena in this case. A small network isn’t going to do what you think it is going to do precisely because it’s over constrained.
Thanks, I think this is what I’m getting at. Is there an inherent advantage to all in one over modular? And it sounds like they’re is. I know over constraining is an issue with training and there is no scenario with ML or LLM where you get to 100% accuracy. It’s just not the point of the technology. But I could focus on getting an image editing tool 95-99% of the way there and test that vs. having that functionality bundled up with everything else and potentially have that function suffer as we improve another area. If a bigger transformer is benefiting from the other areas of expertise, that is interesting. I still believe you have to hit a point of diminishing returns where more bigger no longer equals more better
So I have a book on my shelf on complex systems analysis and that might be a place to start, but this concept of emergent properties in complex systems isn’t a new one, and it’s well established in complex systems theory, and especially true in network and graph theory.
Basically, complex systems, and especially networked systems, develop different emergent properties as they scale.
Do you have a source for ‘unexpected’?
“Attention is all you need” is the place to start that question, as this is where the transformer gets introduced, and it’s authored by the team at google brain. Notably, not OpenAi, who later authored a paper introducing gpt-1. Transformers were introduced as a way to shrink the model and support parallelization, not to make it larger.
Convolutional networks, lstms, kernel based vision models, unet all of that had already existed before, and yes, people had just thrown more complexity at the matter myself included, but it never resulted in the kind of pay off that transformers seemed to have been able to achieve.
So it’s not like the community hadn’t tried just throwing more compute or scale at the issues, it’s just that it didn’t result in this kind of emergent complexity that we’ve seen with transformers. And that’s true if networks in general. There is no guarantee that throwing “just more complexity” at the system is going to result in different properties. But there is also no guarantee it won’t. Practitioners of complex systems analysis understand this, that there are no guarantees regarding boundary conditions in complex systems.
And this is the second leg of support for unexpected, because if it was to be expected, why not just go there? And we can see that with the relatively asymptomatic performance we see in frontier llms. They are still improving. But marginally compared to the massive jumps we saw, for example between gpt1 and gpt2, or gpt 2 and gpt 3. Even at gpt 3 and gpt 4 we saw that asymptote beginning to form. And since four, improvements have been very meh, inspite of just throwing more parameters at the problem. Things have been improving but it’s largely around the engineering around the models, not the models themselves, inspite of throwing more and more complexity at the problem.
And maybe at one trillion parameters, there is some new boundary conditions which result in new emergent properties. But we don’t know that. So if we go there and find that out, it would be unexpected, at least in that we’ve been throwing exponential complexity at a problem to get sublinear performance improvements.
Attention is all you need: https://arxiv.org/pdf/1706.03762
Gpt-1: https://cdn.openai.com/research-covers/language-unsupervised/language_understanding_paper.pdf
Complex systems: https://onlinelibrary.wiley.com/doi/10.1155/2020/6105872
Note ML is a tech stack and AI is now a marketing term. It used to be a field of study and still is but the marketing term is so widely used its reduced the meaning of the original term. Kind of how 5G is a tech spec but then companies kept labeling their networks 5G before the 5G spec was certified. Then when the 5G spec came out it created even more confusion.
Ai is just the new buzzword. We remember the cringe of “cloud”.
There are plenty of small models that you can run locally, and they can be fine tuned on different types of content.
Back in around 2018 (2019?) it was all the rage, to try to “help” neural networks by smartly selecting the best network tp use, like you describe. Turns out it was just better throwing it all in one net, it makes a better selector too…
Then I have a question, what would you use it for?
That is interesting that the all in one approach had better results.
To your question, I still don’t have a solid use case to justify the fuss and energy consumption. I see it is a novel way to interact with your computer. I like the idea of an assistant I could talk to, but the surveillance piece is too creepy to get over using regularly. I’ve had ideas for specific tasks like 3D modeling and origami, but I’d like to know which models are good at that first. The only real uses I’ve encountered are trip planning or translation, but for the 2nd one I’d go to Google translate directly before Gemini. My question is mostly that I see the fuss and I don’t get the approach the industries seem to be taking.
For how LLM specifically struggle with that and what is done about it s white nice article from 2023 about MOE models:
https://huggingface.co/blog/moe#what-is-a-mixture-of-experts-moe
They explain the issue with “you need size but want specialization” quite well in my opinion.
Yeah MoE seems like basically exactly what they’re describing. Train individual models to be specialized in specific tasks, and string them together under a central coordinating model.
They kinda are. they hit a wall and started making specialist llms that then get queried by the main one. even have things where the main one evaluates repsonses from a few. im not 100% sure what all they are doing but its not monolithic.
There are examples like Spleeter, it separates music into individual tracks (drums, vocals, piano, guitar etc) Based on its training model.
But for marketing a paid product people expect one thing does it all.
