I recently helped evaluate it for company use. My test considered of vibecoding an App that takes a Pipewire screencast and does some basic image processing on each frame. In C#, which doesn’t have great options for talking to Pipewire. I did several runs with various iterations of Claude.
On the upside, it had few problems navigating DBus to negotiate a screencast handle. So that’s one annoying API out of the way.
On the downside, all attempts at taking to libpipewire through P/Invoke failed, usually because Claude hallucinated parts of the API or set constants to incorrect values. I only got a working program when I allowed the use of a prerelease Pipewire NuGet package.
The generated code was of acceptable quality but I wouldn’t allow it into a codebase without a refactor. The code has zero consistency and one time the whole solution didn’t have any namespaces. The fact that the LLM writes and rewrites the code during a single prompt means that you can get mild spaghetti as an initial state.
Honestly, I can see it for something like rapid prototyping or implementing basic scaffolding for an annoying API. But damn is Claude bad at detail work.
I recently helped evaluate it for company use. My test considered of vibecoding an App that takes a Pipewire screencast and does some basic image processing on each frame. In C#, which doesn’t have great options for talking to Pipewire. I did several runs with various iterations of Claude.
On the upside, it had few problems navigating DBus to negotiate a screencast handle. So that’s one annoying API out of the way.
On the downside, all attempts at taking to libpipewire through P/Invoke failed, usually because Claude hallucinated parts of the API or set constants to incorrect values. I only got a working program when I allowed the use of a prerelease Pipewire NuGet package.
The generated code was of acceptable quality but I wouldn’t allow it into a codebase without a refactor. The code has zero consistency and one time the whole solution didn’t have any namespaces. The fact that the LLM writes and rewrites the code during a single prompt means that you can get mild spaghetti as an initial state.
Honestly, I can see it for something like rapid prototyping or implementing basic scaffolding for an annoying API. But damn is Claude bad at detail work.