Accepting AI-written code without understanding how it works is growing in popularity.
See full article...
See full article...
Is “vibe coding” with AI gnarly or reckless? Maybe some of both.
- Thread starter JournalBot
- Start date
Yeah but if Facebook programs their next algorithm feeding me ads that way or EA codes their next FIFA installment that way, I’m sure I couldn’t care less.
Upvote
15
(16
/
-1)
Almost all of my formal education in programming comes from law classes. I have noticed over time that length of experience in the field seems inversely proportional to someone's likelihood to laugh and think it's a joke.
I know that a lot of people compare programming to math, but to me the logic concepts seem much more verbal/language (in the WAIS VIQ sense). And debugging, of course, is just fancy proofreading.
But if you don't understand the underlying logic of the system itself, I don't see how you can possibly hope to have your intended effect on it, whether you're editing source code or legal code.
It's almost like I'm watching people running desperately away from a place to stand while ditching every lever...
I know that a lot of people compare programming to math, but to me the logic concepts seem much more verbal/language (in the WAIS VIQ sense). And debugging, of course, is just fancy proofreading.
But if you don't understand the underlying logic of the system itself, I don't see how you can possibly hope to have your intended effect on it, whether you're editing source code or legal code.
It's almost like I'm watching people running desperately away from a place to stand while ditching every lever...
Upvote
23
(23
/
0)
Upvote
18
(18
/
0)
I am literally just a guy who learned programming from law classes, but in this case isn't the actual problem here the fact that you have undocumented code?
That's like suggesting iron supplements and isotonic sports drinks to counteract hypovolemia and exsanguination from being stabbed. The bleeding knife wound might be the obvious starting point.
Upvote
19
(21
/
-2)
Look I'm not a coder, I'm a network engineer. But we have the same types floating around. They figure out how to make an EVPL and they save the little script and punch it in. 95% of the time it's fine, it works. But the 5% of the time they need to step out of the script a little and maybe pop two tags, or swap tags, or whatever... it doesn't work and they have no idea why. Everything will look absolutely fine in the router, because it doesn't check to see that you have the same VLAN on both ends. And because they don't understand the actual commands they've entered, how it affects the traffic, and how the router interprets the traffic, they can't spot it, they can't fix it, and frankly it'd be faster for me to just do it right the first time than getting roped into fixing it and soothing the customer after they've spent an hour on the phone with the vibe network engineer trying to fix it getting nowhere.
edit: hit enter too soon on my rant.
Last edited:
Upvote
41
(41
/
0)
Hell, there's always the old fallback if you need to test the maximum size of a single integer your system can handle:
integer i=1;
while(i>0)
{
++i;
}
clean up the pseudocode for whatever language you're using, ought to compile just nicely. It'll even work just great for a while...
Upvote
9
(9
/
0)
And took many of us down a notch when the lab assignment we turned in, that we thought worked perfectly, didn't in some of the edge cases the TAs' software tested for. And that was long before LLM AI existed.
Getting a deserved bad grade usually teaches most CS majors that their code isn't infallible long before they get into the professional world. (Just one reason a CS degree from a reputable college/university is valuable.)
Hell, I even did a lab assignment on circuitry once that I then compared to a friend's who had taken the class before. (He got an A on the lab.) Mine was basically identical, but it turned out the TAs' had found and started testing for a new edge case, so I got a B on the same lab. Just goes to show that even code that seemed perfect in the past can be hiding bugs. LLM AI generated code's going to be rife with them. It's how coding and LLM logic works.
Upvote
38
(38
/
0)
Upvote
9
(9
/
0)
What do you mean? Putin understands the American government very well.
Upvote
12
(12
/
0)
icebergFromLaputa
Wise, Aged Ars Veteran
If we can have vibe-based programming, we can have vibe-based legislation too. Just select random citizens (method known as Lottocracy) to write laws, except they can ask ChatGPT to write laws for them. Then give resulting laws to professional lawyers for review (analog of the compliator), if something wrong send their feedback to ChatGPT, in hope it will fix problematic places. /s
Upvote
9
(9
/
0)
How is tonight different from all other nights?
No, really, this is the system we already have if you just replace "ChatGPT" with "lobbyists".
Upvote
9
(9
/
0)
icebergFromLaputa
Wise, Aged Ars Veteran
Because lobbists have some kind of a plan in their mind, they optimize the system in direction of powerful corporations. In contrast, ChatGPT has no fricking idea what it is doing and will write random meaningless laws.
Upvote
9
(10
/
-1)
I heard that Grok will be writing the new automation for Flight Controller Software Automation for the FAA...
Upvote
4
(4
/
0)
Using "AI" (Large Language Models) for anything but the most basic things, that you should be anyways considering and manually coding is insane.
For anything serious (requiring extensive documentation, requiring no useless code, requiring optimizations and reproducibility, in short, anything that requires safety or accuracy, which is... almost everything), this is absolutely nuts.
The moment that you have to do anything integrated into a bigger system and then build on it, it's game over, since the lack of documentation or understanding would make the code take longer to debug than to start from scratch and properly document.
Unlike "coding", the actual jobs that require software writing are engineering, which requires understanding of a problem, carefully weighting in pros and cons of implementations, language used, hardware where it will run, and edge cases, none of which LLMs can remotely come near to touching, let alone writing anything even 1% correct.
For anything serious (requiring extensive documentation, requiring no useless code, requiring optimizations and reproducibility, in short, anything that requires safety or accuracy, which is... almost everything), this is absolutely nuts.
The moment that you have to do anything integrated into a bigger system and then build on it, it's game over, since the lack of documentation or understanding would make the code take longer to debug than to start from scratch and properly document.
Unlike "coding", the actual jobs that require software writing are engineering, which requires understanding of a problem, carefully weighting in pros and cons of implementations, language used, hardware where it will run, and edge cases, none of which LLMs can remotely come near to touching, let alone writing anything even 1% correct.
Upvote
6
(9
/
-3)
Vibe-programming?
Why does that make me think of Donald Sutherland playing a hippy tank commander in a WWII movie with Clint Eastwood and Telly Savalas?
"vibe" + "programming" are two concepts that really don't belong together.
Why does that make me think of Donald Sutherland playing a hippy tank commander in a WWII movie with Clint Eastwood and Telly Savalas?
"vibe" + "programming" are two concepts that really don't belong together.
Upvote
21
(21
/
0)
They actually sounds like a decent alternative to what we got. No /s
Upvote
4
(6
/
-2)
Why stop there? Let's just run everything on vibes - vibe engineers, vibe doctors, vibe lawyers, etc. (who needs boring medical school or law school?)
And when silent bugs slip into production, you can feel the vibes of market losses and lawsuits. At scale!
There's something greatly disturbing about current society's apparent pride in laziness, corner-cutting and low effort, all while desiring maximum returns (the "social media influencer" effect, I suppose).
What happened to acquiring skills? True, artificial barriers to entry should not exist, but there should be qualification thresholds, which require effort to be met.
And when silent bugs slip into production, you can feel the vibes of market losses and lawsuits. At scale!
There's something greatly disturbing about current society's apparent pride in laziness, corner-cutting and low effort, all while desiring maximum returns (the "social media influencer" effect, I suppose).
What happened to acquiring skills? True, artificial barriers to entry should not exist, but there should be qualification thresholds, which require effort to be met.
Upvote
23
(23
/
0)
I really hope that my competitors use these AI code creation tools. I mean it. I'll clean up.
Anyway, instead of some sort of AI code generator, I'd like a tool that relieves me of the tedious aspects of programming (I use C++ and really only as a bastardized C). I'd especially like a system that generates code at runtime to handle the given inputs. For example, I have code all over the place that deals with array bounds, etc. That's because it has to deal with all possible input parameters. On-the-fly generated code could pre-validate the inputs and generate clean code that would be much faster.
For example, if I want to apply a 5x5 kernel to an array of data, I need to deal with the edge cases, where the indexes need to be checked to see if they're outside the bounds (and what to do if it is). I'd much rather "describe" to the compiler the problem and have it generate the required code.
Modern compilers have exceeded my ability to generate x86 code in bulk. It's not even close. There are few remaining areas where I could beat the compiler, but those are mostly special cases where I know ahead of time what input parameters will be given.
Anyway, instead of some sort of AI code generator, I'd like a tool that relieves me of the tedious aspects of programming (I use C++ and really only as a bastardized C). I'd especially like a system that generates code at runtime to handle the given inputs. For example, I have code all over the place that deals with array bounds, etc. That's because it has to deal with all possible input parameters. On-the-fly generated code could pre-validate the inputs and generate clean code that would be much faster.
For example, if I want to apply a 5x5 kernel to an array of data, I need to deal with the edge cases, where the indexes need to be checked to see if they're outside the bounds (and what to do if it is). I'd much rather "describe" to the compiler the problem and have it generate the required code.
Modern compilers have exceeded my ability to generate x86 code in bulk. It's not even close. There are few remaining areas where I could beat the compiler, but those are mostly special cases where I know ahead of time what input parameters will be given.
Upvote
-2
(3
/
-5)
While I can see the appeal of the ease AI coding, I feel it is hopelessly naive and a dereliction of duty to also throw away your unit tests. It is probably the case the AI can write those too, but mindlessly copy pasting code in without proof of function will make your name Mud soon enough. Don’t do that.
Upvote
9
(9
/
0)
This is what libraries like Intel Performance primitives or Accelerate.framework are for. They are faster and much better tested.
Upvote
17
(17
/
0)
Question:
I am now retired, 45 years programming , mainly in 'C' and laterly C++ on Unix then Linux at the systems level.
Being retired I have no interest in playing with AI tools but my question to those that have is this.
Probably 40% of what I programmed within any specific program was code that didn't actively move you towards whatever the solution was but was rather error checking (inputs, intermediate result sets, final outputs, resource allocation/opens/closes, sub process execution checks etc)... so do these AI tools give you the full enchilada (i.e. with error checking) or just some bare bones slab of code that will fail with the slightest/smallest edge case or intermediate error?
Bluck
I am now retired, 45 years programming , mainly in 'C' and laterly C++ on Unix then Linux at the systems level.
Being retired I have no interest in playing with AI tools but my question to those that have is this.
Probably 40% of what I programmed within any specific program was code that didn't actively move you towards whatever the solution was but was rather error checking (inputs, intermediate result sets, final outputs, resource allocation/opens/closes, sub process execution checks etc)... so do these AI tools give you the full enchilada (i.e. with error checking) or just some bare bones slab of code that will fail with the slightest/smallest edge case or intermediate error?
Bluck
Last edited:
Upvote
17
(19
/
-2)
This sort of thing can happen to humans too. I was for a while responsible for writing tests for some powerPC SIMD code. For the 3.2 people out there who have done that, you will know that running off the end of the array is super easy to do by mistake, even for experts. We fixed this by exhaustive brute force testing against buffers at various alignments against guard pages. It was exhaustingly effective at finding bugs, with emphasis on exhausting. It can be super depressing to be pulling bugs out of your code for day after day for a week on a simple function.
I wasn’t the only guy on the team. A big problem was that with the mental fatigue of fixing bug after bug after bug and sometimes painting yourself into a corner, eventually engineers would get worn down and put in “whatever! It works!” fixes that did not in fact work but either broke the reference implementation or managed to copy a bug in the reference implementation to the optimized code.
It is very important when fixing bugs to understand why your bug fix is the correct fix. If you don’t understand that, odds are good your tests pass for all the wrong reasons. I’m quite skeptical the AI is really capable of that level of professional integrity.
Upvote
23
(25
/
-2)
Why would anyone pay someone to ask an AI to write code when they could just ask the ai to write code?
Upvote
9
(9
/
0)
The data I'm processing can have "missing" values, which require falling back to smaller kernels and then to a separate routine that works in the face of missing data. I don't imagine a stock library will handle that case (and why would it, that's not a standard convolution operation).
And let's not get into the hairball of multithreaded programs, where you need to protect access to common data structures efficiently.
Upvote
-2
(2
/
-4)
So there is another problem. If you don’t preserve your prompts that generated the code, how maintainable is this code? Is it one and done for the AI and humans maintain it forever more, or are we talking about full synthesis here.
I ran into a similar problem in my career with lookup tables. Often people quite happily write down a pile of numbers without commenting how they were derived. This is a big problem for vectorization which, with some exceptions, generally can’t do lookup tables but would be faster if we just put the original calculation back in.
It can be oh so much fun to spend a day reverse engineering a pile of numbers back into a closed form function and even more fun when you conclude the table is probably wrong because it rounds down instead of to nearest or one number is missing and the rest of the table shifted to compensate. WTF do I do now? Fix the original table then vectorize it? Maybe there is something I’m missing here? Bug for bug compatible with a nasty comment?
Mapping back to AI, it is easy to see how the original work from the prompt engineer is easily discarded and getting fixes out of another prompt engineer (once all the C++ guys are dead like COBOL) is going to be a major hassle. / complete rewrite. Probably by then the AI will have changed too.
Upvote
11
(11
/
0)
Agents absolutely can. One way to do it is, as I hinted earlier, to start with high level docs and examples then divide and conquer. Like a human does.
It’s silly to copy and paste from a language model. Not necessarily silly to have multiple agents working together towards a common goal, using tools to validate their code.
And in some languages, with some constraints the worst that can happen is a logic error. You can even make panics impossible with the right constraints.
My advice to anybody getting into software engineering would be learn to write these agents as you learn about a production environment.
Experience delegating work to people or even to yourself translates into directing agents. Coordinating. Planning, documenting, writing tests, writing the code, and then, finally, deploying.
Upvote
-3
(3
/
-6)
icebergFromLaputa
Wise, Aged Ars Veteran
I guess this is why people try to create more foolproof languages, like by creating Rust. The ultimate goal is such languahe, that if the code compiles, then it must work, and if it is broken then compilation should fail. Under such conditions you can hire less educated and experienced programmers. At the limit, literal monkeys randomly hitting keys, while poor compiler allows only such random code changes from the monkeys that, by lucky coincidence, make sense.
Last edited:
Upvote
-18
(1
/
-19)
I would like to think my husband and I will be employable forever thanks to that, but I don’t really believe it. AI will reach a point where even their C++ is better.
And that’s not necessarily hard. At the risk of a self own and offending my husband, C++ developers, in general, have an overinflated view of their ability.
It’s why I prefer Rust and favor generating in Rust over any other language. I can, at least, be certain some classes of bugs can’t happen.
And I’d it compiles, and tests pass, especially if you write them first, there’s a very good chance the code will work as expected and require minimal maintenance.
Upvote
-3
(3
/
-6)
There are sparse libraries out there for BLAS, for example. If your convolution is locally rectangularly dense then these things should still work. VImage for example has options to turn off its own internal multithreading if you would like to set up your own for the dense segments. It can also render out of the middle of a larger image structure and doesn’t require an opaque data container. It just needs a pointer and a row stride and rectangular size.
Upvote
1
(1
/
0)
Or ones who realize, historically, C and C++ is dangerous for anybody or anything to write in.
The kernel we all use for almost everything daily is chock full of memory safety issues because of arrogant assholes who think they’re “educated” enough to not blow both feet clean off.
Upvote
7
(7
/
0)
Depends on what you write. The software development pipeline you design for your agents. The limits are your creativity.
They can absolutely be given a suite of tools and cover every last line if you want.
Upvote
-8
(1
/
-9)
As a programmer, I love this notion that one might be able to craft and test law using a legal programming language of some sort. This is because from a professional point of view the effort and rigor used to craft legislation is laughably poor, and frankly shocking. It could use some rigor and well, unit tests.
Please do start with tax law and a few general failure cases such as anything that triggers a tax cliff is a bug.
Republicans make great hay over the law of unintended consequences, but for software engineers, dealing with those surprises is all in a days work. It is not actually necessary to ship problematic legislation. “We’ll fix it after passing the bill in maybe 10 years if the right lobbyists pay.” It’s scandalous what passes for rigor in law.
Last edited:
Upvote
13
(13
/
0)
Well, at least in CS, they will do what they always did since they are already having to solve the find-the-work-of-the-Bard-in-million-monkeys problem. Write a pile of unit tests and see who passes. They will not find it hard.
I do feel for the Humanities or social science professor who has to detect fake citations though.
Upvote
-1
(1
/
-2)
The judge can exact vengeance. Doomed airline passengers, not so much.
Upvote
8
(8
/
0)
I think there is an important difference between using a traditional software application to program a computer versus using an AI tool to code. Traditional software is deterministic: you give it the same instructions and the computer will do the same thing each time. Even if you don't "know" the programming language, you could (with sufficient time and patience) read and comprehend the code.
But an AI is a figurative black box that no-one can properly explain, and more importantly, it doesn't always generate the same output for the same input. It seems inevitable that anything but the most straightforward code generated by an AI will have something that is inexplicable, something that makes you ask "Why is that there? What does that do?"
Upvote
8
(9
/
-1)
Don’t worry, AI is going to create a glut of work for experienced devs someday, because someone’s going to have to clean up all the damage “vibe coding” will do.
Upvote
9
(9
/
0)
Nah, the AI is writing source code just like a human and compiled using the same compiler using the same ISA and hardware. It will be deterministic and if it isn’t, a human could make the same non-deterministic error. Various compiler flags / developer tools to spot things like uninitialized variables or undefined behavior are standard tools to clean up the rest. At worst you can chose something like Swift or Rust with is designed to catch a lot of that stuff automatically at compile time. There are too many crufty coders out there for bad code not to be a heavily explored sector out there with long experience dealing with it. One more monkey trying to write Shakespeare is t going to upset the apple cart.
This criciticsm is valid for self modifying code but we humans have learned the error of our ways on that one and few would be so dumb as to teach an AI to do that. That is, you won’t find examples in code except loosely in a couple of JITs and loaders, so it seems unlikely to be heavily reinforced in the AI model.
As far as indecipherable code, other engineers and sometimes even previous-me do that to me all the time. You just need to get good at reading the tea leaves! “What fresh new hell is this?” will be the daily refrain of the senior engineer. There are people who make a living deciphering assembly looking for exploits. This is a solved, if highly time consuming problem.
Nobody except the AI knows how it works?! This is hardly different from Joe’s code after Joe moved on to a different company. Nor is it a new problem, see automated translation tools like f2c. Somebody just going to have to learn. Own the code.
Last edited:
Upvote
-11
(3
/
-14)