I wouldn’t normally talk about topics like this here, but I felt like this was such a profound result that it deserved a place among the other subjects I’ve talked about.
Last year, I got a lot of questions on Quora.com re. whether computer science was worth studying “if AI is going to be doing all of the programming.” I had been hearing results of test runs using what’s now called “AI” (really LLMs, Large Language Models) to write whole applications, or games, and the results didn’t impress me much. They were slow and buggy. I’d also heard accounts from software engineers who had tried it, and also had low expectations for it, from the results, saying they needed to be checked over, because they didn’t necessarily conform to what was needed.
Other times, there were some embarrassing disasters reported, using AI code generation.
I have not used AI tools in software development, because I haven’t felt the need to. I have not been employed in the software industry since 2006. I’ve been off doing my own thing with programming for many years. So, I don’t have a dog in this fight right now, but I am sympathetic to those who are looking at this and wondering what to do.
The impression I was left with in my research on this was that AI tools were good for writing small, basic routines, an aide to software developers. Given this video from Bryan Lunduke, from a recent test run he did using Grok Bot, I am given a much different impression.
He talked about writing simple apps. for a new distribution of Linux he’s been working on (LCOS–Lunduke Computer Operating System), and he described how each took a few hours to develop, using the agents that were available on the platform; what would otherwise have taken weeks or months of work, if he were developing them himself.
To top it off, he talked about using the same AI platform to revise the Linux kernel he’s working on, adding back support for the Intel 486 (where the Linux kernel org. has dropped support for the 486, as Lunduke has documented). Testing the result, he’s seen it work on old 486 hardware.
It seems like what’s made this work well is a change in approach. Rather than using a single system to write the code, Grok Bot offers a set of agents that seem to specialize in certain software development tasks. Some of them are specialized in generating code. Some are specialized in writing test suites. You can take the generated code, hand it over to the test suite agents, and they’ll run a battery of tests to make sure the code meets some requirements. Lunduke talked about how in some cases, the test agents had found some bugs. This feedback was then handed back to the code generation agents, which then fixed them.
He talked further about how other parameters could be given to these agents, to further revise the code, such as making the software take up less memory, and he saw how they were able to accomplish this, while, it seems, not introducing new bugs.
What impressed him most was that while the early versions of the software he had this system work on were buggy, it wasn’t that hard to get the bugs fixed, and only increasing the stability of the software, rather than playing whack-a-mole with bugs in further software revisions.
Another thing was he said the performance of the software was good.
He was definitely seeing that such platforms were viable for generating usable software.
He sounded some cautions re. how people who weren’t careful could adopt this technology and produce junky software with it.
One of my concerns has been re. code maintenance. How do you add features to software after you’ve gotten a “1.0” version out of this? You haven’t written a jot of the code, and so it’s going to be unfamiliar, and difficult to change yourself. Lunduke seemed to answer that, saying he found it easy to revise and add features to the software, just by instructing the agents on what to do. So, he didn’t have to get his “hands dirty” with the code itself.
He said, overall, he felt like he was acting like a tech CEO. I’d say it’s more like being a project manager, or VP of Engineering. Rather than writing the code, you’re directing the overall design, and the agents are writing the code.
He ended by saying he had second thoughts about the title of his video, asking, “Is AI the future of programming?” It’s not the future of programming, he said, but it is the future of software development.
This, to me, was a watershed moment, because I didn’t think we would be seeing something this good for many years, if ever. I still have questions about the limits of this software development methodology. It seems to me for commercial software that you want to be reliable and secure, software engineers need to review the generated code to get rid of latent bugs and security vulnerabilities, because, as I’ve been saying all along, these systems don’t have a sense of what a correct implementation is. They’re very good at recognizing, and doing things with patterns, with some other functions put in to provide useful support for desired tasks. If I was in charge of software development, I wouldn’t feel comfortable with everything being a black box that couldn’t be inspected. This isn’t to say that these AI systems hide the source code. What I mean is people being lazy about these issues, and not wanting to do reviews on the code.
Lunduke’s analysis of this was interesting, saying that the productivity gains are so good that it’s worth the risks re. what can go wrong.
The implications for the software industry are quite large, because it will cause people to question its current structure. If someone at home can write professional-quality apps. in a few hours, or a few days this way, then what’s the point of having a staffed software company for the same types of applications? So many more people will be able to generate their own software, just for the price of using AI tools, that it’s going to make many software companies obsolete.
Granted that not everyone will have the desire to do even this activity, I can imagine that software consultants could make a business of generating software for customers, but it seems reasonable that these would be small shops, and that there would be a multitude of them, because the barrier to entry will be lowered so much by this technology.
It reminds me that back in 2009, after I’d attended a computer science summit called Rebooting Computing, I had some conversations with attendees. In one of them, I expressed concern that there wasn’t enough emphasis at the summit, or in the industry, on funding basic research in computer science. This left the industry running on “fumes,” wringing value out of the ideas that had been generated from research done decades earlier. The LLMs that have such hype around them now are based on this decades-old research. What’s made them possible is that hardware speed has gotten so fast, and storage capacity has gotten so cheap and scalable, that it’s possible to build these systems. The industry has only been focused on optimizing these old ideas. I predicted that one day, this would leave those trained in computer science with a shrinking amount of relevance, because the optimizing process would be relentless, possibly eliminating much of the work in programming for commercial purposes. I didn’t have a good idea of how that would happen. I was just projecting forward, based on what I’d already seen. I see that playing out here. Though, these AI systems are based on knowledge in computer science. So, its relevance isn’t completely going away.
On the other hand, I’ve seen an example with AI development that has born out a prediction that was made by many a couple years ago, that if these tools take over development, it will just be a new form of programming. In this example, what was given to an AI system looked like a programming language of some sort. So, I see this technology cutting both ways.
It’s going to change the software industry in some profound ways. I don’t think there’s any denying that, anymore. It doesn’t seem to me that we’re at the stage yet where a complete novice can just say, “Give me an app. with these features,” and it’ll just pop out. Even with what Lunduke did, I can tell some technical knowledge was still necessary. So, programming is not going away, but the “feel” of doing that work is going to change quite a bit in the industry. I agree with Lunduke that it feels unsettling to admit this, because there is an element of fun in this work, and it’s not something that we like giving up.
However, we don’t know what this will bring, because every productivity increase creates an excuse to have ideas that wouldn’t have been entertained otherwise. So, I don’t think that “work is going away.” It will feel that way, at first, for people in this business, but that will change as time goes on. It is going to require people to adjust their skillsets. Skills that had once seemed relevant, for decades, are increasingly going to be seen as obsolete. New skillsets will be needed in this new paradigm.
I don’t say any of this to say that what I’ve been exploring, and discussing on this blog is now obsolete and pointless. Like I said, I don’t have a dog in this fight. While I once had the notion that the ideas I was exploring could be used for a competitive edge in the commercial software industry, I realized that was missing the point. The point, which is in the vein of computer science, is to use programming as a means to model ideas, to run experiments on them, and to learn about them. This mode of work has no interest in competing in the realm of producing software that people will use to get utilitarian tasks done. Though, it is not hostile to that pursuit.
