Doctors have to follow the Hippocratic Oath, swearing to do no harm to their patients. Developers ought to be following a similar oath, promising to do no harm to their codebase when implementing new features or making changes.
Mitchell Johnson, chief product development officer at Sonatype, explored this concept and if it’s even still possible in the age of AI-assisted development during the most recent episode of our podcast What the Dev.
“In the context of the medical field, physicians are taught ‘do no harm,’ and what that means is their highest duty of care is to make sure that the patient is first, and that they do not conduct any sort of treatments on the patient without first validating that that’s what’s best for the patient,” said Johnson. “When they roll a patient in and the chart says, ‘we need to cut this patient’s leg off,’ obviously, it’s the responsibility of that physician to make sure that’s the treatment that the patient needs. They can’t point to ‘hey, it was on the chart.’”
The responsibility for software engineers is similar; When they’re asked to make a change to the codebase, they need to first understand what they’re being asked to do and make sure that’s the best course of action for the codebase.
“We’re inundated with requests,” Johnson said. “Product managers, business partners, customers are demanding that we make changes to applications, and that’s our job, right? It’s our job to build things that provide humanity and our customers and our businesses value, but we have to understand what is the impact of that change. How is it going to impact other systems? Is it going to be secure? Is it going to be maintainable? Is it going to be performant? Is it ultimately going to help the customer?”
Before AI, developers were spending about 40% of their time writing code and 60% reviewing it, but now AI is allowing them to generate code at such a rapid pace that these ratios are no longer accurate.
Johnson posed the question that if developers are generating code 50 times faster than they used to, can they still do those quality checks and follow the developers’ Hippocratic Oath? He believes the answer is yes.
He explained that the problem, however, is that this speed creates pressure to ship without doing as thorough of an inspection, because if code is being written faster, there’s a desire to get it to production faster.
Last year’s DORA report showed that a 25% increase in AI adoption was associated with a 1.5% decrease in delivery throughput and a 7.2% reduction in delivery stability.
“What’s interesting is what actually creates speed,” Johnson said. “We all love speed, right? But faster coding is not actually producing a high quality product being shipped. In fact, we’re seeing bottlenecks and lower quality code.”
He went on to say that testing is the discipline that could be most transformed by generative AI. It is really good at studying the code and determining what tests you’re missing and how to improve test coverage.
He said that the best organizations are not just using generative AI to write code faster, but to do everything else faster as well. He did warn, however, that we’re not quite at the point where generative AI can 100% write the code and then test that code. This is largely a result of the fact that the biggest problem with generative AI is that it’s trained on old data.
“You can do a simple experiment: go out and ask your favorite generative AI model to pick a simple dependency on a project you’re working on, and you’ll see it often recommends dependencies that are 12 months or even two years old, which is obviously a very dangerous thing. The bad actors out there are hoping that the world starts adopting two year old dependencies,” he said.
He believes the solution to this problem lies in spec-driven development, a new practice in which designers, developers, security teams, and product managers are all working together and writing specifications that are optimized for generative AI models.
“You can make sure that it has your context, and you can make sure that the non-functional requirements around testing, security, and compliance are baked into the specs,” Johnson said. “And you can start having those specs and those rules files preceded in the context of your generative AI and you can really effectively touch on those other areas, not just can I write code faster? The organizations that are getting the most out of generative AI are adopting this spec-driven approach and incorporating things like security and testing as a first-class citizen in the generative AI SDLC that they’re adopting, and they’re starting to see not just speed gains, but quality gains and security gains.”
