A cruel irony of coding agents is that everyone who blew off automated testing for the past 20 years is now telling the AI to do TDD all the time. But because LLMs were trained on decades of their shitty tests, the agents are also terrible at testing.
I've noticed a direct correlation between "took automated testing seriously" and "has success with coding agents". So there's that, at least.
I’ve seen some advocate that the test code shouldn’t ever be generated by AI - that it needs to remain a human task, which I think is a good balance.
This…this is just straight up not true. The anti-AI karma farming is getting a bit obsurd on LinkedIn. Sure, if you crack open a project for the first time with a brand new coding agent and say “write me some tests”, they are going to be absolutely terrible. Just as terrible as the tests you would get if you asked a brand new engineer to write and expected the results to come back the same day. However if after you’ve provided information about the project, its structure and purpose, information on how to test including what to do and what not to do, and when to think harder or ask for questions, what workflow to use when writing tests, etc through the mechanisms available to the coding agent — well, now suddenly the tests you’re getting are pretty great aren’t they? Like _real_ farming, input quality matters to maximize potential output, and you can’t neglect the care needed in between to increase your chances of hitting the max.
It’s turtles all the way down. Only we’re the middle man to keep the chaos in order.
Oh this is why I always spend a bit of time getting a couple of test examples ready, it's way better when you've got tests written in a way that aren't using a million mocks
Tests aren't usually to make sure it works. They are to make sure someone else doesn't break it. (Which may include your future self). If you give a project to an agent. It won't care about existing tests and will just update them all inclusive. Thus defeating the primary purpose of tests. I'm speaking in a general sense and a bit humorously. I value good tests.
This is spot on. They don’t just copy bad tests. They copy all of our bad habits. I find LLMs to be particularly bad at writing Terraform. I posted about. This a few weeks ago. https://www.proactiveops.io/archive/bots-write-bad-terraform-and-its-all-your-fault/
1. Denial - AI makes me 10x more productive! 2. Anger - you don’t know how to prompt, it’s a skill issue 3. Bargaining 👈 we are here 4. Depression 5. Acceptance
TypeScript / JavaScript / Java / Jenkins / AWS
3wSee I just skip all this bullshit by doing the psycho thing of writing my own code and tests.