> AI Agents are starting to write more code. How do we make sure we understand what they're writing?
This is a good question, but also how do we make sure that humans understand the code that _other humans_ have (supposedly) written? Effective code review is hard as it implies that the reviewer already has their own mental model about how a task could/would/should have been done, or is at the very least building their own mental model at reading-time and internally asking 'Does this make sense?'.
Without that basis code review is more like a fuzzy standards compliance, which can still be useful, but it's not the same as review process that works by comparing alternate or co-operatively competing models, and so I wonder how much of that is gained through a quiz-style interaction.
I imagine the quizzer could ask better questions along those lines with better context engineering (taking entire repo contents, design docs, discussions, etc and compressing those into a mental model). I just took the PR code changes and comments, so there's a lot of improvements that could be made there.
Code review, to me, is not about validating the output. It's about a 2nd set of eyes to check for foot guns, best practice, etc. Code review is one step above linting and one step below unit tests, for me.
and I knew that `userModel` throws an exception when it doesn't find a user (and this is typescript, not java, where exceptions are not declared in the method prototype) then I would tell them to wrap it in a try-catch. I would also probably tell them to change the return type to `UserDTO | null` or `Result<UserDTO>` depending on the pattern that we chose for the API. I don't need to know anything about the original ticket in order to point these things out, and linters most likely won't catch them. Another use for code review is catching potential security issues like SQL injection that the linter or framework can't figure out (i.e, using raw SQL queries in your ORM without prepared statements)
Depends how good your QA is. Where I am it is terrible so most of the time I spend in “code review” is spent checking out the code locally and testing it myself.
Just submit a PR that removes the action so it doesn't run on the branch before the merge! If devs aren't reviewing the code anyways, will they even catch that kind of change?
LLMs are quite bad at understanding intent behind the code if it is original and involves math-heavy tricks. But for most apps it will probably be fine. What's the workflow if it makes a mistake though?
Saw an actual PR that says "this was generated with claude, please review carefully". Since when did we stop taking responsibility for what is submitted?
Ok I’ll bite: putting « only » implies this is not a big deal and a lesser of 2 evils, between an AI model provider harvesting prompts for retraining and a 3rd party hosting provider most probably only storing logs for security and accountability…
So yes this is the second part of the privacy statement
See, I think this is a good idea even for reviewing non-agentic human-written PRs!
We've got a huge LGTM problem where people approve PRs they clearly don't understand.
Recently we had a bug in some code of an employee that got laid off. The people who reviewed it are both still with the company, but neither of them could explain what the code did.
The only way I’ve ever seen engineers care about PR’s is if the software or product is tied directly to their paycheck. If uptime or bugs directly impact a quarterly bonus, or result in a layoff / getting fired, they spend a lot more time reviewing PR’s. Furthermore, the work and its estimate is expanded to include enough time for the team to thoroughly review the change.
Unless someone is getting fired for bad code the “lgtm” culture will never die.
I would probably be putting devs on a pip or firing them if they failed these quizzes often...understanding your own prs is the bare fucking minimum, even without AI help.
This is a good question, but also how do we make sure that humans understand the code that _other humans_ have (supposedly) written? Effective code review is hard as it implies that the reviewer already has their own mental model about how a task could/would/should have been done, or is at the very least building their own mental model at reading-time and internally asking 'Does this make sense?'.
Without that basis code review is more like a fuzzy standards compliance, which can still be useful, but it's not the same as review process that works by comparing alternate or co-operatively competing models, and so I wonder how much of that is gained through a quiz-style interaction.
If someone were to submit this code for review:
and I knew that `userModel` throws an exception when it doesn't find a user (and this is typescript, not java, where exceptions are not declared in the method prototype) then I would tell them to wrap it in a try-catch. I would also probably tell them to change the return type to `UserDTO | null` or `Result<UserDTO>` depending on the pattern that we chose for the API. I don't need to know anything about the original ticket in order to point these things out, and linters most likely won't catch them. Another use for code review is catching potential security issues like SQL injection that the linter or framework can't figure out (i.e, using raw SQL queries in your ORM without prepared statements)The initial idea was applied to classroom settings.
An Inquisitive Code Editor for Addressing Novice Programmers’ Misconceptions of Program Behavior https://austinhenley.com/pubs/Henley2021ICSE_Inquisitive.pdf
> Your code is only sent to the model provider (OpenAI)
When has this become an acceptable « privacy » statement?
I feel we are reliving the era of free mobile apps at the expense of harvesting any user data for ads profiling before GDPR kicked in…
So yes this is the second part of the privacy statement
We've got a huge LGTM problem where people approve PRs they clearly don't understand.
Recently we had a bug in some code of an employee that got laid off. The people who reviewed it are both still with the company, but neither of them could explain what the code did.
That triggered this angry tweet
https://x.com/donatj/status/1945593385902846118
Unless someone is getting fired for bad code the “lgtm” culture will never die.
They don't, that's why we need the PR in the first place.