> I almost always leave a comment on each PR I review, even just observations: “This class is getting big, we might want to consider adding a presenter,” or praise: “Thanks for cleaning this up!”
Things like that I'd much rather leave as comments in the code, rather than dangling as off-hand things in some unrelated PR. Probably no one will see those PR comments unless they go splunking for some reason, but having them right in the code is much more evident to all the people passing through, or even reminding yourself in the future too.
In general I feel like PR reviews are basically reviews happening too late, and if there is a lot of stuff to review and agree upon in the PR, you'll be better off trying to reduce that up front by discussing and noting design decisions before the work even starts. If there is so many unknowns first, do one step to figure out how it can be done, then agree on that, again before starting the actual work.
That leads to PRs basically just being spot-checking, not deep analysis nor even space for nitpicks. If you as a reviewer spots things after the implementation rather in the discussion beforehand, that ends up being on both of you, instead of just the implementer who tried to move along to finish the thing.
You might be approaching PR comments differently than I've seen. When a comment is something to be addressed, it's either put into a new development task (i.e. on something like Jira), or it is completed before the PR merges. I'm not sure that having comments in the code surfaces that information in a useful manner. The code is for the code, not for what the code could be. The comments on what it could be should be handled outside the code at a different abstraction layer.
> In general I feel like PR reviews are basically reviews happening too late, and if there is a lot of stuff to review and agree upon in the PR, you'll be better off trying to reduce that up front by discussing and noting design decisions before the work even starts.
I agree in general and tried to push for a more iterate approach in our team.
However, my fear is that this would multiply the effort because it's very easy to discuss many things that do not matter or turn out later to not matter.
It seems it's easier for people to only discuss the important stuff when presented as an actual implementation.
We are talking tendencies here, of course, general design decision must always be done beforehand.
> It seems it's easier for people to only discuss the important stuff when presented as an actual implementation.
LLMs help a lot here, create two prototypes with both designs, compare them together :) Could even evaluate how ergonomic some future potential change is, to see how it holds up in practice.
I used to use pen and paper to essentially do the same, minus having real code and instead just talking concepts, but it does suffer from the issue that some need to be confronted with code in front of them to visualize things better for themselves.
I'm not following this. PRs are the first time your reviewers have seen that change, so there is no opportunity downstream to do the things you're suggesting.
You're essentially suggesting pre-PRs, but it is circular, since those same pre-PRs would have the same criticism.
PRs are about isolated core changes, not feature or system design. They answer how not why.
Usually by the time a PR has been submitted it's too late to dig into aspects of the change that come from a poor understanding of the task at hand without throwing out the PR and creating rework.
So it's helpful to shift left on that and discuss how you intend to approach the solution. Especially for people who are new to the codebase or unfamiliar with the language and, thanks to AI, show little interest in learning.
Obviously not for every situation, but time can be saved by talking something through before YOLOing a bad PR.
Yes, it should be cheap to throw out any individual PR and rewrite it from scratch. Your first draft of a problem is almost never the one you want to submit anyway. The actual writing of the code should never be the most complicated step in any individual PR. It should always be the time spent thinking about the problem and the solution space. Sometimes you can do a lot of that work before the ticket, if you're very familiar with the codebase and the problem space, but for most novel problems, you're going to need to have your hands on the problem itself to get your most productive understanding of them.
I'm not saying it's not important to discuss how you intend to approach the solution ahead of time, but I am saying a lot about any non-trivial problem you're solving can only be discovered by attempting to solve it. Put another way: the best code I write is always my second draft at any given ticket.
More micromanaging of your team's tickets and plans is not going to save you from team members who "show little interest in learning". The fact that your team is "YOLOing a bad PR" is the fundamental culture issue, and that's not one you can solve by adding more process.
Asking a more junior developer or someone who "show little interest in learning" to discuss their approach with you before they've spent too much time on the problem, especially if you expect them to take the wrong approach seems like the right way to do things.
Throwing out a PR of someone who doesn't expect it would be quite unpleasant, especially coming from someone more senior.
> PRs are the first time your reviewers have seen that change, so there is no opportunity downstream to do the things you're suggesting.
Yes, but I'm arguing for that it shouldn't be the first time they hear about that this change is planned and happening, and their input should have taken into account before the PR is even opened, either upfront/before or early in development. This eliminates so many of the typical PR reviews/comments.
Figure out where you are going before you start going there, instead of trying to course correct after people already walked places.
I think it's a fine line to walk. At my job what we do is discuss any complex implementation or risky change or blockers in the dev eng meeting. For smaller stuff, or more straightforward solutions, we don't bring it up. If you make it a hard rule to first discuss all tickets, it just seems draconian.
Code review is specifically for code quality, more lower level stuff.
> If you as a reviewer spots things after the implementation rather in the discussion beforehand, that ends up being on both of you, instead of just the implementer who tried to move along to finish the thing
This is accurate, but it's still an important check in the communication loop. It's not all that uncommon for two engineers to discuss a problem, and leave the discussion with completely different mental models of the solution.
I do the same as the author, and for my comments I make it clear what is non-blocking/blocking with Conventional Comments: https://conventionalcomments.org
To make it easier for myself to leave the CC tags ("nit(non-blocking): ", etc), I use the macOS text expander in System Settings and created mnemonics to easily insert them.
Example: If I type "+pnn", that maps to:
+p = pull request comment
n = nit
n = non-blocking
Example 2: If I type "+pc", that maps to:
+p = pull request comment
c = chore
(and it's blocking because I didn't type "+pcn", the non-blocking version).
While one of the best things about PRs can be raising the floor on quality, I think the another powerful factor is forcing an opportunity to talk about the code. This article even pre-empts this confusion by acknowledging that commenting on a PR while approving it is "weird". This implies that many people don't understand the indirect benefits of the PR and think the process is all about approval and raising the quality floor.
I think raising the floor, gives diminishing returns once someone is used to the team and the code base, but the conversation always remains relevant. Sometimes teams that resist things like PRs (e.g. "they just slow us down") are actually teams that are having those conversions elsewhere (in-person, on slack, during standup or sprint planning, etc).
Totally agreed, but it always comes back to managing the average over the best. It is a lot more effective to enforce a culture of approvals than a culture of collaboration. Managers should be striving for the later, but not every team is built the same way and many in leadership just need assurance that the bare minimum is getting done.
> Additionally, some repos can be configured to automatically merge PRs when all requirements are met, one of which might be your approval.
If anyone at GitHub is reading this, I’d love a fourth checkbox in the “leave a review” modal that is “Approve but disable auto merge” (alongside Comment/Approve/Request changes)! Even just surfacing “this PR has auto merge enabled” near the Approve button would be great.
This feels like a suboptimal solution to me because I personally like to keep comments in "unresolved" state so that they remain visible and other folks can weigh in on them if they want, but in a way that doesn't block the PR. Basically I wish that GitHub would either separate the "collapsed" and "resolved" concepts, or add this "approve without merging" button.
This one resonates, because it's largely how I comment on PRs too!
One thing not mentioned is how important it is to acknowledge the comments too. People are taking their time to review your PR, and not even giving a reaction will make the commenter question whether or not it was even received. I'm not looking to throw my thoughts out into the aether. That's what microblogging platforms are for!
I can't tell you how many times I got no response/acknowledgement on a comment that actually surfaced something critical. I haven't been keeping track, but I think my comments could have prevented dozens of outages at this point. It's quite exhausting. In my own experience, the worst offenders of this are senior devs.
> Why approve, if I’ve left comments that I think are worth implementing?
>
> Because I trust my team. I know that my comments will be considered, and if they’re useful, implemented.
I do this a lot too. It's critical that PR authors don't burn that trust either. If they make substantial changes that warrant another review, I hope they do request it. Too many times in my career have colleagues just went ahead and made bad changes after my approval that I would have easily caught, merge, and things go
High trust, high alignment environments move so fast, and you know when you're in one and know when you have your colleagues' trust. It feels really good!
Isn’t a change to the code that incorporates your suggestions enough acknowledgment? Presumably the code change would be required to get your approval?
Of course! I didn't state it explicitly, but what I was referring to were comments that went completely unacknowledged in any way, be it explicit or implicit. No changes. No reaction. No "resolve comment". Radio silence.
As mentioned, I've experienced it too many times where not addressing a question/concern I put on the pull request led to outages that could have been avoided. I think it's typically a certain personality. It's not a common occurrence, but I have experienced it.
I see where the author is coming from, but still this feels like a reinvention of commit trailers ("Acked-by", "Reviewed-by",...) especially for these non-blocking and "only appraising" comments.
Commit trailers even have the benefit that they are recorded in the comment, making them discoverable and even allowing statistics, while "human language" comments are not that easy to do statistics on (modulo AI I guess, but that's another topic).
But PR discussions about lintable style issues always surprise me. The ideal solution is to add a rule in the linter. But when the team won't agree on the rule, and is open to multiple styles, the author should decide, simple! Had a team mate recently who'd block PRs over T[] vs Array<T>, forcing people to stick with Array<T> for simple types like number[] even though TypeScript's own docs and tooling push T[].
Linters often don’t provide the constraints you desire. While they’re a great (arguably essential) tool, they often are not sufficient.
When a project has well-established patterns, part of the job is to just follow the pattern, whether you like it or not, until you find a reason to 1) change the pattern everywhere, or; 2) make an exception to the pattern with intention.
It's quite simple to write a rule, e.g. for the example I shared https://typescript-eslint.io/rules/array-type. But I've never seen any reputable project arguing against T[] for simple types. In my opinion, that says a lot about the reviewer, not the contributor. If a linter rule doesn't exist, you should argue against the idiomatic convention.
Yeah, I think all style comments should be handled by either a linter/formatter or be written in a style guide. Everything else is up to personal preference.
Then, when a style comment comes up in a PR, the answer is "Oh, do you think we should add that to our style guide? If so, let's discuss that in slack. Until then, though, that's not blocking."
This is exactly the pedantry I try to avoid in my teams.
The point of a review shouldn’t be to make sure it’s exactly how the reviewer would do it. Is it safe? Does the author understand the area? Is there evidence the code has been exercised? Does it follow the major conventions?
Beyond that, I try very hard to build a culture of approvals vs nitpicking, and let the linters enforce the rest.
I'm a big believer in this workflow and generally agree with all the guidelines about when to do and not do this.
Code review has value, but I don't think we are always honest about the costs. At most places I've worked, there has been an informal understanding that code should be reviewed within 24 hours or so, but there is a world of difference between getting a review within 2 hours and 23 hours.
If you have to go back and forth a second time, it's so much more likely that the approval comes much later due to straddling end-of-day in someone's timezone.
Tangentially, if you are designing policies for code review at your org, try to think both about minimum requirements (e.g. PRs should get a first look within 24 hours) and typical requirements (e.g. most PRs should get reviewed within 2-3 hours). What typically happens is what determines overall velocity for your org, so it's much more important than whatever strict SLA policy you have. These are also fixable problems if you have targeted conversations with people. E.g. "I noticed X, Y, Z times, you had unreviewed PRs at the end of the day. Can you set aside some time to review code before EOD? Or try installing PR reminder?"
Totally agree, and I go even farther: I consider that the PR author is totally responsible of merging their PR or not, and so no comment is blocking. I can make a comment I consider blocking, but there might be very good reasons that I can't see to still merge the PR. Fixes can come in a follow-up PR. As long as everyone takes responsibility for their work, it is a well-working system.
One thing that helped my team a lot was tagging review comments with prefixes like "nit:", "question:", and "blocker:". Authors stop guessing what's required to merge versus what's a preference, and reviewers feel free to leave the small style notes without holding things up. Also a quick voice or pairing call before reviewing anything contentious saves an absurd amount of round trips.
This is me, I think it comes from a personality of wanting to make sure I say something but generally trusting that the author has more context than I do. I float questions and nits but generally will approve if I don't see anything glaring. If I spot anything where I think "we should definitely go another direction here" I ask for changes instead of approving and make them super clear.
I woke up one day and the industry is gating all code changes behind PRs w/ approvals. Anyone still able to commit to main without approval from someone else? I’m even seeing companies require multiple approvals on every PR. It’s insanity.
I’m skeptical. Software is as buggy as it ever was. I come
across teams shipping terrible quality software, where every line change was approved and reviewed. I come across teams that require every change have an approval, but don’t require 100% test coverage. I’m seeing senior engineers have to get approval from juniors for a copy change.
It’s theatre. It’s bad management. It’s a cargo cult. It isn’t actually driving code quality.
Code review is one thing. Code review is good. But requiring every change have an approval is something else.
1. Does it work? Then ship it. This is great for early on, high-velocity where the goal is to get something working in the wild. AI and AI proponents love this option. It's easy to spot obvious problems, but very unlikely to lead to feedback on structural changes to abstractions and architecture to increase overall _long-term_ velocity.
2. We assume this works, but is it "correct"? This is where long-term code maintainability is created. The quality and effort put into a review like this is obviously far more involved than option 1. People working long term on a code base love this option.
We've been biased towards #1 for a long time, but I feel like we dont have enough people capable of doing #2.
I've worked with some people who only seem to care about 2. as in, they don't try the feature in any way, but come back with comments about "this isn't tested enough" even though it has higher coverage than the codebase's average, and refuse to approve even though they'll never meaningfully review the content. it does seem to be mostly just theater in my experience
exactly, the kind of reviews which have a point are the ones you do before starting work. But that's a design review, not a code review. My team does "commit to master", although I did catch a fair few regressions by looking at the committed code.... but as long as it doesn't go live, who cares?
Just want to comment here. I hate reviewer leave a bunch of nits and stamp the PR. This is ambiguous, are these nits asks or just you opinions? What if I dont address all of them. Also folks need to take rejection lightly - your reviewer wants you to address something, thats really it.
If I've approved the PR, then these are changes I'm asking you to do, but not ordering you to do. You are free to say "no" to my request
> What if I dont address all of them
Then you will have decided that you don't agree with my recommendation and that's OK.
I only ever do this with people I trust - I am trusting you to review each of my nitpicks and make an informed decision if they're necessary or not. Generally I'd like you to reply to the ones you don't do with a reason though.
Azure-dev ops explicitly has an "Approve with suggestions", I suspect it was designed for when there are multiple acceptances required, but I sometimes use it anyway.
Whether those comments get read once approved, I don't know.
This is more than half of my code reviews (when functionally correct). It works pretty well, when you can trust that the person doing the changes won't accidentally (or otherwise) slip in behavioral changes. Like half just got merged as-is and half of those got a separate cleanup later.
Most people are fine, some routinely abuse it to do much larger changes that should normally imply full review - double check afterward! Your name is still on it, it is still your responsibility.
I started doing this a few years ago, it works a treat with non-junior engineers. With the right testing in place, review becomes more about communicating the ideas clearly rather than gatekeeping.
I do this too. I generally skim PRs just to make sure the person isn't doing anything too crazy. I trust my team to write good code. Pulling a branch and testing the code is usually a waste of time unless it's a critical bug or feature.
Things like that I'd much rather leave as comments in the code, rather than dangling as off-hand things in some unrelated PR. Probably no one will see those PR comments unless they go splunking for some reason, but having them right in the code is much more evident to all the people passing through, or even reminding yourself in the future too.
In general I feel like PR reviews are basically reviews happening too late, and if there is a lot of stuff to review and agree upon in the PR, you'll be better off trying to reduce that up front by discussing and noting design decisions before the work even starts. If there is so many unknowns first, do one step to figure out how it can be done, then agree on that, again before starting the actual work.
That leads to PRs basically just being spot-checking, not deep analysis nor even space for nitpicks. If you as a reviewer spots things after the implementation rather in the discussion beforehand, that ends up being on both of you, instead of just the implementer who tried to move along to finish the thing.
I agree in general and tried to push for a more iterate approach in our team. However, my fear is that this would multiply the effort because it's very easy to discuss many things that do not matter or turn out later to not matter.
It seems it's easier for people to only discuss the important stuff when presented as an actual implementation.
We are talking tendencies here, of course, general design decision must always be done beforehand.
LLMs help a lot here, create two prototypes with both designs, compare them together :) Could even evaluate how ergonomic some future potential change is, to see how it holds up in practice.
I used to use pen and paper to essentially do the same, minus having real code and instead just talking concepts, but it does suffer from the issue that some need to be confronted with code in front of them to visualize things better for themselves.
You're essentially suggesting pre-PRs, but it is circular, since those same pre-PRs would have the same criticism.
PRs are about isolated core changes, not feature or system design. They answer how not why.
Walking this road to the end you get pair programming.
So it's helpful to shift left on that and discuss how you intend to approach the solution. Especially for people who are new to the codebase or unfamiliar with the language and, thanks to AI, show little interest in learning.
Obviously not for every situation, but time can be saved by talking something through before YOLOing a bad PR.
I'm not saying it's not important to discuss how you intend to approach the solution ahead of time, but I am saying a lot about any non-trivial problem you're solving can only be discovered by attempting to solve it. Put another way: the best code I write is always my second draft at any given ticket.
More micromanaging of your team's tickets and plans is not going to save you from team members who "show little interest in learning". The fact that your team is "YOLOing a bad PR" is the fundamental culture issue, and that's not one you can solve by adding more process.
Asking a more junior developer or someone who "show little interest in learning" to discuss their approach with you before they've spent too much time on the problem, especially if you expect them to take the wrong approach seems like the right way to do things.
Throwing out a PR of someone who doesn't expect it would be quite unpleasant, especially coming from someone more senior.
Yes, but I'm arguing for that it shouldn't be the first time they hear about that this change is planned and happening, and their input should have taken into account before the PR is even opened, either upfront/before or early in development. This eliminates so many of the typical PR reviews/comments.
Figure out where you are going before you start going there, instead of trying to course correct after people already walked places.
Code review is specifically for code quality, more lower level stuff.
This is accurate, but it's still an important check in the communication loop. It's not all that uncommon for two engineers to discuss a problem, and leave the discussion with completely different mental models of the solution.
To make it easier for myself to leave the CC tags ("nit(non-blocking): ", etc), I use the macOS text expander in System Settings and created mnemonics to easily insert them.
Example: If I type "+pnn", that maps to:
+p = pull request comment n = nit n = non-blocking
Example 2: If I type "+pc", that maps to:
+p = pull request comment c = chore
(and it's blocking because I didn't type "+pcn", the non-blocking version).
I think raising the floor, gives diminishing returns once someone is used to the team and the code base, but the conversation always remains relevant. Sometimes teams that resist things like PRs (e.g. "they just slow us down") are actually teams that are having those conversions elsewhere (in-person, on slack, during standup or sprint planning, etc).
If anyone at GitHub is reading this, I’d love a fourth checkbox in the “leave a review” modal that is “Approve but disable auto merge” (alongside Comment/Approve/Request changes)! Even just surfacing “this PR has auto merge enabled” near the Approve button would be great.
One thing not mentioned is how important it is to acknowledge the comments too. People are taking their time to review your PR, and not even giving a reaction will make the commenter question whether or not it was even received. I'm not looking to throw my thoughts out into the aether. That's what microblogging platforms are for!
I can't tell you how many times I got no response/acknowledgement on a comment that actually surfaced something critical. I haven't been keeping track, but I think my comments could have prevented dozens of outages at this point. It's quite exhausting. In my own experience, the worst offenders of this are senior devs.
> Why approve, if I’ve left comments that I think are worth implementing? > > Because I trust my team. I know that my comments will be considered, and if they’re useful, implemented.
I do this a lot too. It's critical that PR authors don't burn that trust either. If they make substantial changes that warrant another review, I hope they do request it. Too many times in my career have colleagues just went ahead and made bad changes after my approval that I would have easily caught, merge, and things go
High trust, high alignment environments move so fast, and you know when you're in one and know when you have your colleagues' trust. It feels really good!
As mentioned, I've experienced it too many times where not addressing a question/concern I put on the pull request led to outages that could have been avoided. I think it's typically a certain personality. It's not a common occurrence, but I have experienced it.
But PR discussions about lintable style issues always surprise me. The ideal solution is to add a rule in the linter. But when the team won't agree on the rule, and is open to multiple styles, the author should decide, simple! Had a team mate recently who'd block PRs over T[] vs Array<T>, forcing people to stick with Array<T> for simple types like number[] even though TypeScript's own docs and tooling push T[].
When a project has well-established patterns, part of the job is to just follow the pattern, whether you like it or not, until you find a reason to 1) change the pattern everywhere, or; 2) make an exception to the pattern with intention.
Then, when a style comment comes up in a PR, the answer is "Oh, do you think we should add that to our style guide? If so, let's discuss that in slack. Until then, though, that's not blocking."
The point of a review shouldn’t be to make sure it’s exactly how the reviewer would do it. Is it safe? Does the author understand the area? Is there evidence the code has been exercised? Does it follow the major conventions?
Beyond that, I try very hard to build a culture of approvals vs nitpicking, and let the linters enforce the rest.
Notably, prevent github merges if there are unresolved comments, so you know they glanced over them before collapsing it.
Code review has value, but I don't think we are always honest about the costs. At most places I've worked, there has been an informal understanding that code should be reviewed within 24 hours or so, but there is a world of difference between getting a review within 2 hours and 23 hours.
If you have to go back and forth a second time, it's so much more likely that the approval comes much later due to straddling end-of-day in someone's timezone.
Tangentially, if you are designing policies for code review at your org, try to think both about minimum requirements (e.g. PRs should get a first look within 24 hours) and typical requirements (e.g. most PRs should get reviewed within 2-3 hours). What typically happens is what determines overall velocity for your org, so it's much more important than whatever strict SLA policy you have. These are also fixable problems if you have targeted conversations with people. E.g. "I noticed X, Y, Z times, you had unreviewed PRs at the end of the day. Can you set aside some time to review code before EOD? Or try installing PR reminder?"
I’m skeptical. Software is as buggy as it ever was. I come across teams shipping terrible quality software, where every line change was approved and reviewed. I come across teams that require every change have an approval, but don’t require 100% test coverage. I’m seeing senior engineers have to get approval from juniors for a copy change.
It’s theatre. It’s bad management. It’s a cargo cult. It isn’t actually driving code quality.
Code review is one thing. Code review is good. But requiring every change have an approval is something else.
1. Does it work? Then ship it. This is great for early on, high-velocity where the goal is to get something working in the wild. AI and AI proponents love this option. It's easy to spot obvious problems, but very unlikely to lead to feedback on structural changes to abstractions and architecture to increase overall _long-term_ velocity.
2. We assume this works, but is it "correct"? This is where long-term code maintainability is created. The quality and effort put into a review like this is obviously far more involved than option 1. People working long term on a code base love this option.
We've been biased towards #1 for a long time, but I feel like we dont have enough people capable of doing #2.
> are these nits asks or just you opinions?
If I've approved the PR, then these are changes I'm asking you to do, but not ordering you to do. You are free to say "no" to my request
> What if I dont address all of them
Then you will have decided that you don't agree with my recommendation and that's OK.
I only ever do this with people I trust - I am trusting you to review each of my nitpicks and make an informed decision if they're necessary or not. Generally I'd like you to reply to the ones you don't do with a reason though.
The 'auto merge on approval flag' PR authors can flip on GitHub breaks this flow though, as it will just merge as soon as you hit approve.
We also have most of our repos set to block if unresolved comments. I think it’s a flag on branch protection rules
Whether those comments get read once approved, I don't know.
Most people are fine, some routinely abuse it to do much larger changes that should normally imply full review - double check afterward! Your name is still on it, it is still your responsibility.
I have now commented on it.