About the author

Will Barrett is a Software Engineer, Technical Lead, and Engineering Manager from the San Francisco Bay Area with over 14 years of experience. He’s a Superset PMC Member of The Apache Software Foundation. He’s held staff software engineer and senior engineer roles at Change.org, Entelo, Sqwiggle and Preset.

Will is the author of On Learning to Program, a blog for new Software Engineers entering the industry. Will is also a certified reviewer on PullRequest where he’s caught hundreds of bugs, security issues, and other critical issues for over 40 teams.

Two years ago I onboarded with PullRequest as part of their reviewer network. I was impressed by their approach and the level of quality control in their process. Here is my story.

The Beginning: A Shower Thought

I was thinking one fine day: can you get paid to review code? It seems like a great idea. I love doing code review and I thought it would be neat to act as an external reviewer for other engineering teams. Figuring that any good idea has already been had more than once, I did a quick Google search and came across PullRequest. I filled out their online application - which included a few questions and asked for links to things like my GitHub and LinkedIn profiles - the whole thing took about 15 minutes then I went about my day.

Phase 2: A Conversation

An employee from PullRequest reached out and scheduled a time to talk. Evidently I made it through the profile screening ;). We had a conversation over video chat. I remember describing my experience and answering a series of technical questions and non-technical questions like “why do you want to be a reviewer?” - I relayed my shower thought story and talked about my enjoyment of reading code and providing feedback to other developers. Turns out, a lot of the PullRequest reviewers found out about the same way I did - a Google search to see if it exists.

We also discussed which languages, frameworks and tools I had expertise in so PullRequest would know which teams to match me with and which reviews to route my way. The questions they asked made it clear that they’re pretty selective as to who they advance, but the whole thing felt very casual. It felt like having a good conversation with another software engineer, not like a scripted screening call with a recruiter. We even had a laugh or two. Shortly after, within about a day, I got word that I was approved to advance - I was in!

Phase 3: Paperwork

Depending on the location and reviewer’s personal circumstance, PullRequest pays its reviewers as either contractors or part-time/seasonal employees. There’s no schedule or minimum time commitment.

There’s of course paperwork that needs to be completed, but I was surprised by how easy it was. Their HR platform will have one form for taxes and another for configuring bank account details for direct deposits. Since most of the code review done on PullRequest comes from private repositories for companies, it came as no surprise that they require a signed confidentiality agreement and needed to run a criminal background check. Like the online application, the administrative stuff all took about 15 minutes.

Phase 3: Set up a Login

PullRequest has its own special reviewer portal. After I had completed all of the on-boarding steps, my active review account was enabled and I was able to log in to the reviewer portal. All I needed to do at that point was configure two-factor authentication.

The PullRequest reviewer portal has a number of videos and helpful guides explaining how things work and the expectations PullRequest has for engineers in the reviewer network.

Phase 4: The PullRequest Reviewer Slack Workspace

There’s a private Slack Workspace for PullRequest that reviewers join. In the Slack Workspace, there are channels specific to various languages and types of code reviews. In addition to the reviewer portal, this is also a good place to find high-priority pull requests to review. PullRequest has a Slack bot that will also ping you when a developer replies to you or pushes a fix to address an issue you brought up as well as notifications when you earn and when payments are issued each week.

The Slack Workspace is also the heart of the PullRequest reviewer community. It’s full of incredibly talented and interesting software engineers and a great place to learn and network.

Start Reviewing

PullRequest is a unique service that connects development teams with experts from their reviewer network, so when you first start it can feel like reviewing a pull request on your first day of a new job. You have to navigate carefully things like context, what their development style is, what’s optional vs. a blocker, what’s out of scope but worth mentioning, etc.

This can be jarring, but the PullRequest team helps you through it.

My first few reviews I did on PullRequest were actually approved by PullRequest before it was posted to the customer’s pull request. If a review needs to be revised or modified, a member of the PullRequest team will reach out and work with you on it. This is helpful even for things like choice of words (e.g., writing “I have some security concerns related to this policy…” instead of “This policy you created is bad and insecure”).

After the first few reviews for a client, my comments started getting posted instantly. Occasionally they still approve reviews I submit, especially for clients that I haven’t reviewed much for or it’s been a while.

It didn’t take long for me to notice that reviewing on PullRequest was making me a better reviewer for the other teams I work on.

Getting Paid

Instantly upon my review being sent to a client I get a direct Slack message with the amount I was paid for the review. I also see a counter in the reviewer dashboard with the amount that I’ve made on that day or week from the reviews I’ve completed. Once a week, a direct deposit is issued to my bank account. Every Tuesday, these funds are available to spend!

The PullRequest Team

Every now and again something odd will happen - some exceptional circumstance will come up where I need some help from PullRequest to resolve the issue. The full-time staff at PullRequest are always super helpful and responsive when issues arise. I remember once I was getting an error in the reviewer dashboard and Dan, the company COO, reached out to me a couple minutes later (before I reported the problem) letting me know they were working on deploying a fix. Yup, they’re that on-top of things.

Find this useful? Be sure to check out these additional articles by Will Barrett:

• What We Can Learn from the Ruby on Rails Project about Code Review
• 5 Ways I Provide Value as a PullRequest Reviewer When I Start Reviewing a New Project
• How to Catch the Top OWASP 2021 Security Category - Broken Access Control - in Code Review Part 1 and Part 2
• Tips for Migrating to a New Computer for Programmers
• The Top 5 Most Common Security Issues I Discover When Reviewing Code
• Reviewing One’s Own Code
• Handling Code Reviews with Empathy
• What We Can Learn About Code Review From the Apache Superset Project
• What We Can Learn About Code Review From the Apache Spark Project