Take-home assignments are not free work
9 January 2022 ‧ 5 min read
Many companies require candidates for product-related positions, such as product managers, product designers, UX researchers, analysts, content strategists, or data scientists to complete a written take-home assignment.
I am planning to write a longer article about the pros and cons of homework assignments for product managers specifically, but I wanted to already address one point more generally that is often brought forward as an argument against take-home assignments: that they amount to unpaid work (if they are in any way related to the company's actual product).
I will preface this by admitting my biases: I have hired using take-home assignments before, and I've had to complete assignments for each of the three companies that I've joined in a product role (and a few others that I interviewed for but didn't end up joining). I will also admit that this article is limited to product-related roles (because I have too little experience hiring in other functions) – it's entirely possible, for example, that if you interview a bunch of graphic designer candidates and ask them each to design you a new logo that you could just use one of the results without compensation.
For product-related roles, however, the “free work” argument couldn't be further removed from reality in my experience. Whatever a candidate produces as a result of their assignment will not be usable in any shape or form by the company because it will necessarily be lacking in a number of respects:
- Time
- Context
- Feedback
- Complexity
Lack of time
Let's start with the most obvious point first: unless it's a truly horribly designed assignment, candidates should not spend more than a few hours on their homework – and that includes time to understand the assignment and do research. That is only a fraction of the time that any given person at the company has. Because of this limited time investment, the depth of thought and quality of execution of the candidate response will necessarily be limited and not address a large set of concerns and edge cases.
Lack of context
Regardless of how well the assignment is set up in terms of providing additional qualitative and quantitative data, a candidate will never have the same depth of understanding of the relevant context as people working on the product full time.
Perhaps the most important aspect here is the intimate knowledge of customers and their problems, ideally built up over frequent and regular customer interactions. Great product people will correctly intuit some of these customer problems, but their ideas in a short homework assignment will never have the same richness and diligence as the work product of an experienced team member.
Lack of feedback
Building upon the previous point, any initial idea will have to be validated and iterated upon, for example by testing prototypes with customers. Very often, the final versions of ideas that come out of this process look very different to what went in (or the idea gets discarded altogether).
Some of the best assignment responses I've seen have included some way of validating the candidate's ideas through guerrilla research, but that's of course not the same as actual customer research. It mostly shows that the candidate is aware of the need to validate and solicit feedback, and scrappy enough to collect some data themselves.
The same holds for feedback from the rest of the team. Product development is a team sport, and different team members have different perspectives on the risks involved in product development. A beautifully designed solution, for example, is useless if it's not technically feasible (or very costly to implement).
Lack of complexity
The last point that somehow flows from all the other points is that the assignment necessarily strips the problem of most of its complexity. This means that even at best, all that a candidates write-up can contain is the spark of an idea or insight, and all the hard work still remains. Ideas are dime a dozen, the hard work is fleshing them out, validating them, and realizing them.
Moreover, because of the lack of context that the candidate will inevitably have, it's extremely unlikely that the candidate will come up with an idea or insight that the team hasn't already had in the past.
Let me illustrate this with an example. When hiring product managers and product designers at fitness and nutrition app 8fit, we would ask them for ideas to improve 8fit for people with injuries (e.g. helping to recover from an injury or modifying workouts in a way to not put strain on an injury). Out of many candidates' responses, there wasn't a single one that contained a good idea that was so unique that we hadn't discussed it before. In fact, the best candidates were the ones that prioritized ideas that we had already thought about as high potential.
Even in cases where the candidate brings unique skills to the table and thereby is able to produce ideas that the team hasn't had, that doesn't mean that it can just be stolen as “free work”.
Let me give another example: when hiring the first data scientist at RevenueCat, we asked candidates to come up with an answer for “what is the customer retention” (given an example dataset similar to our production database). In this case, there was no data scientist on the team yet who would have had a “textbook” answer to that question, so candidates had more of a chance to tell us something in their response that we didn't already know. The poorest responses included answers that didn't make sense. The adequate ones had answers that I (as a data literate product person with domain knowledge) agreed with and could have put together myself (albeit probably more slowly). The best responses all told me something new about survival analysis and different estimators. However, even in these cases, there was no way that we could have just taken these responses and done anything remotely useful with them—bringing these toy examples to the level of being able to be implemented in production is where most of the complexity lies.
What if the candidate truly has a novel idea or insight?
I've never seen it happen that a candidate comes up with an idea that is so novel, unique and mind-blowing that it has the chance to truly alter the product's trajectory. However, for the sake of the argument, let's assume that a candidate does manage to produce such an idea. Would the company then dump the candidate, take the idea for free, and run with it? Of course not! They'd immediately hire the candidate! Firstly, because of the chance that there's more where that idea came from, but also to see the idea through to implementation. As mentioned, most of the complexity comes after the initial idea, and who would be better positioned to flesh out the idea than the person who came up with it?
In summary, worries about being used as a supplier of “free work” when asked to complete a take-home assignment for a product-related role could not be more misplaced. The chances of producing something that is even remotely usable by the company are extremely slim, and in the unlikely case that you do produce something novel and uniquely useful, the company would just hire you as fast as they can.
I hope you found this article useful. If you did, feel free to follow me on Twitter where I share thoughts and articles on product management and leadership.