The article “Technical interview performance is kind of arbitrary. Here’s the data.” looks at technical interviews and how they are not a good indicator when looking for job candidates. The article uses a variety of unprofessional language and a large number of graphs to visualize the data, to help form the conclusion.
In the article, the author uses data from interviewing.io, a platform where people can practice technical interviews anonymously. The author visualizes this data with a variety of graphs displaying the mean technical performance of interviewees from 1 to 4. Furthermore, the author uses the data to find the probability of a candidate with a given mean score would fail an interview. Lastly, the author makes two final conclusions. The first is that technical interviewing itself is “fucked” and doesn’t provide a reliable idea from one interview. On the other hand, the conclusion is that aggregative performance can help correct for poor performance and that an interviewer should interview one more time before making a final decision.
The article does a study which looks at 299 interviews and 67 interviewees. While 299 interviews seem like a good amount, with only 67 interviewees between them that is not a large amount per interviewee. In addition, with no mention in the article of the average amount of interviews per interviewee, we have no idea if some interviewees can skew the results with some getting only a score of 2s or 4s. Another largely overlooked problem with the analysis is that the statistics used are not independent as the past interviews can influence future interviews. This can occur as people may get better over time at taking interviews or the interviewers may get a better grasp at what they are looking for over time. This failed approach may also just favour those who are good interviews and not necessarily their skills. This overlooked problem may impact the overall mean score for interviewees and interviewers. What the analysis should do is look at how these scores change over different time periods and not just at the overall picture, which may lead to large differences.
Another point to look at is some of the wording the author chooses to use in writing the article. When looking at the performance from interview to interview, the article describes the graph below, stating “every represents the mean technical score for an individual interviewee who has done 2 or more interviews on the platform”, which a missing mention of what every represents, most likely assuming the every visual on the graph. Along with just general forgotten words, the use of swear words in the article doesn’t help the study seem very professional in trying to make conclusions on professional technical interviews.
In the end, the way the article is written with frequent swears thrown around makes the study hard to take as a professional analysis. Furthermore, with the analysis failing to take into account different time periods taken, the results used will lead to false conclusions. Therefore, to improve the analysis, the swearing should be removed with a larger pool of candidates and better stating of information about candidates used in the study.