This was an interesting quiz, but perhaps not the way the authors intended. The second half was straightforward first-year computer science, but the first half challenged me to speculate about the syntax of a novel scripting language.
Agreed. I've been a professional developer for nearly 20 years. I've shipped kernel C, windows C++, java,python, ruby, clojure to production, but the last time I wrote python was nearly a decade ago. Since then, I've been firmly in the Clojure community, with no intention of leaving.
I know I'm a good programmer, and I got nearly every python-specific question wrong. I understand closures and call-by-reference tyvm, but I don't remember or care about the quirks of python.
Triplebyte in particular seems to not care about the difference between "good at programming" and "good at programming in python". The skillset needed for C, python, clojure and JS are wildly different. They do a disservice to their users by pretending otherwise.
For what it's worth, we run our interviews in the language of the engineer's choosing (including clojure), and get people jobs at Clojure and Haskel and C++ companies. We went with Python for this (toy) quiz because it's the most common language we see among applicants. I tried to add comments to explain anything python specific, and make the question about the underlying logic. But it seems it's still easier for Python folks.
I caught 3 "Whoops sorry you missed a weird bug in a specific programming language that you weren't explicitly being tested on". Sure, if I was doing a Python3 quiz, I'd expect those. But this wasn't what the quiz claimed to be.
Turns out, the "fill in the blank" code was easy, got all of them. Along with all the 1st year theory questions.
I agree. Coming in with a LOT of experience in JS and a good damn chunk in c++, but none in python... this is really bad test.
You ask dev about aspects of the language that are essentially memorization:
Does matrix.append(x) in python append by reference or by value? Both are equally likely without knowing anything about the language.
the question is also pretty easy to answer correct IF I know that, but I have no reason to know that. You're not testing skill in any sense of the word. You're purely testing familiarity with this specific language (and worse, this language's standard library)
> Does matrix.append(x) in python append by reference or by value? Both are equally likely without knowing anything about the language.
I don't think they're equally likely. The question would have the same answer in Javascript, Ruby, Lisp, Perl 6, Java, and many other languages. You can see from the code samples that Python is a high level, dynamic, imperative language, so I think it's fair to assume you would expect that behaviour from it. There's a bit of guessing, but you can make an educated guess.
Way late, but your response sort of proves the point. You're having to argue about what I can infer about the language.
What JS/Ruby/Lisp/Perl/Java do is irrelevant. Hey, C/Go/D/C++ would all be more likely to append by value, but neither of us care a whit because it's asking about Python, which I know nothing about.
So (and this is really hard) try to forget that you've got experience with the language, and approach it as a random pseudo language: this question has no clear answer, because it depends on the memorized implementation details of the standard library of a language...
That's bad.
---
Finally: JS doesn't have matrix as a base type (boolean/number/string/null/undefined/symbol is it, baby), so for this same question, I'd strongly argue that you're simply wrong about bucketing JS with those other languages. Again, for JS the question would revolve around the implementation details of the matrix object and its prototype. Trust me, I can write a useful matrix implementation that appends by value in JS.
> What JS/Ruby/Lisp/Perl/Java do is irrelevant. Hey, C/Go/D/C++ would all be more likely to append by value, but neither of us care a whit because it's asking about Python, which I know nothing about.
I guess my point was that you should be able to infer that Python would be more likely to behave like JS/Ruby/Lisp/Perl/Java than C/Go/D/C++. If you know or can tell by looking at it that it's much more in the first camp then the second, then you should be able to guess. You're probably right that I overstated how easy that is to tell just from the syntax if you don't know anything at all about Python, but it's also pretty short of memorizing implementation details.
> Finally: JS doesn't have matrix as a base type (boolean/number/string/null/undefined/symbol is it, baby), so for this same question, I'd strongly argue that you're simply wrong about bucketing JS with those other languages. Again, for JS the question would revolve around the implementation details of the matrix object and its prototype.
The 'matrix' in the question wasn't a special matrix type, it was just a list of lists. Word-for-word translated into Javascript it would look like:
let matrix = []
let row = [0, 0]
for (let i = 0; i < 2; ++i) matrix.push(row)
matrix[1][1] = 1
Question: If you were given a new block of code below, what is the result of X if X is passed by reference?
[Block of code]
1. 34
2. 17
3. 43
4. 22.5
5. Not listed
____________________
This would be a much more clearer way of handling pass by ref/value. 2 of the answers are correct, so this could be reused to ask the other one. If they're being tricky, they can award half-credit for choosing the wrong pass-by-*. And 3 other answers are junk. And this abstracts out any sort of programming neologims or fetishization of $random_language.
But that LITERALLY is memorizing implementation details.
Here are the rules for JS, which I happen to know, because I use it often:
1. Javascript is always pass by value, but when a variable refers to an object (including arrays), the "value" is a reference to the object.
2. Changing the value of a variable never changes the underlying primitive or object, it just points the variable to a new primitive or object.
3. Changing a property of an object referenced by a variable does change the underlying object.
Those are not trivial "just infer it" details. That's highly language specific, and absolutely unrelated to the skills the test claims to be evaluating.
Agreed - this is a really Python-heavy thing that shouldn't be. I got 13/15 largely through guessing the syntax and semantic weirdness of a language I only rarely touch and never for long (like, the matrix question, by-value or by-reference? I had a 50/50 chance to get it right).
Possibly, the "Is Python call by reference or call by value" question could be rewritten in a language independent form by providing correct syntax Python, the correct python output, and asking verbally "Given either this evidence OR your Python coding experience, is Python call by reference or call by value?"
I've never used Python. I breezed thru the theory and regex questions, presumably I could be a halfway decent Python programmer in a week or two, it looks reasonably non-tricky. Thats another issue, some jobs merely require syntactically correct code, others require good algos, and its a lot easier on the job to learn syntax than to learn algos.
There's certainly nothing objectionable about choosing a common language for a toy quiz like this. It did feel jarring to have the early questions be language-specific, and the UTF question was sort of unknowable, but I got it by guessing what the most likely meaning of the syntax was.
I was frustrated that language features felt trickier than actual coding knowledge in the early questions, but given that it's nothing with actual stakes there's obviously no harm in it.
Those comments definitely helped. But there were a couple questions which came down to "does Python copy this data structure or simply pass a reference", and all I had to go on there was "if I was designing the language, how would I do it?"
In one case, I apparently think that Python got it wrong. :-)
If you wanted to make a generalist quiz, why not ask the questions in psuedocode?
Asking the user to choose between 'queue.extend(node)' and 'queue.extend([node])' is purely a Python syntax question; it has nothing to do with programming per se.
The correct answer is node.children, which isn't syntax-specific, it's about the pseudocode of whether you want to visit the node, or the node's children.
Using pseudocode is a great idea. We've talked about that. We've not done it because we still need to decide what style of syntax to use (and that will still preference certain programmers). But maybe we should think about it again!
Maybe start off with a The Elder Scrolls style background quiz, where people are asked things such as "Ever since childhood you've been having terrible nightmares about (a) curly braces (b) dynamic typing (c) memory management.
And then, after 20 background questions, you could know with high certainty what style of pseudo code (or even language!) a person prefers.
Although it would be a fairly decent extra amount of work ... make the quiz available in the top 20 languages and have the person choose which language to view the quiz in.
I have only ever written maybe 300 lines of python, and I was able to get every python question correct. The questions could easily be translated line by line to Ruby of Javscript and the answers would be the same. Even if you only knew C you should be able to get 5 out of the 9 python questions. If you are familiar with Ruby or Javascript then you should get all of them. The only reasons I can think of for getting all or most of them wrong as a competent programmer are that you are not good at multiple choice tests (the answers give a lot of hints), and/or slightly different syntax ruins your understanding.
Breakdown:
findMax: language agnostic - should be easy for any programmer
BFS: mostly language agnostic - requires knowing about the .extend method, but the meaning of it can be deduced from context (I did)
modify: requires knowing how variables work in python (storing references to objects), this is the same as Javascript, Ruby, Java, and other. If you only used languages with a different model like Haskell or C, missing this is totally excusable
matrix: same as modify, just a little more complex
factory: requires the variable knowledge + knowledge of closures
int_to_base: language agnostic - only requires knowing recursion
merge: language agnostic - just requires thinking through how merging works
decode-utf8: pretty much just a python thing, but it shouldn't be too hard to guess well if you know about utf-8 and notice the name "bytearray"
I think it's reasonable for a person with good experience of multi-paradigm dynamically typed languages to work out the solutions to those questions.
As a person who has strong JS/Lua knowledge, I didn't recognise the language as Python, but managed to get them right. I don't think my past experiences of C++, Java, C# would have been at all useful. I've also had a look at languages like Clojure, Elixir, Swift and Rust in my own time, and again that's not useful for this quiz.
It's a fun quiz, but maybe some of the assertions they make ("Only 3% of programmers can get all the way through") are not very fair, given it's not a true assessment.
Oh good, I wasn't the only one. The second question had me wondering if it was testing my knowledge of tree searches or python queues. I don't know python at all.
Java turns out to be a solid reference point for this. JavaScript might be even better. But there are still issues - not only would this be hard for a C++ programmer, it's hard for a Java programmer who overthinks the thing. Sure, I know how Java handles pass by reference, but unless I know that Python works the same way I'm still guessing.
