Why though? AI is already part of how a lot of engineers work, and people are already using it to game traditional coding interviews. so trying to enforce “pretend AI doesn’t exist” feels a bit like giving a kid a calculator every day, then deciding the exam should test whether they can hide from the calculator.

i’d rather accept that the tool exists and test the part that still matters: can they reason, verify, catch bad output, understand the codebase, and make a safe change?

the AI shouldn’t be the thing being tested. the engineer’s judgment should be.

you absolutely can, and honestly you probably should be able to.

my point is more that if the interview explicitly allows AI, then using it well becomes part of the skill being tested too. not “let AI do it for me,” but knowing when it helps, when it’s wrong, and when it’s faster to just do the thing yourself.

yeah, obviously 😭

i’m not saying AI replaces knowing how to code. i’m more interested in what comes after that baseline can you actually work through an unfamiliar repo, use AI without blindly trusting it, and make a safe change?

yeah, the platform exists.

this post is more me trying to validate the idea and hear what people would actually want from something like this before i over-engineer it.

i can share the link if you want to try it.

this comment actually sent me down a rabbit hole lol

the “score the trace, not just the diff” part is the bit i keep coming back to. the final fix is easy to measure, but the interesting signal is probably what they inspected, what they tried, whether they verified the AI suggestion, etc.

i’ve been building something around this since that thread and that’s still the part i haven’t figured out properly - Groundwork but i’m curious, how much of the investigation do you capture before it starts becoming fake/surveillance-y?