Can You Really Trust a Tech Test Anymore? (Yes, But It Takes a Bit More Work)

Technical assessments have always been a mixed bag. Some love them. Some loathe them. But with AI now doing a decent job on take-home tests, hiring managers are asking a new question: How do we actually know who wrote this?

We’re seeing it more and more. A flawless coding test... followed by an interview that falls flat. Or a candidate who nails a take-home in 45 minutes (suspiciously fast), but can’t explain their thought process. Tools like GitHub Copilot, ChatGPT, and Replit Ghostwriter are here and they’re not going anywhere. So what now? Do we ditch tech tests entirely? Not quite. But we do need to change how we use them.

Start With Intent, Not Just Tasks

Before you send over a test, ask: What are we really trying to learn here?Are you testing syntax fluency? System design? Problem-solving under pressure? Once you're clear on the outcome, you can design an assessment that’s harder to game, and easier to assess fairly.

Go Beyond the Black Box

If your tech test can be solved entirely offline (and privately), it can also be solved by AI.

Our advice?

• Use live coding sessions (with guardrails, no gotchas).

• Ask candidates to talk through a past project and how they approached certain problems.

• Include a collaboration component, like pairing on a ticket or debugging together.

It's not about catching people out, it’s about seeing how they think, communicate, and approach real-world scenarios.

Don’t Penalise AI: Contextualise It

Here’s a hot take: using AI isn’t cheating. It’s how engineers work now.

If a candidate uses Copilot or ChatGPT to help structure their answer, great. But can they explain what it’s doing? Do they understand why it works? Could they spot if it didn’t?

It’s not just about writing code anymore, it’s about curating, adapting, and owning it.

Bias Creeps in When You’re Guessing

When a hiring manager gets spooked by a “too good” submission, the temptation is to second-guess. But that can lead to unfair assumptions, especially with candidates from non-traditional backgrounds. If you’re unsure about a test result, don’t guess. Dig in. Ask for a code walkthrough. Run a follow-up interview. Keep the signal strong, not the suspicion.

The Pair People Perspective

We get it: tech hiring has never been harder to assess, and AI just made it murkier. But we’ve helped founders, heads of engineering, and talent leads reshape their process to be more human, more hands-on, and more AI-aware.

Here’s what we recommend:

• Use assessments to start conversations, not end them.

• Design tests that reflect your actual work, not contrived puzzles.

• Treat AI as part of the modern stack, not a red flag.

Because the best engineers aren’t avoiding AI: they’re using it wisely. And the best hiring managers know that how someone solves a problem matters just as much as what they submit.

Need help designing a better tech hiring process? We’ve paired with scale-ups across the ecosystem to help them get it right: early. Let’s build it better.