After 20 years building over 200 systems and interviewing hundreds of engineers, I need to say something that's been brewing for the last 12 months: we're still interviewing like it's 2015, but we're hiring for 2025. The gap between what we test and what engineers actually do has never been wider, and it's costing us great talent.
What Changed in the Last 12 Months
A junior dev on my team debugged a critical accuracy issue in hours that would have taken days five years ago - the kind of problem requiring careful simulation and verification. They succeeded because they had better tools and knew how to approach problems systematically.
The old model of assigning stories doesn't scale anymore. Developers turn around tasks so quickly that ticket assignment has become the bottleneck. They're taking problems, implementing one solution, using AI tools to transform it into alternative approaches, benchmarking each option, and choosing the best path - all in the time it used to take to build just one approach.
The cost of prototyping has dropped to nearly zero. Developers implement a solution, ask their coding tool to refactor it using different patterns, benchmark the alternatives, and pick what works best. The skill we need now is exploring solution spaces and making informed choices, not just executing specifications efficiently.
The Honest Truth About Modern Engineering Work
About 70-90% of our time is spent reading code - both in our repo and AI-generated code. We're validating whether AI followed our coding best practices or deviated from them, checking if it wrote overly long functions, duplicated code that already exists elsewhere in our application, or repeated the same logic without abstracting it into reusable methods. This is a new skill: critically evaluating machine-generated code requires understanding intent, spotting logical errors, and knowing when "it runs" doesn't mean "it's correct" or maintainable.
The remaining 10-30% goes to writing new code, integrating APIs, adding features, refactoring, and making architectural trade-offs. Exactly 0% is spent solving algorithm puzzles or implementing data structures from scratch. Yet that's still what most interviews test for.
What AI Changed (Whether We Admit It Or Not)
In the last 12 months, LLM-assisted tools have fundamentally changed what "knowing how to code" means. Need Python's asyncio syntax? Cursor autocompletes it. Forgot Terraform module structure? Claude generates it in seconds. But the real shift isn't about syntax - it's about exploration.
Developers now prototype a solution with Postgres, ask their AI tool to refactor it using Redis, generate a third approach with in-memory caching, benchmark all three, and choose the best before lunch. The constraint isn't implementation time anymore - it's judgment about which approach will be maintainable, scalable, and reliable six months from now.
The engineers who succeed aren't the ones who memorized the most. They're the ones who know what questions to ask, how to validate AI-generated code for quality and correctness, and how to explore solution spaces efficiently. AI has commoditized implementation and elevated judgment.
What Actually Separates Good Engineers Now
After two decades, here's what matters. System thinking and problem decomposition - can you break ambiguous problems into solvable pieces, understand where systems fail, and reason about how data flows through services? Can you explain why performance degrades at scale? This can't be automated.
Reading and validating code is now the core skill. Engineers who can open unfamiliar codebases and understand structure quickly, spot bugs that three people missed, review AI-generated code for best practices violations, catch duplicated logic and missed abstractions, and identify when code works but isn't maintainable - these engineers are worth 10x someone who can implement quicksort from memory.
Debugging ability has become critical. Production breaks in unexpected ways - APIs return bad data, edge cases surface, AI generates code that passes tests but fails in production. I need engineers who trace through logs systematically, reproduce failures reliably, and fix issues without creating new ones.
Engineering judgment and cross-platform thinking separate good from great. Anyone can ask Claude to generate Kafka configuration. Not everyone can decide if you actually need Kafka or if Postgres is simpler, understand that relational databases share core concepts even when syntax differs, or reason about consistency versus availability trade-offs. Understanding concepts across platforms beats memorizing tool-specific commands.
Reliability-focused engineering means designing with retries, exponential backoff, dead-letter queues, and checkpointing from the start. Great engineers think about idempotent operations, durable pipelines, and what happens when - not if - things fail. They own outcomes, not just implementations.
What We Should Stop Doing
Whiteboard coding without IDE, docs, or AI tools needs to end. Nobody writes production code without autocomplete anymore. Algorithm puzzles unrelated to your stack tell you nothing about exploring solutions or debugging microservices. Trivia questions about framework internals miss the point - engineers look this up instantly in real work. Speed-based coding under pressure selects for rushed code, the opposite of thoughtful solution exploration.
What We Should Start Doing Instead
After 18 months of iteration, here's what works. Code reading and validation: Give candidates a small service with intentional bugs and AI-generated code. Watch how they navigate unfamiliar code, spot best practice violations, identify duplicated logic, and explain their reasoning. This 45-minute exercise reveals how they'll work with modern tools.
Real-world problem exploration: Give them a problem and ask them to explore multiple solutions with their IDE and AI tools. "Here's a feature. Explore different approaches, use AI to generate alternatives, benchmark them, and explain your recommendation with trade-offs." In 60-90 minutes, you see how they decompose problems, evaluate options, and make architectural decisions.
Practical system design: Skip "design Twitter." Give them something concrete - design a webhook handler processing 50,000 events/hour reliably. Discuss failure modes, retries, dead-letter queues, scaling, and observability. Ask how they'd prototype different approaches. This mirrors actual staff engineering work.
Code review exercise: Have candidates review both human and AI-generated code with problematic patterns. Can they spot when AI duplicated code, wrote overly long functions, or missed abstractions? This reveals their ability to maintain code quality and communicate feedback effectively.
Behavioral rounds: Focus on real experiences - how they explored solutions for ambiguous problems, debugged production incidents, and learned from architectural decisions that didn't work out. Real stories reveal real capabilities.
The Engineers We're Missing
I've seen brilliant engineers fail our interviews, then build systems processing billions of events daily, debug impossible production issues, and explore innovative solutions we never considered. We rejected them not because they couldn't do the job, but because they couldn't perform in artificial test environments.
Meanwhile, I've seen people ace algorithm rounds but struggle to validate AI-generated code, explore solutions independently, or think about reliability until things break. Whiteboard performance and job performance have zero correlation.
What This Means For You
If you're hiring: Your interview process is probably filtering for 2015 skills. Engineers who can explore solutions, prototype rapidly, and validate AI-generated code might be failing your whiteboard rounds. Test for judgment and modern tool usage, not memorization.
If you're interviewing: Find companies that let you use your IDE, AI tools, and explore multiple approaches. That's a green flag. Whiteboard coding and API memorization tests signal they're stuck in the past.
If you're a new engineer: Focus on reading and validating code (including AI-generated), exploring solution spaces efficiently, and debugging systematically. The engineers winning in 2025 use modern tools to explore and validate solutions, not memorize LeetCode patterns.
The Bottom Line
After 200 systems and 20 years, here's what I know: Engineering isn't about memorization or coding speed. It's about exploring problems, validating what AI generates, and building systems that don't fall over. Our interviews should reflect that reality.
The next generation of senior engineers won't stand out because they memorized algorithms or performed well under artificial pressure. They'll stand out because they can take ambiguous problems, explore solution paths, validate AI-generated code, and make thoughtful architectural trade-offs. That's what real engineering looks like now.
If we expect engineers to solve modern problems with modern tools, then our interviews need to evolve and measure the skills that truly matter.