Today I caught up with a former colleague from my first job out of college. It got me thinking about how I got into marketing research in the first place, and what I was doing as an entry-level data analyst over 20 years ago.
If you started in research after data books disappeared, a data book was the printed set of crosstabs from a survey. Every question, cut by every banner. Hundreds of pages. A big part of my job as an entry-level analyst was checking them.
It was detailed, careful work. It turned out to be the best training I've ever had.
Checking a data book meant comparing every table to the questionnaire and the tab specs, line by line. Bases had to match the skip logic. Percentages had to add up. Nets had to include the right codes. Significance letters had to make sense.
Doing that over and over taught me things no training deck could. I learned how studies are built. I saw how a question becomes a table, a table becomes a finding, and a finding becomes an actionable insight. I started recognizing patterns in where errors are hiding and what specifically to look for, like a filter on the wrong question, a base that is too small, or a multiple-response question based on the number of responses instead of the number of respondents. I also developed a healthy skepticism and a gut feeling for results that look too clean to be true.
And catching a reversed scale was the best feeling. When I did, I felt like a hero who saved my client from going down the wrong path based on false assumptions.
By the time I was writing reports, I knew where every number came from. That is what let me trust my own judgment and defend it in front of a client.
Today, data books are long gone. Tables live in platforms, and quality checks are automated. In the last few years, AI has taken over much more of the entry-level work. It drafts questionnaires, codes open-ended responses, writes toplines, and summarizes decks.
That is super helpful, no doubt. I use AI-assisted workflows in my own work, and they save real time. But the work AI now does is the same work that used to train junior researchers. When a tool hands you a clean summary, you never see the bad filter or the thin base underneath it.
The risk is not that AI gets things wrong. It sometimes does. So do people. The risk is a generation of researchers who will never learn to tell the difference.
Research is not the deliverable. The deliverable is evidence strong enough to back an important decision.
That means a researcher's value is judgment, not production. Knowing which question to ask. Knowing which sample, and how large, can answer it. Knowing when a result is solid enough to act on, and when it is not. And being willing to say so to a senior stakeholder who wanted a different answer.
AI makes production faster and more streamlined. That makes judgment more valuable, not less. There is more output to sort through, and more of it looks convincing. A great-looking chart built on a weak sample is more dangerous than a messy one. Why? Because nobody questions what looks to be perfect.
So, if you are early in your career, or if you manage people who are early in their career, how do you build critical thinking skills in the absence of a data book? Before learning how to use a calculator, kids learn how to count. Before learning how to build Excel models with complicated formulas, students build an understanding of what these formulas mean. In the same way, our education and at-work training should emphasize understanding the processes behind the black box called AI. How do you do that while still getting the value AI tools bring?
- You check what it does and understand how the results were produced.
- You read the questionnaire before fielding it.
- You read the raw open-ended responses, not just the AI’s summary of them.
And, if you lead a research team, build checking back into junior roles. Treat it as an investment in training, not overhead. Ask people to explain where their insights came from. And always review the reasoning behind the work, not just the output. After all, isn’t this what critical thinking is all about, in the research world and beyond?
Yes, the tools we work with have evolved. But the need to understand the logic and meaning behind what you are recommending has not.