I grew up with opinion polls on television. Every evening, after the news — always the same ritual: a bar chart, a round number, and a grave voice explaining what "the people" thought. Nobody asked where that figure came from. Neither did I.
Now I ask. And the more I ask, the more I find that the numbers in surveys resemble reality less and less, and look increasingly like a tool — sometimes for informing, sometimes for manipulating, and most often for brutally simplifying a complicated world.
I am not writing this as a statistician — I am not one. I am writing as someone who has learnt, over time, to ask a few questions before believing a figure. And I have the distinct impression that these questions — simple enough, when you think about it — are absent from far too many articles, debates, and Facebook posts.
A Headline That Should Give You Pause
"Survey: Romanians have lost faith in institutions."
Or: "60% of employees are dissatisfied with their boss."
Or, in the more polished corporate variant: "Study: 9 out of 10 customers recommend our product."
All of these headlines share one thing: they present a figure as though it were a universal truth, whilst saying nothing about how that figure was produced. And that is precisely where the whole story lives.
An honest headline would look quite different. It would read something like: "INSCOP survey, 1,067 respondents, telephone interviews, October 2025: 60% of urban adults report having no confidence in Parliament." That is information. Everything else is background noise with pretensions to revelation.
The Questions You Should Ask of Any Survey
It took me years to understand that the figures themselves are not the most important thing in a survey. What matters is the context in which they were produced. And that context has several clear components — ones you can verify every time you read an article based on survey data.
1. Who conducted the survey?
This is not a rhetorical question. The polling organisation matters enormously. INSCOP Research and CURS have established reputations in Romania. A "study" produced by a company's in-house marketing department carries an entirely different epistemological weight.
There is a fundamental difference between an independent research firm — one whose methodology is published and open to scrutiny — and a "survey" assembled on Google Forms and shared on the brand's Facebook page. Both can produce figures. Only one produces data.
2. Who paid for it?
This is the uncomfortable question that the Romanian press most consistently avoids. The commissioning party of a survey is not a technical footnote — it is essential information for understanding any potential conflict of interest.
A satisfaction survey paid for by the very company being evaluated has, structurally, an incentive to produce positive results. That does not mean those results are false — it means they must be read with additional caution. An electoral poll commissioned by a political party? Add a significant layer of scepticism before you share it.
3. When was it conducted?
A survey carried out in August 2024, presented as "relevant" in March 2026, is at best an old photograph of a reality that has since changed. Surveys are, by their very nature, snapshots. They are not eternal truths.
This is perhaps one of the most consistently ignored limitations in the Romanian press: old data recirculated as though it described the present.
4. Who was asked, and how many?
The sample is the heart of any survey. How many people were asked? How were they chosen? Who exactly are the "Romanians" in the headline?
A sample of 200 people, selected by convenience — meaning whoever happened to respond — cannot represent 19 million people, however authoritative the headline sounds. A sample of 1,000 to 1,500 people, randomly stratified by region, age, gender, and place of residence, has an entirely different representational power.
The standard margin of error for a sample of 1,000 people is ±3%. That means a difference of 31% versus 32% is statistically meaningless. They are, for all practical purposes, the same number. How many articles in the Romanian press explain this? Very few.
5. How were they asked?
The wording of a question can radically alter the answer. This is not theory — it is documented psychology.
Ask "Are you satisfied with the service you received?" and you will obtain one percentage. Ask "Which aspects of the service could be improved?" and you will get something else entirely — not because people lied in response to the first question, but because different wording activates different cognitive frameworks.
Leading questions — "Do you agree that this tax is abusive?" — are a classic mechanism for generating the desired figures. Neutral questions — "What is your view of the new tax?" — produce far more balanced distributions of opinion.
If the article you are reading does not include the precise wording of the survey questions, you have every right to be sceptical of the interpretations on offer.
6. How was the information collected?
CATI (telephone), CAWI (online), face-to-face — each method introduces its own distortions.
Telephone surveys tend to over-represent older people who still use a landline. Online surveys over-represent those with internet access and a habit of filling in digital forms — in Romania, this implicitly means an urban and educational bias. Face-to-face surveys are more expensive, but also less susceptible to certain kinds of systematic error.
There is no perfect method. There are methods suited to specific contexts — and articles that ought to explain what the researcher chose and why.
Surveys in Business: The Same Logic, Different Stakes
Everything described above applies equally to the corporate world — perhaps even more acutely, because the financial stakes are more direct.
When a company sends its customers a satisfaction survey and obtains an NPS (Net Promoter Score) of 8 out of 10, what does that actually mean? It depends on how many customers responded, how many declined to respond and why, how the question was worded, and precisely when the survey was sent.
A survey dispatched immediately after a positive interaction will produce higher scores than one sent a week later. A survey sent only to active customers omits the very voices of those who have left — the people most important for understanding where the company has failed.
The standard tools used by companies — NPS, CSAT (Customer Satisfaction Score), CES (Customer Effort Score) — are useful when applied correctly and interpreted honestly. When they are deployed to generate attractive figures for board presentations, they become decorative.
I have seen enough "satisfaction studies" from Romanian businesses to recognise the pattern: a small sample, questions framed so that a positive answer was entirely predictable, no awkward open-ended questions, response rate undisclosed. Result: "94% of our customers are satisfied." And then the same company wonders why it keeps losing clients.
The Misreadings That Both Journalists and the Rest of Us Make
A few recurring patterns, which I encounter constantly — in the press and in everyday conversation alike:
Hyperbole. "Everyone believes X" when the data shows 51%. A statistical majority is not unanimity. There is always a significant minority that thinks otherwise.
Ignoring the margin of error. A difference of one or two percentage points in electoral polls is not a "victory" or a "remarkable recovery." It is statistical noise. But news headlines treat it as revelation.
Confusing correlation with causation. A survey showing that happier employees are more productive does not prove that happiness causes productivity. It may well be that productive companies create conditions in which employees are happier. The direction of causality matters enormously, and it cannot be established by a survey.
Treating a survey as a prediction. "The poll shows X will win the election." No. The poll shows that, at the moment the data was gathered, X was preferred by a certain proportion of respondents. Voting intentions change. Some people never make it to the ballot box. Others respond differently from how they vote — there is even a term for this: the Bradley Effect, the tendency of some respondents to give the "socially acceptable" answer rather than the honest one.
What a Good Article About a Survey Actually Does
This is, in the end, the standard against which any text based on survey data should be measured. I am not inventing it — I am drawing it from serious journalistic practice and plain epistemic common sense.
A rigorous article about a survey specifies:
- Who conducted the research and whether the organisation has a verifiable reputation
- Who commissioned and paid for the study
- When the data was collected
- How many people participated and how they were selected
- How the data was gathered (telephone, online, face-to-face)
- What the margin of error is and what it means in practical terms
- How the questions were worded — ideally, with exact quotation
- What the response rate was — how many people declined to participate
If you are reading an article in which half of these elements are absent, you are looking at either superficial journalism or a press release dressed up as analysis.
Why Any of This Matters
I am not writing this out of a passion for statistics. I am writing because I have seen enough decisions — personal, commercial, political — made on the basis of figures presented with authority and stripped of context.
I have seen entrepreneurs invest in entirely the wrong directions because a "market study" with an opaque methodology showed them what they wanted to see. I have seen public policies justified with surveys funded by the very people who stood to benefit from those policies directly. I have seen ordinary people convinced that "everyone thinks X" because a news headline reproduced a press release without a moment's scrutiny.
Surveys are not bad by nature. They are instruments — and like any instrument, they can be used well or badly, honestly or with deliberate intent to manipulate.
The difference between a reader who swallows the figure and one who asks the first two questions — who conducted it? and who paid for it? — is enormous. You do not need to become a statistician. You only need to hold on to your curiosity.
A Quick Checklist (One I Apply to Myself as Well)
The next time you read an article based on a survey, run quickly through this list:
- [ ] Is it stated who conducted the research?
- [ ] Is it specified when the data was collected?
- [ ] Is it mentioned how many people participated?
- [ ] Is the margin of error given?
- [ ] Is the collection method identified?
- [ ] Is it known who paid for the study?
- [ ] Are the precise questions — at least in part — reproduced?
- [ ] Does the article avoid generalisations of the type "Romanians believe that…"?
If you tick fewer than half, read with reservations. If you tick nothing at all, you are looking at a headline — not a study.
Conclusion: Listening with Discernment
There is a difference between hearing and listening. And there is an equally important difference between reading a figure and understanding what it represents.
Good surveys — conducted with transparent methodology, presented honestly, with their limitations openly acknowledged — are among the most valuable tools we have for understanding how the people around us think and feel. Companies that use them properly make better decisions. Citizens who read them critically are harder to manipulate.
I do not think we all need to become experts in quantitative methodology. I think we need to preserve the simple habit of asking: where does this figure come from? That is enough as a starting point.
Ten Questions with Answers
On surveys, numbers, and the habit of asking
1. Why should a headline such as "60% of Romanians believe that…" give us pause before anything else?
Because that percentage says nothing about how it was produced. A figure without methodological context is, at best, an honest approximation of a partial reality — and, at worst, an instrument of persuasion disguised as objective data. The headline compresses a complicated world into a round number, and we, accustomed as we are to recognising the authority of figures, forget to ask where it came from. Before believing a figure, ask who produced it and under what conditions.
2. What are the two minimum questions any reader should ask when confronted with a survey?
Who conducted it? and Who paid for it? They are simple questions — almost embarrassingly so — and that is precisely why they are most often omitted. An independent research firm with a published methodology produces something entirely different from the marketing department of the company being evaluated. A survey paid for by the party that stands to benefit from its results is not necessarily false, but it carries a structural conflict of interest that you need to be aware of before reading it.
3. What does the margin of error mean, and why do we ignore it so readily?
The margin of error for a sample of 1,000 people is ±3%. That means a difference of one or two percentage points between two candidates, two options, or two attitudes is statistically insignificant — they are, for practical purposes, the same number. The press, however, treats a 1% difference as a "remarkable recovery" or a "clear victory." We ignore it easily because our minds prefer clean narratives to statistical noise — and because headlines that read "statistically insignificant difference" do not generate clicks.
4. How can the wording of a question radically alter the answer, without anyone having lied?
By activating different cognitive frameworks. "Are you satisfied with the service?" and "What could be improved?" do not measure the same thing, even though both appear to be asking about satisfaction. The first invites a global, binary evaluation. The second opens a space for specific criticism. People are not lying in response to either — they are simply answering the question they actually hear, not the researcher's abstract intention. If you do not know how the question was worded, you do not really know what you have just read.
5. What is the Bradley Effect, and why does it complicate the interpretation of political polls?
The Bradley Effect describes the tendency of some respondents to give the "socially acceptable" answer rather than the honest one. In an electoral context, this means that some people will declare support for the candidate they believe the pollster or society at large approves of — not the one they actually intend to vote for. The result is that polls may systematically overestimate "politically correct" candidates and underestimate those considered controversial. A survey shows you what people say they think — not necessarily what they think.
6. Why can corporate satisfaction surveys produce misleading figures, even without any fraudulent intent?
For several structural reasons: they are often sent immediately after a positive interaction, when the emotional impression is freshest; they target only active customers, omitting precisely those who have left; they have low and undisclosed response rates; and they rarely include awkward open-ended questions. Each of these methodological choices naturally tilts the results towards the positive — not through falsification, but through design. When a company wonders why it is losing customers despite a satisfaction score of 94%, the answer is usually to be found in the methodology it chose.
7. What is the difference between saying a survey reveals a preference and saying it predicts an outcome?
It is an enormous difference, and one that is routinely collapsed in headlines. A survey photographs a state of intention at a single moment in time — it does not predict future behaviour. Voting intentions change. Many people never reach the polling station. Others respond differently from how they vote. To transform an instantaneous snapshot into a prediction is to add a level of interpretation that the data cannot support. The survey says: "At date X, Y% of respondents declared a preference for candidate Z." Everything beyond that is editorial speculation.
8. Which method of data collection — telephone, online, or face-to-face — is the most reliable?
None, in absolute terms — each introduces its own distortions. Telephone surveys over-represent older people. Online surveys over-represent urban populations and those with ready access to technology — in Romania, this means a significant educational and geographical bias. Face-to-face surveys are more expensive and less susceptible to certain errors, but they are not immune either. The question is not "which is perfect?" but "which is appropriate for what you want to measure?" — and the article ought to explain that choice.
9. How do we distinguish a serious journalistic article about a survey from a press release in disguise?
By the presence or absence of methodological context. A rigorous article specifies: who conducted the research, who paid for it, when the data was collected, how many people participated and how they were selected, what the margin of error is, how the questions were worded, and what the response rate was. If half of these elements are absent, you are in the presence of superficial journalism. If you cannot tick anything on the list, you are reading a headline — not a study.
10. Why does any of this matter beyond academic methodological concern?
Because real decisions — commercial, political, personal — are made on the basis of figures presented with authority and stripped of context. Entrepreneurs invest in entirely the wrong directions because a "market study" with an obscure methodology showed them what they wanted to see. Public policies are justified with surveys funded by those who benefit directly from them. Ordinary people are persuaded that "everyone thinks X" because a news headline reproduced a press release without the least scrutiny. Critical reading is not a luxury intellectual exercise — it is a form of basic cognitive hygiene, in the absence of which you become vulnerable to manipulation without ever realising it is happening.
Author's Note: Limitations, Tensions, and Rhetorical Devices
There is a discomfort I feel every time I finish a text of this kind, and I cannot dismiss it without being dishonest to whoever is reading.
I write about critical reading of data. I write about scepticism towards figures presented without context. And yet my own text contains not a single independently verifiable statistic, not one case study with an open source, not one testable methodological comparison. I am preaching a discipline that my own text does not fully practise — because this is a journalistic essay, not an academic paper, and journalism has different obligations from research. But this tension deserves to be named, not concealed.
The first limitation is that of generalisation. When I say "the Romanian press avoids asking who paid," I am doing precisely what my critique of surveys identifies as a vice: I am generalising from examples I have accumulated over time, without a systematic sample, without a methodology for analysing the press. I have seen enough to form a solid impression — but my solid impression is not the same thing as a media content study. It is worth knowing that.
The second limitation lies in the rhetorical structure of the text itself. I am using a technique I recognise as effective and, at the same time, slightly dishonest: I pose the question, then answer it myself. The simulated critical-reading questionnaire I propose is, at bottom, a persuasive device — it creates the illusion of dialogue and of the reader's autonomy, whilst steering towards a predetermined conclusion. It is not necessarily a manipulative technique — it is simply rhetoric. But it is worth naming it as such.
The third tension is more personal. I write from the position of someone who grew up with polls on television and who now asks questions. There is something self-congratulatory in this narrative movement — I have grown, I have understood, and now I am here to help you do the same. It is a form of authority built on autobiography rather than credentials. It works rhetorically precisely because it sounds sincere. And it sounds sincere because, in large measure, it is. But sincerity is not a guarantee of accuracy.
The principal rhetorical device I use is the reductio ad absurdum negative example: "a survey on Google Forms shared on the brand's Facebook page." It is an example constructed to appear ridiculous beside INSCOP — and it works precisely through contrast. It is not a logical argument; it is an effect of illumination through juxtaposition. It persuades not through demonstration, but through image.
And yet, for all these limitations, I remain convinced that the text does something useful: it does not teach you statistics, it does not give you research tools, but it gives you permission to ask. And sometimes, permission is the first step.
The rest you do yourself.
References and Sources
- Sheldon R. Gawiser & G. Evans Witt — A Journalist's Guide to Public Opinion Polls, 1994 (foundational principles of survey methodology and journalistic responsibility)
- Pew Research Center — Methodology (public section, pewresearch.org) — reference standard for methodological transparency
- INSCOP Research Romania — public reports available at inscop.ro — examples of good methodological disclosure practice in the Romanian context
- American Association for Public Opinion Research (AAPOR) — Code of Professional Ethics and Practices (international standards for opinion research)
- Eurobarometer — methodology published in full, a model of transparency for large-scale surveys
- Daniel Kahneman — Thinking, Fast and Slow, 2011 (chapters on question wording and framing effects in the perception of statistics)
- Darrell Huff — How to Lie with Statistics, 1954 (reprinted; a classic of critical thinking about data)
- Critical Thinking in Media — public resources from ActiveWatch and CJI (the Centre for Independent Journalism, Romania)