Does the internet provide reliable information?

Bennet Holmes — Wed, 25 Feb 2009 16:29:00 +0000

The EXPLOSION of online research !

At a time when the internet was still a “trendy” tool used by the younger generation, mostly written in C++, accessible mainly in cities due to lack of adequate networks in rural areas, and subject to ADSL connections (still rare 10 years ago)… online research was given a bad label.

The main problem was “non-representativeness”; a “ball-and-chain” that made it inadequate since it did not provide proper sampling.

In other words, for the internet to become a viable research tool, the surveyed cross-section should be more representative of the overall French population. This has not been the case and a disparity still exists even now. Experts respond to this by questioning whether representation is truly necessary.

Upon further reflection…

According to the figures published in the study referenced below, the profile of the French population versus that of internet surfers is as follows: 42% men as opposed to 53% amongst internet surfers; with respect to the CSP+ (upper socio-professionals): 20% men in the French population as opposed to 34.4% amongst internet users. To be even more precise: 18.5% in France as opposed to 22% on the Ile de France.

In spite of a significant internet penetration rate, there are still some very significant disparities in terms of age: 72% for 12-17 year olds and only 40% for 60-69 year olds (even though the growth in the number of internet surfers for the latter population was very strong from 2007 to 2008).

The question one might ask is whether or not marketing research groups systematically demand national representation for their research? Apart from surveys on general topics, political surveys and a few other research fields, the answer is NO.

Research companies often prefer criteria that are more representative for a given population: women, professionals, children, managers, etc.

In other words, if the methods are employed properly (to avoid surveying targets that don’t use the internet very much) and in proper context, the internet appears to be an excellent research tool, mainly because of its speed of implementation and low cost.

Sensitive data: sampling methods

Throughout history, promoters of “probabilistic” sampling methods were always opposed to “pragmatists” (non-probabilistic). They have always believed that research results cannot be correctly interpreted without calculating error margins. For them, only the scientific method can produce reliable results.

On the other hand, pragmatists, who favour quota methods and defend a more “open” approach, such as the one that uses the internet, do not dismiss results obtained using non-probabilistic methods. Today these people are among those who accept online methods consider the results obtained thereby to be sufficiently convincing.

In another, recently-published bulletin that addresses the benefits of using online research for market evaluation, we state that online sampling cannot be probabilistic, for at least one good reason: it is impossible to send thousands of emails to internet surfers since anti-spam laws forbid this in some countries (France allows this, as long as it is for research purposes).

Standard methods of research definitely have the advantage of being probabilistic but, in spite of this, they have quite a few drawbacks, as even the authors of the research cited below will concede. It forces them to admit that true probabilistic sampling does not exist! Surveys are often restricted to large cities, due to cost reasons, and some areas are never sampled (suburbs and other areas), etc.

Proving its worth

To counter these online “problems”, survey and research companies have quickly realised that it was useful to create completely self-contained online Access Panels and some companies have made it their speciality (e.g. Panel on the web). It is also important to diversify the panellist recruitment methods as much as possible by mixing off-line and online as a means of ensuring reliability and quality.

At least two pitfalls still need to be avoided: the results have to be adjusted to compensate for the differences between the sampling structure and that of the targeted population and, above all, to avoid “self-selection” that allows internet surfers to declare themselves as participants in a studies that only collect names, by simply clicking on a pop-up or some other internet media.

Even with these precautions, online studies must be set up perfectly in order to produce reliable results. To ensure this, some organisations perform “counter-studies” in order to validate their online results.

It is due to this “professionalisation” and the use of multiple filters that, today, close to 25% of research is performed “online”.

This reflection and this article were taken from a very interesting document published in September, 2008 by Syntec Etudes Marketing et Opinion /afm: http://www.revue-ram.org/documents/entreprises/terradot.pdf

ActiSKU - Tag - Online Interviews

Does the internet provide reliable information?

The EXPLOSION of online research !