Comparison of Study Techniques (source: Synthec Etudes, AFM):

Benefits of online studies versus in-field studies

Accessibility to numerous and varied targets. Ability to recruit rare profiles with low incidence rates collection speed. Background information and pre-qualification of respondents Sincerity and richness of responses. Sometimes lower cost.

Quality problems with online results

There are problems due to the very nature of online sampling but these will become less significant over time as the Internet penetration rate increases. Non-representation o Non-probabilistic sampling Risk of “professionalisation” of respondents Collection method: Can an Internet surfer represent a non-surfer? Projectability of the results (external validity)

Quality problems with “in-field” results

In-field testing, which is often considered to be more “qualitative” and more costly, is, in fact, rarely probabilistic:

o Cost-wise: in order to minimize travelling expenses, surveys are often limited to large cities o Street surveys are often limited to the busy main streets o Digicodes are making many areas and random streets more and more difficult to access o Some difficult suburbs are never surveyed ( this is also true for very rich areas ) o As well as certain entire regions that are rarely visited by pollsters (the “ the center of France and most rural areas).

Comparison of the quality of in-field and online surveys results

Variations in one particular study, conducted in 2007 with 4,427 volunteers randomly selected from an online XL panel (Kristof De Wulf and Sam Berteloot), were analysed according to the recruiting t methods used (phone, face to face, emails, web-posted forms, ad campaigns on the internet etc.) while the socio-demographic characteristic of the sampled population was filtered out so that the differences were only due to variations in the sourcing method.

In this analysis, the following factors were evaluated:

The participation rate. The time to complete the questionnaire. The consistency of responses. The depth of responses to open questions (number of words). The response style (monotone or varied). The level of knowledge. The consumption profile

The results show that:

There is no difference in the consumption behaviour (food, drink, equipment, leisure, political involvement etc.). There is no difference in the quality of responses (consistency, depth), except in responses obtained by email (fewer responses), and by Internet (higher click rate). There is no difference in motivation to participate in the study

Motivation of volunteers in online panels

One online, XL panel study of 2,084 panellists used random drawings and donations to charitable institutions as a means of motivation (Annelies Verhaghe 2007):

I want to express my opinion: 32%. I want to help the researcher by participating in the study: 27%. I want to have a chance to win a prize: 20%. I enjoy answering questionnaires: 10%. This is a way of supporting charity: 8%. I like learning things: 2%. Other: 1%

Impact of the duplication of panellists in several panels

Two studies in England conducted by Harris Interactive (2006) with 1,983 panellists (of which there were 133 multi-panellists) and 2,071 panellists (of which there were 166 multi-panellists) showed that:

There is no difference in age or sex. There were slightly more unemployed persons amongst the multi-panellists (6.7% as opposed to 5.2%). They respond more but also abandon the survey more often, thus an identical rate. The quality of their responses is slightly higher (consistency, depth). They are not more motivated by incentives. On the other hand they actually show a greater desire to learn new things