1 Introduction

Over the past quarter century, the relationship of survey researchers to survey respondents has undergone a significant shift. With the proliferation of online access panels (Callegaro et al. 2014), the majority of survey respondents today receive material incentives—money or gift cards—as a direct reward for completing a survey. This process of commodification potentially represents a profound change from past survey practice.

According to classic sociological theory, when a good becomes commodified—that is, when it is exchanged for money as a standardized good—the consequences for social relations are widespread, inevitable and mostly negative (Healy 2006:5). But more recent scholarship in economic sociology argues that even when money changes hands in a transaction, participants are often able to maintain desirable qualities in their relationship by doing “relational work” that defines and emphasizes the social or emotional aspects of the exchange (Bandelj 2020). This article explicates the potential tension between commodification and relational work in survey access panels, arising from the fact that nearly all panel participants are paid for completing questionnaires. Increasingly, monetary incentives are also used in one-time cross-sectional surveys, but the present study focuses on access panels because their offer of repeated earning opportunities magnifies the potential for negative effects.

There is a considerable and growing recent literature about the management of online access panels, focusing on such issues as panel recruitment, attrition, panel conditioning, and minimizing fake or insincere responses (e.g. Callegaro et al. 2014, 2015; CIPHER 2025). As is the case with the field of survey methodology in general, the primary theoretical underpinnings of this stream of research come from social and cognitive psychology, as was pointed out by the late Don Dillman toward the end of his career (2020). This article seeks to place the complexities of panel management in a larger context, illuminated by concepts drawn from the discipline of sociology, which has much to offer this field (Brenner 2020). The theoretical background for this analysis incorporates the sensitizing concepts of commodification and relational work, drawn from the growing field of economic sociology, and—following sociologist Amitai Etzioni’s (1961) theory of complex organizations—considers survey panels as compliance structures. The relationship of survey researcher to survey respondent is here considered from the viewpoint of symbolic interactionism (Blumer 1969), recognizing that words, images, gestures and actions actively create shared meanings that define and maintain the nature of their relationship. More specifically, this article examines the relational work that online access panels perform, showing wide variation in how they characterize incentives for their participants. The data come from in-depth interviews with panel managers, analysis of communications across a diverse sample of active panels, and monitoring of social media used by certain panel participants. Both the qualitative and quantitative data will show that different types of panels differ substantially in the relational work they undertake.

The correlational results of this study are congruent with the thesis that effective relational work is a key feature of panels that produce higher quality survey data. However, proof of causation (i.e., effective relational work causes respondents to behave better) lies outside the scope of the current analysis. The current study is well able to measure the independent variable in that proposition (relational work). But direct measurement of the dependent variable (panel outcomes) is not possible because most panels treat as strictly proprietary their internal data on respondent attitudes and behaviors, panel attrition or the quality of survey data that their respondents produce. Absent such panel-specific data, the evidence presented here can only be suggestive of the role that relational work plays in keeping panel participants engaged and responding as panel managers expect.

2 Panel structures

Before further explication of the theoretical background for this study, it will be helpful to review briefly how survey access panels vary in their structure. Like the rest of the survey industry, the access panel sector is diversified and stratified with respect to the quality and price of survey products, as well as firms’ reputation. One differentiator between panels is their method of recruitment. Probability panels recruit by probability-based sampling methods exclusively, originally via random telephone and in-person sampling and more recently through mailings to scientifically drawn, stratified samples. Most afford a means for invitees lacking Internet access to participate in studies. People are not permitted to join these panels as volunteers.

In contrast, non-probability or “opt-in” panels allow any qualified adult to join, securing sign-ups from advertisements on the internet and other channels such as endorsements by social media influencers. There is substantial research literature showing that probability panels yield more accurate results than non-probability panels (Malhotra and Krosnick 2007; Yeager et al. 2011; Kennedy et al. 2016; Dutwin and Buskirk 2017; Dutwin 2026). Probability panels charge their clients considerably more per interview than non-probability panels do—sometimes many times more. Government agencies and academic researchers prefer probability panels, but most market research surveys today are conducted on the internet using non-probability surveys. Some panels recruit through both probability and non-probability methods.

An important distinction among non-probability panels is whether they allow direct contact between the panelist and the researcher. Amazon Mechanical Turk (AMT) represents one end of this distinction. This service was advertised from its start in 2005 as a “crowdsourcing marketplace” that gives individuals and businesses access to a “global, on-demand, 24 × 7 workforce.” “Requesters” post tasks on the platform, and participant workers (who refer to themselves as “Turkers”) sign up to complete the tasks. Requesters specify qualifications for a survey or other task and how much they will pay per completion. Requesters can approve or reject a Turker’s work, and Turkers can rate requesters, yielding public rating scores for those on both sides of the transaction—the new currency of trust in the internet age (Brooks 2014). Other crowdsourcing platforms like Prolific and CloudReseach are similarly structured but use various controls to offer researchers access to vetted and better-performing respondents. What these platforms have in common is that they offer researchers virtually direct access to potential respondents, much like gig-economy platforms in other industries. If market platforms are “permissive potentates,” as argued by Vallas and Schor (2020), these are the most permissive platforms in the survey world, allowing the researcher to define the task, the qualifications and the price to be paid to the worker.

In contrast, all probability-based panels and many non-probability panels mediate actively between research clients and their pool of participants, vetting survey scripts, setting reward amounts, and blocking excessively burdensome or poorly designed surveys. They allow no direct interaction of the research client and the panel participant. Participants are invited to complete surveys that often appear to come from the panel company, without mention of the research client.Footnote 1

3 Theoretical background

3.1 The changing survey bargain

At the heart of the survey process is a transaction: the respondent provides information, in the form of their responses to a questionnaire or interview script, in exchange for some mix of rewards provided by the researcher, or—more commonly–by a professional survey firm or organization acting on behalf of the research client. Taken collectively, these exchanges have traditionally taken place within an informally constituted survey response market, in which survey organizations from the several sectors of the survey research industry are continuously approaching individual members of the public, selected with various sampling methods, to solicit their engagement in a successful survey transaction that yields an acceptably completed interview or questionnaire.

In contrast to focus group participants and patients in randomized clinical trials, who have generally been paid for their participation, until the rise of online access panels survey respondents have not usually been offered financial incentives contingent on survey completion. Footnote 2 Instead, since the time of George Gallup, survey organizations and their interviewers have traditionally offered only symbolic and normative benefits to sample members, asking them for a one-time response to a specific study’s questionnaire.

The classic survey bargain was framed with these incentives:

  • You are special, chosen at random to be part of a scientific sample.

  • We’ll keep your answers confidential.

  • You get a chance to have your voice heard; we will listen intently and record your responses.

  • We won’t judge you for your opinions, behaviors, level of knowledge or your life circumstances.

  • Your responses will be part of a consequential report.

  • You’ll find the questions to be interesting, perhaps informative, too.

  • We’ll be grateful, respectful and appreciative of your honest answers.

The respondent was expected to fulfill their part of the survey bargain by agreeing to participate, being attentive, answering questions honestly, being reasonably deliberative in formulating answers, and completing the whole interview or questionnaire.

A quarter century ago, the second edition of Dillman’s classic book on the “Tailored Design Method” (Dillman 2000) advocated small, non-contingent advance cash incentives (typically $2 in cash) as a means of building trust and obligation. These soon became the norm for one-shot mail-out surveys. These were presented as “tokens of appreciation,” as part of a social exchange and not as compensation or pay as would be expected in an economic exchange. Guterbock et al. (2016) showed that the small advance incentives do more than create obligation; they also alter respondents’ perceptions of the survey’s legitimacy and its burden in favorable ways. Thus, the survey researcher or survey organization could invoke the symbolic power of the gift (Schwartz 1967) without altering the fundamental survey bargain.

Over the last quarter century, survey proliferation has increased the demand for survey responses and, as evidenced in drastically declining response rates, has apparently overwhelmed the supply of willing strangers (Leeper 2019). High demand and low supply have had the expected economic consequence: the price of survey completion has gone up, and increasingly survey organizations offer contingent rewards, not only in access panels but even for one-time cross-sectional surveys.

Online access panels proliferated starting around 2000 and by 2023 constituted a $2.25 billion global industry (ESOMAR 2024:14). The transition to using access panels brought significant challenges to the classic survey transaction. As a panel member, the respondent is being asked to do not just one, but many surveys, while communication is less personal over the internet. Trust is essential to the cooperation of the respondent (Dillman et al. 2014). But when all communication is computer mediated, it is harder to establish trust (Wilson et al. 2006) between survey organization and respondent. As a result, most survey panelists are paid modest amounts for each survey they complete, with some receiving cash for just signing up. And panelists are asked to do more in return: in addition to the traditional expectations of attentiveness and deliberation, panel participants are expected to identify themselves authentically, give truthful answers to the intake survey that profiles their personal characteristics, not falsely claim to be qualified for any particular survey, not attempt multiple completions of the same questionnaire, and maintain a reasonable rate of participation in the access panel’s surveys over time.

3.2 The expanded cash market for survey responses

Before homing in on the specifics of the relational work that panels undertake, we should note some key aspects of today’s survey response market, identified in the author’s in-depth interviews with panel managers. These indicate the extent to which the survey response has become commodified.

  • There is wide variation in the amount of compensation that survey panels provide, ranging from less than a dollar for a short screener survey on some panels to over $100 per hour for patients with rare medical conditions. Probability-based panels and panels with specialized participant pools tend to provide higher incentives. In addition, these panels incur higher recruitment costs per participant, which are passed back to research clients in the form of higher fees.

  • In direct exchange panels, the participant is typically offered a dashboard of available surveys. The panel member can see the estimated length of each task, the amount of award being offered, and sometimes other details. Here the buyer of response (the researcher) and seller of response (the respondent) are directly engaged in a monetized marketplace. But the same market dynamics operate below the surface in mediated panels, where there is no dashboard for the respondent but instead a succession of offered surveys, again with each one stating its length and offered reward.

  • Wage differentiation based on respondent characteristics has become widespread. Since it is more difficult to reach, retain and gain cooperation from some respondents, their value to the researcher is increased. Result: many organizations offer higher rewards for specific surveys to respondents from harder-to-reach groups, and some offer members of those groups larger sign-up bonuses during the initial panel recruitment process. For example, one probability panel classifies participants and recruits into three levels of response propensity, with the monetary rewards for those in the lowest-propensity group being four times the rewards for those that are easiest to reach.

  • There is a huge secondary market for respondents among non-probability panel providers. If the front-end firm falls short on a quota, they purchase responses from one or more other panels. According to the panel managers interviewed, pricing and specification of the purchased responses are typically negotiated separately for each study, with higher rates for harder-to-find respondents. The researcher who purchases survey results from a non-probability panel may receive responses collected from many different panels, and the researcher often cannot tell which panel provided a given response (Enns and Rothschild 2022). Respondents often remain unaware that their response has been sold to another firm.

  • Firms like Cint (formerly Lucid) and Qualtrics operate purely as brokers linking researchers to multiple panels to secure their samples. As with other commodities, basic quality standards may be in place, but within any given demographic quota cell (e.g., Hispanic males 18–35) the respondents are an undifferentiated commodity. Thus, the survey response is not merely commodified but also is becoming commoditized (Wikipedia 2023).

  • The market exerts downward pressure on the unit price for responses, a natural product of competition between panel vendors for business from researchers.

  • At the same time, there is some upward pressure on respondent incentives, a product of competition among vendors for willing panel participants. According to a recent AAPOR report, traditional panel membership has been in decline for at least a decade (McPhee et al. 2022:11). Some panels attract sign-ups by advertising that they pay participants more than other panels do.

The development of the world-wide cash marketplace for survey responses is just another example of how the rise of the Internet has disrupted and expanded the market for so many goods. It is not without its benefits. It has greatly expanded the number of people who participate in research, reduced the cost of population-based survey data and improved researchers’ access to rare populations. Moreover, one might argue that it was exploitative to ask people to do for-profit surveys without receiving any payment, so that the switch to “paying for answers” is simply a move toward justice. With that said, there are some potential negative consequences that need to be considered.

3.3 The negative view of commodification

Classic sociological theory holds that the process of commodification has profound effects, changing the orientations and actions of buyer and seller (Simmel [1900] 1978) in undesirable ways. Healy (2006:5) succinctly summarizes the effects anticipated by Karl Marx in Das Kapital (1990:920,926):

. .[B]y allowing market logic to dictate the terms of an exchange, we risk losing something important. Markets reduce different ways of valuing things to one dimension, measured in money. They encourage us to treat people as means rather than ends. They erode our desire to carry out principled actions unmotivated by profit. They undermine altruistic action—whether heroic or mundane—by rendering it unintelligible. (Why do something for free when you could be paid for it?) They break up social relations, networks, or forms of organization and replace them with ersatz equivalents devoid of depth and meaning. It is also often argued that commodification can have negative effects both on the goods being traded and the people who exchange them. Goods can be debased by money. . In the same way, people can be corrupted by money, as they come to act only out of self-interest or for profit, and treat others as means to their own ends.

According to this classic line of reasoning, we should expect to find that commodification affects not only the behavior and attitudes of survey respondents, but those of researchers and panel managers, too. Moreover, the quality of survey samples, survey responses and overall survey results may be negatively affected as well.

There is plenty of evidence that the widespread move to paying for answers has indeed led to negative effects. It is hard to imagine the following phenomena occurring in the absence of monetary rewards:

  • The incentives for satisficing (Krosnick 1996, 1999) by respondents are higher when participants are paid per completion. As a result, attention-check questions and checks for speeding and straight-lining have become mandatory in online panel surveys.

  • A new phenomenon is the “professional respondent,” a person who treats their work completing surveys as a significant source of income. They constitute a significant sub-class of workers in today’s gig economy, described by Vallas and Schor as “microtaskers” (2020:276). We will take a closer look at these respondents below.

  • There has been an increasing incidence of insincere or bogus respondents. Kennedy et al. (2021) estimate that four to seven percent of panel participants are bogus (including insincere humans and automated “bots”), and they find the problem to be particularly acute in opt-in (non-probability) internet panels. In the past year or two, respondents have found new ways of fabricating responses using AI tools such as ChatGPT (Singh et al. 2025; Martherus et al. 2025). In response, major panel firms have developed increasingly sophisticated countermeasures to detect bogus responses, aided by new products from specialized vendors.

  • Criminal enterprises (“click farms”) have emerged that exploit cheap labor outside the United States to collect respondent rewards by completing surveys wholesale, as bogus respondents. (See https://www.cint.com/blog/what-is-a-click-farm/; Jaffe et al. 2026.)

  • Commodification affects researchers, too. Researchers on direct exchange survey sites such as Amazon Mechanical Turk often offer respondents levels of compensation that are far below a reasonable wage. (In online discussions, AMT panelists in 2023 generally considered $8–$10 per hour to be reasonable.) Panelists complain on their chat platforms about these researchers as well as various scams posing as research surveys.

  • Some panel operators try to maximize the number of surveys each participant is asked to do, using programs known as “routers.” This practice can lead to rapid burn-out and excessive churn in panel membership.

Each of these developments tends to lower the quality of collected survey data. Thus, as expected by classic sociology, unbridled commodification can degrade the quality of the goods themselves.

3.4 A more positive view

In a broad challenge to this classical view of commodification, the new sociology of money, as pioneered by Zelizer (1997), has made us attentive to the ways in which people and institutions are often able to maintain the desired quality of social relationships even when monetary exchange is involved. As highlighted by Bandelj (2020), people involved in economic exchange often engage in relational work that allows social relationships to maintain their desired form. Relational work is especially important when people find themselves dealing with socially awkward relations (Bandelj 2020:260). As already noted, paying for answers can and does lead to undesirable behavior by some respondents (and by some researchers) in some contexts. Yet in other contexts relational work seems to be largely successful in maintaining the essentials of the traditional, normatively based social exchange despite the use of contingent material incentives. The structural context of the survey exchange plays an important, parallel role. As already noted, survey panels differ in how they structure or manage the relationship of respondents to researchers. Successful relational work is facilitated by panel structures that allow panel providers to mediate more actively between researcher and respondent. Moreover, the sponsorship and purpose of the surveys that a panel offers to its members can also make non-monetary rewards more or less salient (and plausible) as incentives to participate.

Recognizing the positive effects of relational work is not to deny that it is potentially problematic to use monetary incentives along with the normative inducements used in the classic survey bargain. Viewed from the viewpoint of Etzioni’s (1961) theory of complex organizations, a sample survey constitutes a “compliance structure,” in which higher-level participants (researchers and survey managers) use various inducements and sanctions to cause lower-level participants (the respondents) to perform tasks in ways that meet organizational needs. (An access panel is a more permanent compliance structure than a one-shot survey.) The compliance structure concept is useful here for its generality, as it encompasses employer/employee structuresFootnote 3, voluntary groups, cults of adherents as well as more coercive institutions. In a survey panel, the introduction of monetary payments to panelists introduces a new means of control—remunerative power—alongside the more traditional normative power embedded in the non-monetary inducements used in the classic survey bargain. According to Etzioni, organizations that rely on normative power are able to generate strong commitment from lower-level participants, while those that use remunerative power generate only “calculative commitment” (1961:10).

As will be seen below, many survey panels continue to highlight non-monetary incentives as well as monetary rewards in their recruitment messages. This means that many survey panels operate as “dual compliance structures,” using two types of inducements. But Etzioni sees dual compliance structures as inherently unstable, claiming that when two types of power are used, they tend to neutralize each other. In his words (echoing Healy, cited above): “The application of remunerative powers makes appeal to ‘idealistic’ (pure normative) motives less fruitful” (1961:7). This does not necessarily imply that survey panels are doomed, but rather indicates that, to meet their goals, panel managers must do additional symbolic work to keep the dual compliance structure working. That is, the dual compliance structure can be successfully maintained if relational work is adequate. One alternative, pursued by more than a few opt-in panels, is to focus exclusively or primarily on the opportunity for panelists to earn money, thus bypassing the relational work that a successful dual compliance structure requires. (A less common alternative are panels that rely primarily on non-monetary rewards and do not pay participants per completion.)

3.5 Relational work and panel structures

In all of these panels, nearly all communication between panels and respondents is written or visual (not oral) and most is electronically mediated through web pages or emails. Moreover, communication between panels and participants is mostly one way (panel to participant). Accordingly, the primary vehicle for a panel operator’s relational work is the written and visual communications that flow from the panel to recruits and panelists.

In addition to the relational work embedded in their recruitment communications, panel operators shape their relationships with panelists by how they set the actual conditions of work. Some panels take steps to maintain a positive user experience for their panelists. These actions are found in some—but not all—of the mediated panels. They can positively shape relationships with their panelists by limiting the number of surveys a panelist is asked to do (for example, just two or three per month)Footnote 4; offering monetary rewards well above the minimum used in the industry; offering opportunities for participants to give feedback on each survey; and sharing survey results through newsletters and social media.

Relational work is also evident in the way many panels manage and label the monetary rewards. Many of the mediated panels set up the material compensation they offer as “points” that panelists can convert to rewards, which are often described as “gifts.” This is a fairly clear case of obfuscation as a form of relational work (Bandelj 2020:256) and at the same time represents creation of alternative currencies, as Zelizer (1997) has described. Defining the compensation as a gift helps to maintain a relationship of social rather than economic exchange and makes it less awkward for the normatively-oriented respondent to receive the monetary reward. Another feature that helps participants see their labors as not being done for personal gain is that many panels allow participants to donate their rewards to well-known charities.

Such obfuscation is unnecessary in a purely utilitarian compliance structure, where workers simply get paid money for their work. In fact, most of the direct exchange panels (known to many participants as “beermoney sites”) simply pay cash. One might then expect to see all of the more normatively-oriented compliance structures rewarding their panelists with points and “gifts.” However, using a point system runs afoul of the panel manager’s interest in enhancing the respondent experience. It can be fairly frustrating for the panel member to complete survey after survey and not achieve the threshold for getting a gift. As a result, several mediated panels that are highly focused on creating a positive respondent experience make a point of rewarding panelists with immediate payment by check or funds transfer.

4 Methods and sample

The data reported here come from a mixed-methods approach. The author joined 12 non-probability panels as a survey-taking participant, allowing for direct observation of the language these platforms use to recruit and retain their participants. This also gave the author a taste of what panel participants experience in their on-line labor. The author monitored, over several months, communications among “Turkers” (people who take AMT surveys) via their Reddit and Slack channels, as well as the Reddit channel for members of Prolific, a U.K.-based opt-in, direct exchange survey panel.

For a systematic analysis of how different types of panels vary in how they present incentives, the author assembled a mixed-probability sample of currently active panels. This started with a purposive selection of 13 well-known panels and included most of the U.S.-based probability panels, several mediated panels and a few direct exchange panels. The author conducted in-depth, confidential interviews with panel managers for nine of these, one in Europe and eight in the United States. These included informants from five probability-based panels, one mixed-probability panel and three non-probability (opt-in) panels. Some of the probability-panel informants shared confidential copies of their recruitment materials. Public websites of these firms, which can be quoted freely, often communicate with similar language.

The sample was then expanded by randomly selecting 23 opt-in panels using a published list of 109 opt-in panels as the sampling frame. This frame listed all panels that actively shared respondents with two unidentified opt-in panels included in the study of insincere respondents by Kennedy et al. (2021; Supplementary material, p. 6). Of the 23 panels sampled from that frame, eight were no longer active panel providers. The combined sample thus included 28 active panels (listed in Appendix Table 1 ), of which 4 are probability-based panels, 14 are mediated opt-in panels, 1 includes both probability and mediated opt-in samples, 7 are direct exchange panels and 2 carry out some surveys in mediated mode and others in direct-exchange mode. For purposes of analysis, the panels were grouped into three categories: 5 that are probability based or mixed probability, 14 that are mediated opt-in panels, and 9 that operate either entirely or partially as direct-exchange survey platforms (referred to as crowdsourced panels by Kennedy et al. 2021]).

To test how website content and relational work differ by panel structure, the author coded each panel’s website (or print materials) on several relevant characteristics. For the opt-in panels, the author was able to examine each panel website’s recruitment homepage. To identify unclear panel types, the author signed up as a panel member to observe directly whether each was mediated or allowed direct exchange.

Probability panels do not have public recruitment webpages. However, the author was able to see recruitment materials for all the sampled probability panels because either the materials were shared by interviewed panel managers, the panel had a website for persuading invited prospects to join, or the panel published its recruitment messages in an information package for Institutional Review Board use.

These different methods were deployed to fulfill two research aims: (1) to show how widely access panels differ in their communications with panel recruits and participants, emphasizing variation in the relational work they perform, and (2) to demonstrate that variation in relational work is correlated with differences in panel structure. The first of these aims is primarily achieved through reporting of qualitative results, which are summarized separately for probability panels, mediated opt-in panels, and direct exchange panels. The second aim is then addressed by the quantitative results of the research.

5 Qualitative results

5.1 Probability panels

A good example of the non-monetary appeals used by probability panels can be found on the public website for NORC’s AmeriSpeak panel [www.AmeriSpeak.org]. Since panel recruitment is by invitation only, the appeals on this site are aimed at convincing invitees to actually sign up for the panel. The headline on the site does not mention non-monetary rewards initially, outlining the steps for sign-up as: “Click ‘Register Now’, Create an account, Complete your first survey, Receive Ameripoints worth $20 or more.” However, under the “About AmeriSpeak” tab, the recruit learns that:

AmeriSpeak is a panel of adults scientifically selected to be the voice of their communities. . Surveys are about issues that affect your everyday life: current events and trends in society, new ideas for products and policies, business and finance, health care, and personal technology.

A short video on the site makes the same point with graphic visuals and a persuasive voice-over.

Recruits to SSRS Opinion Panel [SSRSopinionpanel.com], a probability panel with commercial, government and academic clients, are told:

Welcome! As a member of the SSRS Opinion Panel, your opinion will be heard by business leaders, decision makers, news outlets and citizens across the country. . We ask about current events and things that impact your life. About once a month, we will ask you to do a survey on an important topic. Survey topics change each month and could include work, family, politics, religion, technology and more. We make every effort to keep the surveys brief.

The site includes a testimonial from a panel member:

I have enjoyed immensely reading and responding to SSRS surveys. The insightful questions help me to organize my thoughts on many important and urgent questions of our time. I believe my relationship with SSRS has helped to make me a better and more engaged citizen.

SSRS’s messaging thus includes emphasis on the societal impact of the survey results and the intrinsic benefits of participating, while also mentioning the ways that the panel ensures a pleasant participant experience (short surveys, only about once a month.) Of course, the site also mentions that “SSRS Opinion Panel members earn rewards for taking surveys.”

5.2 Mediated opt-in panels

Some opt-in panels also give prominent emphasis to non-monetary rewards. YouGov is a large, international, non-probability survey panel. Their website for signing up [https://account.yougov.com/us-en/join/main] uses very succinct messaging about both monetary and non-material incentives. The headline is:

Share your opinion. Earn money. Shape the news. Join millions of others in sharing your opinion on politics, sport, entertainment and more.

On their main webpage (today.yougov.com), YouGov calls itself “The best survey app ever” and goes on to say:

At the heart of YouGov is a global online community of 24 m people sharing their opinions daily. Our members say our surveys are interesting, our app is easy to use, they appreciate the rewards and love seeing their opinions reported around the world.

Opinion Outpost is a large, mediated opt-in panel run by Dynata. Unlike YouGov, the Opinion Outpost website shows a headline that makes no mention of non-monetary rewards:

Get paid for your opinions. Earn rocking rewards by taking surveys in your spare time.

But the site goes on to describe the work as interesting and influential:

  • At Opinion Outpost, we have created a thriving community where anyone can sign up to our online survey platform and get rewards for their opinions. We work with partners all around the world who want to know your opinions, and they know how much they’re worth!

  • We offer rewards for online surveys across all sorts of different subjects and sectors, which gives you the chance to influence big decision-makers in many areas of our daily lives.

  • . . Surveys are conducted because businesses, governments, public bodies, and similar organisations are interested in the views and attitudes of the people who use their products and services. The more they know about what customers and citizens think, the easier it is for them to improve and adapt what they supply.

The website also emphasizes the flexibility of the “gifts and rewards,” which are recorded as points (10 points = $1). Referring to the panelists as a “thriving community” is another aspect of this panel’s relational work. However, on the webpage that actually collects sign-up information, the headline again focuses on the monetary rewards: “Take Surveys. Get Paid. Earn Cash or Gift Cards. Widest Range!” The wording here gives increased salience to the monetary rewards and makes clear that the gift cards are, indeed, a form of pay.

Ipsos iSay is the mediated opt-in panel run by Ipsos. Their recruitment page (ipsosisay.com) gives considerable play to the influence panelists may have:

“Influence the future. Discover the difference you will make. Want to influence the future and direction of societies, citizens and brands? Ipsos iSay is the rewards community for you. Join now!. . Make an impact: Ipsos iSay offers many survey opportunities across a variety of topics. Your feedback can answer many different questions and influence many different decisions.

These messages are echoed in the graphics shown on the iSay’s website [see Fig. 1].

Fig. 1
Fig. 1
Full size image

Recruitment message from Ipsos iSay

The graphic incorporates both intrinsic and non-monetary incentives (give thoughtful input [light bulb], amplify your voice [megaphone]) and the material rewards. It is noteworthy that the compensation is here represented as a gift.

Nearly all mediated non-probability panels are constituted as dual compliance structures, using both normative and remunerative power, but at least one successful, commercial panel in this group is based solely on a normative compliance structure. The Iniziopanel in Sweden, created in 2014 by the private research firm Demoskop, boasted 65,000 panelists (at the time of their interview with the author) and pays no monetary incentives at all. This non-probability panel is based on a partnership with Sweden’s largest newspaper and its parent media company. Columnists include invitations to the panel in their posted content. Survey results appear in the media and are discussed in the pages of the paper. Panelists are thus given a sense of being in dialog with opinion leaders and making their voices heard. Demoskop protects the panelist experience by keeping its surveys very short and by not giving outside survey panels access to their panel members.

Before moving on to direct exchange panels, one highly unusual non-probability online panel merits attention. Rare Patient Voice was started in 2013 as a nationwide panel of hemophilia patients, which had grown (by the time of their interview) to 125,000 patients and caregivers in several countries, representing 1500 medical conditions, rare or less rare. Rare Patient Voice recruits by taking part in community events, patient meetings and workshops to sign people up in person. They also engage over 3000 “referral partners,” activists in various patient communities. For each new participant a partner signs up, Rare Patient Voice makes a $10 donation to the relevant non-profit advocacy group.

For years, Rare Patient Voice paid $100 per hour to its panelists, but the rate was recently raised to $120 per hour. Payment is immediate, by check or direct deposit (no points).

This panel’s use of unusually strong remunerative power is matched by the strength of its non-material inducements, as seen on their online recruitment message for patients (emphasis in original) [https://RarePatientVoice.com/for-patients/patients]:

Share your Voice to improve patients’ lives! Rare Patient Voice provides patients with the opportunity to share their voices to improve medical products and services. Who knows better than you about your journey and experiences?. . Your voice can help you as well as others with your condition. When you join Rare Patient Voice, you may be invited to participate in interviews, surveys, or online communities where you will share your insights. You will earn $120 (USD) per hour for participating in these studies. . By making your voice heard, you are a catalyst for change. Rest assured that your input will be used to help other patients like you!. . .

The outsized amounts of remuneration are mentioned, but the language clearly gives emphasis to how the patient, and others who share their maladies, will benefit from participation. The opportunity to be a catalyst for life-saving change is, indeed, a powerful motivator.

Due to their in-person recruitment activities, the researchers have a closer relationship to some participants than most panel managers do. Perhaps more importantly, the panelists themselves are often active in patient-support groups and referral networks for the panel and therefore come to know other panelists personally. This means that Rare Patient Voice is able (in Etzioni’s terms [1961:6]) to mobilize social-normative as well as pure-normative inducements to achieve more than mere compliance, but positive commitment to the enterprise. From all appearances, Rare Patient Voice is doing the relational work needed to achieve a balance between high monetary rewards and high normative rewards. The result: a positive and durable compliance structure, an exception that belies Etzioni’s generalization that the use of one type of reward necessarily negates the efficacy of the other.

5.3 Direct exchange (“crowdsourced”) panels

Amazon Mechanical Turk, the original direct exchange panel, poses a sharp contrast to the panels described above. Recruitment information on the AMT website [mturk.com] focuses exclusively on work for pay. The initial page says:

Make money in your spare time. Get paid for completing simple tasks. … Become an MTurk Worker and start applying your skills to the thousands of available tasks in the MTurk marketplace.

The “How it Works” page says: “Find Work, Accept and Complete Work, Get Paid.”

The “Get Started” page shows prospective participants a blue piggybank with a dollar coin about to drop in the slot, over the label “Make Money.” AMT gives wide latitude to requesters to post surveys, without vetting questionnaires for quality or limiting their length. In sum, the platform does no relational work to either define their relationship to the participant as anything other than that of a paid gig worker, or to enhance the worker’s experience in doing surveys. It is evident that MTurk was conceived and realized—in Etzioni’s terms–as a purely utilitarian compliance structure.

Over several months, the author monitored traffic on a Reddit site for Turkers [www.reddit.com/r/mturk], without direct participation in their conversations. It is clear that the main contributors to the conversations there are professional respondents: people who rely on their earnings from MTurk as an important part of their income. Often, these experienced workers are in conversation with newcomers to the site, offering tips on how to get selected for good studies. They complain when the site offers too few tasks or when the tasks offer too little money. They exchange information about irresponsible requesters or those who do not promptly approve their completed tasks. They tell stories about various bad experiences, especially if a requester’s rejection of their work threatens their cherished 97-percent approval rating. They share their monetary goals for the week or month, and brag when they exceed these goals. There is a constant theme of rage that AMT itself is unresponsive and does too little to regulate or improve the site. There are frequent recommendations to Turkers to seek greener pastures on alternative direct-exchange sites, in particular CloudConnect and Prolific.Footnote 5 On the Reddit messaging site for Prolific panelists (https://www.reddit.com/r/ProlificAc), monitored for many months, the themes and orientations are very similar, although criticisms of that platform’s management are less frequent, and expressions of appreciation of the earning opportunities are not rare. Many of the posting panelists mention in their postings that they make hundreds and some make thousands of dollars a year from their work. The attitudes and actions displayed in these online conversations exemplify the calculative, partly or fully alienated worker role that Marx envisioned to be the end result of commodification and that Etzioni attributes to many utilitarian compliance structures.

An interesting avenue for further research would be to analyze the content of these chat platforms more systematically by randomly sampling postings and coding them for themes. Unfortunately, since there are no equivalent chat platforms for participants in any of the mediated panels, quantifying the strong impressions reported above would not lead to a comparison of attitudes across panel types.

MTurk is not alone in presenting only monetary rewards on its website. CfsPanel, a mediated opt-in panel, uses powerful imagery on its website to emphasize the monetary rewards it offers, as seen in Fig. 2. Some, like mediated opt-in panel CinchDollars, communicate their cash focus in the panel name; others do so through their slogan, such as direct exchange panel PanelOptimus, “Where every opinion pays” (see Fig. 3). Another opt-in panel, e-RG/Elite-USA-English [panelist.cint.com] uses text only, text that mentions “pay”, “cash” or “money” over twenty times on a single page.

Fig. 2
Fig. 2
Full size image

Images from cfspanel recruitment site [cfspanel.com]

Fig. 3
Fig. 3
Full size image

Images from two cash-oriented panels [CinchDollars.com, PanelOptimus.com/usa]

6 Quantitative results: comparisons by panel type

As seen in Table 1, the balance of non-monetary versus monetary rewards communicated in recruitment materials varies by panel type. Non-monetary rewards are primary on the websites of 30% of the probability panels, with the remaining 40% of probability panels giving equal weight to these two categories of reward. In contrast, monetary rewards are primary or solely featured in two thirds of the direct exchange panels. Mediated opt-in panels vary across the board.

Table 1 Balance of rewards mentioned
Full size table

The contrast between panel types in their balance of communicated reward is statistically significant when tested in a regression model that represents the balance as a five-point scale, scoring emphasis on monetary rewards with higher values. The statistical test uses the general linear model with contrast between each pair of panel types and takes into account the (estimated) finite populations from which each panel type is drawn.Footnote 6 As seen in Table 4, emphasis on monetary rewards increases significantly as one moves from probability to mediated to direct exchange panels.

Table 2 Use of photos and graphic images
Full size table
Table 3 How survey purposes are described
Full size table
Table 4 Differences in means for three panel types
Full size table

Another indicator of reward prominence is simply which type is mentioned first. Non-monetary rewards are mentioned first in 60% of the probability panel homepages, but monetary rewards come first in 57% of direct exchange panel homepages. Again, mediated panels are more evenly split on which rewards get first mention. This difference is in the expected direction but does not rise to statistical significance.

The images and photographs that appear on recruitment pages are a vital part of panel messaging. (All but three panels used visuals.) Table 2 shows that 57% of direct exchange panels show images or photos of cash, including money bags or dollar (or Euro) signs. In contrast, cash appears on the web page of only one probability panel out of four that used visuals. Gifts were not depicted on any of the probability panel pages but appear on about a quarter of the pages for mediated and direct exchange panels. (As seen in Appendix Table 2, those contrasts are statistically significant.) Half of the probability panels included images suggestive of non-monetary rewards, for example a megaphone to indicate the amplification of the respondent’s voice, or tight-knit groups of young people suggesting communal or social rewards. Only one of the seven direct-exchange panels depicted non-monetary rewards.

Table 3 shows how the purposes or uses of survey information are described on the different types of panels. All five of the probability panels mention scholarly or academic research, but only about half of the other panels mention research. (The low-cost sites are often used for graduate-student survey experiments.) All five probability panels mention government decision making, public policy or community issues, but none of the direct-exchange panels mentioned this type of outcome. (The contrasts for research and government policy are statistically significant.) Commercial research, such as development of consumer products, is mentioned by a majority in all three panel categories.

7 Discussion

A premise of this research is that the practice of paying respondents to answer surveys introduces awkwardness and ambiguity into the relationship between survey researcher and respondent. This tension is recognized in Dillman’s strong distinction between social exchange and economic exchange (2000), Marx and Simmel’s critiques of commodification, Etzioni’s thesis that the use of remunerative power in a compliance structure renders normative appeals less effective (1961), and the discussions in the new economic sociology of morally problematic situations where relational work is needed (reviewed by Bandelj 2020).

Panel managers engage in two types of relational work. First, the language and visual symbols they use when communicating with recruits and panel members are meant to define their relationship in ways that are favorable to the panel’s goals. Second, they can set the conditions of the panelist’s work to maintain a positive respondent experience through such practices as limiting the length of surveys, restricting the number of surveys panelists are asked to do, or providing ways for panelists to give feedback. Based on the author’s interviews with panel managers and the author’s direct experience as a panelist, these practices are more common in mediated panels than in the direct exchange survey marketplaces.

Panels differ, then, not only in their messaging but also in in their social architecture. Direct exchange panels facilitate direct contact between microtasking gig workers and research requesters, while mediated panels block direct contact and act as middlemen between researchers and their panelists. Probability panels, by definition, differ in their recruitment practices from non-probability panels. Considered in terms of their compliance structures, Demoskop’s InizioPanel (which makes no payments to panel members) is an example of a purely normative compliance structure, MTurk and others exemplify a purely utilitarian compliance structure, while many are—at least nominally–dual compliance structures that use both pure-normative and remunerative power.

The mixed-probability sample of active panels used in this study is large and diverse enough to show that probability panels, mediated opt-in panels and direct exchange panels differ from each other in their messaging to recruits and panelists. The non-monetary rewards offered by many panels are no different than those survey researchers have always offered in the classic non-monetary survey bargain. But there are some clear differences among panels in how completely such motivators are described and the salience given to non-monetary versus monetary rewards.

As seen in the coded sample of 28 panels, probability panels are more likely to give prominence to non-monetary rewards while direct exchange panels are much more likely to give prominence to monetary rewards. Photos and images on panel web pages were more likely to depict non-monetary rewards on websites for probability panels, while opt-in panels were more likely to depict cash or gifts. As expected given their respective mix of clients, probability panels gave prominent mention to the uses of their research to set government policy and support academic research, while opt-in panels most often mentioned commercial uses of their survey results.

Is relational work of the types described efficacious in producing compliance—that is, producing desirable behaviors and attitudes from the panel members? The quantitative results show a correlation of panel type with variations in relational work, but correlation alone does not establish causation. The present research has not attempted to measure panelist behavior, although the calculative orientations of professional respondents active in direct exchange panels are evident in the online conversations monitored by the author. As already noted, Kennedy et al. (2021; Table 5) showed that the incidence of insincere respondents was highest in a direct exchange panel, lowest in two probability panels, with the mediated opt-in panels falling in between (see also Dutwin 2026). And Ipsos Knowledge Panel has conducted research showing that probability and non-probability panels differ sharply with respect to limiting survey burden [https://www.ipsos.com/en-us/solutions/public-affairs/knowledgepanel]. However, a full study of the effects of relational work on survey quality, panel attrition and respondent experience is beyond the scope of the current study. Unfortunately, much of the information such a study would require is closely held by panel firms as proprietary data. Short of a large-scale experiment with panelists assigned to different relational work “conditions,” it will be difficult for any researcher outside of a large panel enterprise to establish that relational work is effective in its aim of keeping respondents motivated and behaving as researchers desire. At the least, however, the evidence reviewed here suggest that there is an “elective affinity” between high-quality panel maintenance and relational work that highlights non-monetary incentives for panel participation.

In a world where survey organizations are paying for answers, effective relational work is necessary for defining and maintaining the respondent’s relationship to the survey panel and its clients. When monetary and non-monetary rewards are offered simultaneously, there is a tension or awkwardness (and the potential for undesired behaviors) that calls for symbolic and substantive work to clarify the nature of the exchange relationship between surveyor and respondent. Some panels can plausibly claim that their surveys are world-changing, others have tenuous claims for social impact. But whatever may be the panel’s niche, panel managers must ensure that their communications and the ways they structure the respondent experience effectively symbolize the type of relationship they wish to establish, no matter whether the panel members are defined as low-pay gig workers chasing “beermoney” or as “impactful catalysts for change.”