about   |   mailing list   |   writing   |   archives

Connections. Connectors.

Shanghai: connections
 

A critical part of any ethnographic/design research project is recruiting the right participants for the study – they are the foundation on which the research is built. The default way of recruiting in the commercial research space is to use recruiting agencies to help connect the researcher with relevant participants – generating a list that is often fleshed out by contacts from the team’s extended social network. The ideal recruiting agency list-of-potential-participants contains hundreds of millions of entries and document every aspect of potential participant’s lives – what they are doing, who they are doing it with, the causes they feel passionate about, the brands they connect with, the music they listen to, the places they go – and all updated in real time. Thanks to social networking sites like Weibo, Facebook, Orkut and Mixi this ‘ideal list’ already exists, and comes with a built in mechanism – their advertising platform to engage participants and proximate participants to opt into the study.

The ability to recruit through extended social networks has always been an important part of the researcher’s toolkit – with varying degrees of success depending on the focus of the participants’ profiles, the physical and spiritual distance between the study location and the team, and the breadth of the team’s extended network. The internet has made the planet smaller, social networks more apparent – making remote studies that much easier to run. Today the tools to rapidly and consistently reach and screen participants in any part of the globe are in the hands of every internet connected researcher. My estimate is that 80 to 90% of current recruiting for design research/ethnographic studies (excluding focus groups) that is currently placed through recruiting agencies could from a skill and work-flow perspective, be carried out in-house. This internalising of an otherwise outsourced practice has a couple of costs, which in most cases are easily outweighed by numerous benefits

As the literacy and awareness of these new recruiting practices grow in the community, so will the number of organisations that build out their internal capability to recruit. Recruiting agencies that make money from facilitating connections will require specialising on niches that are not easily met by social networking sites or the kinds of relationships that they support and they rapidly need to find alternative ways to add real value to the process beyond the usual logistical planning, translators, transcription. For researchers this means learning new skills: maintaining an online identity that is a suitable interface for potential recruits; knowing how to gauge reach through which social networking sites, running and iterating an ad-campaign; effectively screening and knowing how to turn leads into participants. Whilst it is relatively early days the effectiveness of the platform and the low barriers to entry will mean that the change will be rapid. You are the agents of this change.

Try Before You Buy

As I’ve written before, Facebook has a particularly effective ad platform to place an advertisement and target all the usual demographics and then some: age, gender, college, degree subject, year of graduation, profession, languages spoken, living within a certain distance of a particular locale and so on. Take 5 minutes to try the following:

  • 1. Go to Facebook Ads
  • 2. Click on the Create an Advert button
  • 3. Fill in dummy Design Your Advert information (it won’t be live)
  • 4. Play around with the Targeting parameters.

You don’t need to run an ad campaign to make use of the Facebook data – the targeting parameters are good at gauging the likelihood of finding local bilingual guides and feeling out the landscape in which the team will work – especially when combined with satellite data from Google Maps – for understanding the spread and shape of the city/town/village which works whether you’re planning to run a study in Estonia or Ethiopia, and have a diverse understanding of living types around the world.

The basic process for running at study in a ‘foreign’ country is:

  • 1. Run advert to recruit (bi-lingual) guides
  • 2. Use guides to translate and run a locally relevant advert to recruit participants
  • 3. Direct participants to an online screening form which will populate a password protected spreadsheet. See a sample screening form here.
  • 4. Have the team review the responses in the spreadsheet as the data flows in, reach out to participants that are suitable. Google Docs is great at supporting synchronous and asynchronous annotating such as disperate team members flagging suitable candidates. A sample of backend data is here.
  • 5. Check in on Facebook Ad Manager and tweak the ads. Running many ads at once makes it easier to understand what works, what doesn’t.

Google Docs make the screening process painless. As with other part of the study having a ‘live’ shared document used by the whole team makes tracking current progress easy – and is particularly useful when working with a large, distributed team on tight deadlines.

The quality of targeting data such as location, age, gender etc varies from country to country – somewhere like the US and Europe it has proved sufficiently accurate, for our recent study in Nigeria where more people are filling in their profile information through a mobile phone, and where bandwidth costs remain an issue it can be patchy. Despite Facebook’s dominance it is not universal by a long stretch – especially in China where there are strong local competitors and anyway access requires a VPN. Knowledge of multiple social networking sites and their ad platforms is a must.

As much as I’d like to think that the platform can be used to raise quality, it also lowers barriers to entry for fly-by-night companies and less critical clients. Even without social networking sites problems occur- in China recruiting a participant in a less-commonly used second tier city can cost 800 Euro (1,000 USD) for a full day’s access to a family – and it is no surprise with such high potential rewards unscrupulous agencies are driven to fraud. In China we’ve had to deal with a local agency that not only falsified participant data to fit our screening profile, but had family members act out the part. A team that conducts the screening themselves and is able to delve into a person’s online profile allow them to make a judgment call on the accuracy of the data being collected. Data trails and shadows will increasingly be tricky to fake to a researcher that knows how to look beyond the headlines.

Assuming you have a Facebook profile and a credit card or PayPal, it can take as little as 20 minutes to set up an ad campaign that points users to an Facebook Event, Page or other web page that can list more details about the study and contact details. Facebook Ad Manager makes it easy to track click-through and other measures of success – and tweak under-performing ads on the fly.

Dynamics

How many participants are sufficient to run a study? Using an example from a recent study – lets assume we want to run a 10 in-depth interviews in participant’s homes in Bangkok, where nobody on the team speaks Thai. The first advert targets potential bilingual guides – I prefer not to use professional translators – since they are overly motivated to creating billable hours, rather than getting in the spirit of the study which in the way that I work requires considerable flexibility. Once hired, the local guides can help write an advert in Thai. Thai’s are generally curious about foreigners, tend to have more variable income than more economically developed countries and many want to practice their English making finding participants relatively easy. Several iterations of a well targeted ad costing around $150 Euro can be enough to generate 30 – 100 leads which can be filtered down to 12 potential candidates – 10 + 2 spares. My preferred sweet spot is 30 to 50 leads – too many and the team spend to much time on filtering, too few and the gene pool is not diverse enough to sustain interesting life. Of course it is about quality candidates, not click-through – it takes a while to figure out the appropriate place to filter-out candidates: in the advertisement; in the landing page; in the screening questions; and in the follow-up screening of the responses.

The cost of Facebook Ads varies considerably according to the demographic you are targeting – it could be as little as a couple of Euro cents a click, as much as 5 Euro a click. The ad platform makes it easy to control daily or total spend.

Every recruiting medium has its own social dynamics and inherent levels of trust. Being able the check out the public profile and photos of candidates that have opted into the study gives the team a far better sense of what they are about. Similarly it can take a few passes before understanding what is the appropriate language, visual, tone for the advertisement. Likewise potential participants can screen the recruiting company or individual when the event page is associated with an individual Facebook account.

Relying solely on respondents to advertising can naturally skew the participant pool and can make the study feel brittle. I like to add breadth, depth and flexility to the team by utilising extended networks, and having a strong local crew. It is worth spending time to nurture local talent around the world both on and off the project for times like this.

In many instances you will want to reach demographics that are not active on social networking sites – probably about ~6 billion people worldwide. – which is where the notion of proximate participants comes in. A proximate participant is anyone or organisation that can provide a connection to an off-line participant – local NGOs, relatives, government agencies and the like. Understand their motivations, use incentives wisely. The biggest failure I see in recruiting is people throwing money at the problem – when the critical success factor is in framing the relationship.

This Is Why We Do It

The key benefits of carrying out recruiting in-house are, making effective use of social networking platforms and online tools are:

  • &#187 Time to market: What used to take a couple of weeks using a recruiting agency can now be done in days in-house.
  • &#187 Progress can be tracked in real time.
  • &#187 Flexibility: the extended team can share thoughts on appropriate respondents in real time, screening questions can be added, tweaked, data can be translated, they own the in-field schedule
  • &#187 Cost: can be 1/10th of the price of a traditional recruiting agency, plus whatever time the team requires for screening.
  • &#187 Ownership: many recruiting agencies want to ‘own’ the relationship with the participants – a very modern form of slavery that can estrange the study dynamics. When own the list, you judge what is appropriate and what is not.
  • &#187 Fraud Detection: it is possible to lie on a screener, but far harder to fake an entire social network.
  • &#187 Depth of understanding: the team that recruits inherently learns about the market and the target demographic before they arrive which leads to more nuanced questions, faster acclimatisation, richer insights.
  • &#187 Quality: the net result is that the quality of the research and the deliverables is higher, the relationships with participants more meaningful for most types of research topic, the team more motivated.

Carrying out recruiting in-house using these tools changes the very nature of the study – making it easier to compress ramp-up and turn an already dynamic process into something that is totally fluid.

Want to join a field study as a fixer/researcher/guide?
Fill in the Fixer/Guide/Researcher/.. Screener

See also the Fixer/Guide/Research Assistant List on Facebook