Chapter 14 Field Work

14.1 Timing

The timing of a survey operation is determined by the resources available, and (more importantly) the need for the information. There is no point reporting an opinion poll of voting intentions after the election has taken place. Results must be timely (released at a useful time).

This may mean that a large interviewing force is required, to get the information collected in a short time window before an important event, or before an important decision is made (a decision relying on the survey results).

Surveys with extended surveying periods can be affected by current events (e.g. the Canterbury eathquake, changes in the economy, or a political scandal) and also by the changing seasons through the year. People may respond differently depending on the time of year (think about a survey of agricultural workers, or of ski instructors). Surveying in the school holidays can lead to a lot of non-contact non-response.

14.2 Survey Mode

A major question in field work is of course the mode of the survey, since it determines to a large extent what kind of field work is required.

  • Interviewer administered, Self administered, Observational?
  • Face to face? Postal? Telephone? Video Call? Web form? Physical measurements?
  • Computer involvement? All on paper?

Some common modes:

  • Personal interview – a face-to-face encounter where the interviewer asks the questions, the respondent replies verbally. The interviewer writes down the responses on a paper questionnaire (or a tape recorder/video camera captures the responses for later analysis – the respondent must know if s/he is being recorded).

    It is common for multiple choice questions to have show cards with the list of possible responses (e.g. ethnicities), and the respondent chooses one.

  • CAPI – Computer Assisted Personal Interview – as above, but the interviewer records the answers on a laptop. The laptop also provides the script which the interviewer reads.

  • CATI – Computer Assisted Telephone Interview – as above, but not face-to-face – instead over the phone. The interviewer records the answers on a laptop (show cards cannot be used of course).

    For very short surveys, e.g. of customer satisfaction, the responses may be collected by touch tone phone or by voice recognition software.

  • CAVI – Computer Assisted Video Interview – like CAPI, but not in person – instead using video conferencing. The interviewer records the answers on a computer, and can if needed display show cards on screen.

  • CASI – Computer Assisted Self Interview – A face-to-face encounter with an interviewer, but the respondent enters data directly into the laptop the interviewer has brought. There may be a CASI section in a CAPI interview, for questions which are very sensitive.

  • CASQ – Computer assisted Self-administered Questionnaire – the questionnaire is provided in, say, a web application. No interviewer is involved.

  • Postal Survey – the questionnaire is posted to the respondent, with a self addressed (and post-paid) return envelope.

The mode of the survey affects the way that people respond – people will respond to the same question differently if it is asked in different modes. This is called the mode effect.

It can be difficult to get detailed or personal information over the phone, and such interviews are usually short. Questions in telephone questionnaires tend also to be short, since there is no opportunity to use visual cues or show cards, and no ability by the respondent to reread the questionnaire. Sensitive questions may be better in a face-to-face situation, although this increases the likelihood that people won’t give honest answers to questions that they feel might make them look bad (uncaring, unfeeling, judgemental, ungenerous), or show them to have views that are held only by a small minority. This latter effect is called Social Desirability Bias.

14.3 Field Work in Practice

Having selected a sample frame, a sample size, and a method of selection, the units from which data are to be collected can be identified and contacted.

The relevant steps should be clearly planned out, and relevant contingencies allowed for.

Example. If the sample is of students inside schools, I may only have a list of schools, but not of classes, nor of students. I’ve decided to do a sample of students selected from classes within schools. How does a questionnaire get from my office to the desk of a selected student, and back again? What actually has to take place?

  • Select a set of schools from the frame (e.g. by SRS or stratified SRS)
  • Write to the principal of each selected school, explain the survey (possibly in an appointment face to face), get agreement, schedule a time for sampling;
  • For respondents under the age of 16 I need to get parental permission – I need to write a letter home, and get them all back again. (Should this letter go to all students’ parents, or only of my selected ones?)
  • I’ll need a list of classes within the school to sample from (can I rely on the admin staff at the school to make the selection for me? probably not).
  • If I have multiple collectors covering different schools I need to train them all up first. I’ll need training sessions and a set of instructions. I may need to run role play sessions with them. They need to all act exactly the same way (it shouldn’t make any difference which collector goes to which school).
  • At each selected school I then need to select the classes randomly (and be sure that students belong to only one class). I need to contact the teacher of each selected class, and arrange a suitable time and place to survey.
  • I need to arrive with enough surveys to distribute, explain and get the students to carry out the survey. If I’m subsampling within the class I’ll need a means of randomly selecting from within the class. I need to decide what to do about students who are absent on the day.
  • I get the students to complete the survey.
  • I then collect all of the surveys back.
  • I need to be able to store the surveys securely until they can get back to the office. If I have lots of interviewers/collectors they’ll all need to have secure locations too.
  • I need a method of getting the surveys back to the office from multiple collectors. I may get the collectors to post their sets of surveys back. I may be able to send encrypted emails of electronic survey data, or better to physically transfer data sticks.
  • I need to capture the data from paper questionnaires into electronic form for analysis. I may get two or more people to enter the data independently and compare results for consistency. Electronic data need to be read from laptops or data sticks into my survey dataset.
  • After data capture – what do I do with the survey forms? Do I keep them? where? for how long? When and how do I destroy them?
  • I need to interpret the unit record data I have collected – coding responses to standard classification systems and perhaps doing text analysis of open ended question answers. I may need to edit or impute data.
  • I need to calculate the survey weights, making adjustments for unit non-response.
  • I finally have a unit record dataset that I can use for analysis.
  • Who will have access to the data? How long will I keep it? Can I give it to anyone else? Can I use it for research I hadn’t planned when I set the survey up?

There is a wide variety of surveys, and correspondingly a wide variety of considerations for field work. Here is a non-exhaustive list of some things to consider:

  • Surveys of animals (including humans) have to take account of the fact that animals move, and that they hide/sleep/migrate/are born and die. How do I adjust for these factors? are they problematic?

    Short time frame surveys can usually avoid the problems of birth and death. Movement is more of a problem.

  • For surveys of animals a strict set of instructions is required – and an observation schedule, analogous to a questionnaire, is needed. The instructions may include the requirement that surveyors do not communicate with each other to ensure the independence of their results. (What do I do about differing levels of skill in observation? What if the weather is bad?)

  • Coverage rules: I need to ensure that each unit has only one chance of selection. If I’m surveying people in dwellings, but it takes 3 months to do all my surveying, what do I do (a) with people who have multiple dwellings, and (b) with people who move during the survey period? or who are on holiday?

  • The flow of documents: how do the surveys get to and from the respondents. Do I need lots of stamped addressed envelopes? How do I ensure the security of the data I am collecting?

  • Is any equipment required? (Laptops, weighing scales, pens and paper, mobile phones, cameras, recording devices, light meters, …)

  • How do I ensure the highest possible response rate? What kind of pre-notification is needed? What permissions are required (e.g. to approach children? or go on to private property?)

    Surveys of humans and non-human animals conducted by Universities and by government usually require an ethics committee sign off. In an actual survey you’d need that all signed off before you start.

  • Am I offering incentives for response (a gift? a prize draw?) How do I distribute these?

  • What do I do about non-response? How do I even know if it is happening? Do I need serial numbers on all the questionnaires? Can I check to see how many people have responded so far?

    In a postal survey it is usually better to send out new questionnaires and a reminder letter, instead of just a letter asking people to complete a questionnaire that they may have lost (what happens if they send back two questionnaires? will you even know if this has happened?).

    How many reminders will you send?

  • In a postal survey – how will you address the envelope? If you are subsampling within a household you can’t ask the housholders to randomly select someone – it won’t be random! (and it probably won’t happen at all).

    You really need the selected person’s name and address – this information needs to be on the frame. (Face-to-face dwelling based surveys can find out names on the doorstep.)

  • What information needs to be supplied to the respondents? Some of this should go in a pre-notification letter if this is being used:

    • Who is running the survey
    • What the survey is for
    • What will happen (e.g. an interviewer will call …)
    • How long the survey will take, and other information about what the respondent will have to do (e.g. complete a paper questionnaire, or answer a set of questions face-to-face with an interviewer)
    • The privacy and confidentiality issues relevant to the survey
    • How the data will be used
    • How to contact the survey organisers
    • An opportunity to see the survey report?

  • How many call backs are required before a collector can finally code a sample member as a non-response? Will you collect information about whether a non-response was a non-contact or a refusal or some other reason?

    What if the selected person cannot speak English? (or whatever other language(s) are being used?)

  • Large surveys may have a dress rehearsal where a small amount of data is collected using the methods and procedures that the survey will use – just to test the field work procedures.

  • Will proxy responses be allowed? (i.e. one person responding on behalf of another)? This is justified only in very particular circumstances. Proxies can work if the information is particularly simple (is this person still unemployed, asked of a family member), or if the person is a caregiver for the respondent, and knows a lot about their situation. But they do carry a significant risk that the proxy respondent can’t (or in some cases won’t) answer accurately.