Milieu Insight Response and Clarification on The Sunday Times Essential Workers Poll

Yesterday (June 15th) the Sunday Times published a set of 3 Article and an infographic about “essential workers” in Singapore that featured data provided by Milieu Insight. There has since been a great deal of discussion about the results online. And although it has been heartening to see some productive debate taking place about the core focus - perceptions toward low wage essential workers in Singapore - there has also been some critical response to the survey findings that dealt with Singaporean views toward essential vs non-essential jobs. As comprehensive details about the methodology and survey design are not freely available, we feel that it’s important we offer full transparency around the survey methodology and design.

As such, we are now sharing information about the background of the study, some details about our (Milieu) panel methodology, the specifications for the Sunday Times poll, and some additional information about the exact questions that were asked, how they were asked and some of the potential limitations of the approach.

“Essential Workers” Poll Background

The poll was commissioned by the Straits Times on June 3rd, 2020. An initial set of questions was provided by the Straits Times editorial team. A review of the survey was conducted by the Milieu team and revisions were proposed, such as changes to question types and survey flow. Such reviews are commonplace for a research provider to ensure questions sent to our community are understandable, answerable and do not maliciously bias or steer respondents toward a particular answer.

Milieu’s Panel Methodology

Milieu manages what is known as an online access panel (also known as non-probability sampling), where we recruit participants to join our opinion sharing community by downloading our app, Milieu Surveys.

To achieve representative results, we use a technique known as sample balancing where apply quotas for certain demographic variables that reflect the current national statistics. In this case, the poll used interlocking quotas of age and gender to achieve a representative sample of Singaporean adults, aged 16 and above.

Beyond sample balancing, Milieu also applies a wide range of fraud detection and quality control mechanisms at both the panel and survey level. This includes a wide range of automated attention checks to ensure respondents are both paying attention and providing Milieu with consistent responses throughout a survey. Milieu is also fully compliant with ESOMAR research and data collection standards.

Specifications of the Sunday Times Poll

As the main article is behind a paywall and not all of the details about the poll methodology were available in the infographic, we feel it’s important to share the core specifications of the survey, which are listed below:

Methodology: Online (Milieu panel)
Sample Size: n=1,000
Target Audience: Singapore adult population (residents and citizens), aged 16 and above
Field Work Dates: June 12th to 15th, 2020
Survey Duration: 19 questions (approx 5 mins)
Margin of Error: +/- 3% at a 95% confidence level

Note. At n=1,000 samples (with sample balancing in place) the margin of error for this study is +/- 3%. What does this mean? Let’s take this statistic as an example - 73% of the respondents indicated that “I respect essential workers more now”. With a margin of error of +/- 3% this means that we’re 95% confident that between 70% to 76% of the population would answer this way if we interviewed the entire adult population of Singapore. You can read more about Margin of Error here.

Design of the Sunday Times Poll

The Sunday Times “essential workers” poll included 19 questions. Most of the questions in the survey addressed current perceptions and attitudes toward low-wage essential workers in Singapore, such as cleaners or security guards, and whether there is support for paying them more.

Understandably, much of the focus has been on the data points related to essential vs non-essential jobs, and we’d like to offer clarity on how, exactly, these questions were asked.

First, respondents were prompted with the following definition at the beginning of the survey:

In this survey we're going to be asking your opinion about essential workers in Singapore. By "essential workers" we mean someone who is engaged in work deemed necessary to meet basic needs of human survival and well-being, such as food, health, safety and cleaning.

This statement was included to ensure there was a baseline understanding of the term “essential workers” among the respondents from the beginning of the survey.

Next, respondents were shown 2 questions:

Q1. Please select the jobs that you think are MOST essential from the list below.
Q2. Now select which of the remaining jobs you think are absolutely NOT essential.

The following list was displayed to respondents for both questions. However, it’s important to note that for Q2, only options that they did not select in Q1 were displayed (a mechanic known as negative response piping). The response list was also randomized, meaning each respondent saw a different ordering of the jobs list.

• Artist
• Cleaner
• Construction worker
• Business consultant
• Corporate lawyer
• Deliveryman
• Doctor
• Engineer
• Garbage collector
• Hawker
• Human resource manager
• IT technician
• Nurse
• Politician
• PR specialist
• Security guard
• Social media manager
• Teacher
• Telemarketer
• University professor
• I don't think any of these are essential / non-essential

Both Q1 and Q2 were programmed as multi-select questions (choose all that apply) and not as a ranking question. This means that respondents were allowed to select as many or as few options from each list as they saw fit. There was also no cap placed on how many options they were allowed to select (i.e. they could select all or none). So respondents were not asked to rank the jobs/professions, rather, they selected the ones they deemed essential vs not essential given the definition provided. And although the question was not presented as a ranking to respondents, it is common practice to present the results of such a question in this way, showing the options from most selected to least selected.

It’s also important to highlight that respondents were not shown an exhaustive list of jobs, nor were they allowed to key in an open-ended response. They were shown a shortlist of 20 jobs/professions. There are some important considerations for interpreting the data based on this design approach. In particular, respondents were sorting jobs based on what they deemed essential vs non-essential according to a) the definition of “essential jobs” provided and b) the shortlist of jobs shown.

We also noted a critique of other questions in the survey that dealt with whether Singaporeans think essential workers should be paid more, or if they would be willing to pay more for services if they knew the additional money went toward the workers. The common critique we observed is that respondents would not have the necessary context or experience to provide a meaningful response. And so we wanted to clarify that respondents were also provided with content screens for such questions to provide the necessary context.

For example, for this question in the survey:

Do you think persons employed as construction workers, cleaners or security guards should be paid more?

• Yes
• No

Respondents were shown the following statement before seeing the question:

The monthly basic minimum wage of a security officer and a general cleaner under the Progressive Wage Model is $1,250 and at least $1,236 respectively. The median monthly salary for construction labourer's wage in 2018 was around $1,200.

Beyond the clarifications on the survey design and methodology covered above, we also felt it would be useful to add some further insight into what the results mean.

Does the context of the Singapore Circuit Breaker affect the results?‍

Absolutely. As with any poll, timing matters and responses can be affected by a multitude of environmental factors. In this case, “essential workers” has been a recurring narrative for many Singaporeans in the news over the last few months, and so respondents were answering in a post-Covid-19 context. That, and we provided a specific definition to ground their response (“…someone who is engaged in work deemed necessary to meet basic needs of human survival and well-being, such as food, health, safety and cleaning.”)

Did the respondents rank the jobs?‍

No, respondents selected jobs/professions based on a list of 20 options provided, first by what they deemed essential, and then by what they deemed non-essential. And this was grounded on the definition of “essential workers” provided at the beginning of the survey. The respondents were able to select as many or as few jobs from each list as they saw fit.

Do Singaporeans think artists and professions such as HR are non-essential?‍

This one requires some unpacking, but the simple answer is, not necessarily. Again, respondents answered according to the definition of “essential workers” that was shown at the beginning of the survey, which was clear about meeting “basic human needs”. Some could argue that art is a basic human need, and many of us at Milieu would agree. But in the context of a post-Covid-19 world, and based on the definition of essential workers provided, these are the jobs, based on a shortlist of professions, that Singaporean adults deem essential vs non-essential.

This does not mean that Singaporeans view professions such as artists or HR professionals as not valuable or valued, or even essential in the context of something else, like happiness or stability. And these are important questions to be asking, perhaps in another study. We’re confident that, if asked, most Singaporeans would agree that artists and their work, be it a musician, painter, dancer or filmmaker, are absolutely essential to the enjoyability of our lives and our own mental health and well-being. Or that many Singaporean working professionals would likely also agree that HR is the backbone of most successful, productive, and progressive companies today.

So there are indeed questions that the survey doesn’t answer, as well as some caveats for how we interpret the results of the questions that were asked. Either way, we hope that this offers greater transparency around the methodology, as well as some added perspective on the implications of some of the findings.

‍

Milieu features online survey tools aimed at aiding your market research initiatives. Moreover, delve into the essence of workforce engagement surveys in the present day.