• Tidak ada hasil yang ditemukan

Qualitative Analyses of Open-Ended Responses in a Large Panel Survey

N/A
N/A
Protected

Academic year: 2023

Membagikan "Qualitative Analyses of Open-Ended Responses in a Large Panel Survey"

Copied!
57
0
0

Teks penuh

This paper explores the public sentiment of the people of Bangladesh regarding the lockdowns imposed by the Bangladesh government in 2021 in response to COVID-19. Through open-ended question design and natural language analysis using NLP and sociolinguistic techniques, we reveal detailed, nuanced feelings, as well as common themes and discussions where these feelings are located. Across the world, government-imposed lockdowns included a sustainable strategy to limit the surge in COVID-19 cases.

The impacts of these blockades were multiple and had significant impacts on the lives of the residents. For this project, Decodis partnered with the BRAC Institute of Governance and Development (BIGD) to explore questions about public sentiment about government-ordered lockdowns in Bangladesh. We explore these questions through an open-ended question design implemented within a large panel survey conducted by BIGD with 3,000 participants.

By leveraging Natural Language Processing (NLP) and sociolinguistic techniques, we are able to better establish that the feelings of these Bangladeshi residents at a granular level are "on the ground". In addition, we conduct further analyzes in this project of counters' discourses and the resulting potential effects on data.

Method

  • Question Design
    • Questions are Interactional
    • Respect for Participants
  • Question Choice
  • Data Collection
  • Transcription and Translation
  • Data Analysis
    • Natural Language Processing (NLP)
  • Sociolinguistics
    • Emotional models of interpretation
    • Discourse Analyses
  • NLP + Sociolinguistics: Clusters

The questions chosen for this project reflected Decodis' best guess as to where we could add the most value to the panel surveys. Question (English) Question (Bangla) Q72.3 What do you think of the. government's decision to introduce a lockdown to contain Corona. In addition, they were instructed not to provide follow-up information, but simply to ask respondents to answer to the best of their ability and understanding.

This would ideally minimize the influence of the recorder on respondents' response to open-ended questions. Utterances usually correspond to sentences, but, depending on the nature of the language, the participants and the context of the conversation, they can be less than one sentence or multiple sentences. As such, it is up to the researcher to examine clusters of words to better understand their associations and, subsequently, to determine a set of themes/themes.

Drawing on research topics such as Attitudes, Attitude and Speech Emotion Recognition, we seek to relate features from the acoustic signal to the emotional/attitudinal framework of individual speakers. First, it is important to note that interpretations of speech signals (emotional analyses) are layered on top of textual analyses. These analyzes consider the pressures and constraints of the discursive context on the behaviors themselves and the ways in which speakers navigate, push, or reinforce these factors.

To do this, each response is first annotated according to the topics it contains as well as specific speech signal measures.

Data, Results and Discussion

Attitudes to COVID-19 Lockdown

  • Prevailing Themes on the Lockdowns
  • Theme-Sentiment connections
  • Discussion: Theme Sentiment Connections

Sentiment-based segmentation of responses to "What do you think about the government's decision to impose a lockdown to ameliorate Corona?". Following this methodology, we found three consistent themes related to congestion that emerged in the data: “security”, “the poor” and “economic impact”. As a topic, "the poor" includes discussions of the effects of lockdown on poor people specifically.

The use of personal pronouns (or their grammatical analogues) performs a kind of "social deixis", a positioning of the referent at social distance from the referent (Hart 2010, Cummings 2005). That is, if a respondent answers, for example, "We poor people starve", the use of "we" expresses the speaker's personal experience of poverty (they refer to themselves) and further connects them to a collective experience (they use "we " not "I"). We found that over two-thirds (69%) of respondents who speak of “poor” self-identify as poor in this way (see Figure 3.2).

Another reason may be that these respondents are only poor, becoming poor during the pandemic or because of the lockdowns. Indeed, many of the responses we see in this sample seem to achieve this: "poor people, like us, suffer". Overall, we find that 98% of those who identify as poor have a negative view of the lockdowns.

That is, responses that provide a positive sentiment tend to focus on discussions of "safety," while negative responses focus primarily on topics of poverty or the economic impact of closures. The government always wants to protect the measure we need to know that." These responses call for a shared discourse of lockdowns that help slow the spread of COVID-19 and the health risks of COVID-19. Meanwhile, the negative responses focus on the harmful effects of the lockdowns, which, as expected, disproportionately affect the powerless and those without means (ie "the poor").

Furthermore, as can be seen in this example, the positive parts are usually very short, oversimplified parts of the answer; they lean towards a "general" positive - sort of a "lip-service" positive. The counter coding results showed that nearly half of the sample said the decision was "good", while the majority of the rest were coded as "other". The only real expressions of "good" - as in the discussion above - were those who were "nuancedly positive", which was only 11% of the sample.

Segmentation of responses to the question "What do you think about the government's decision to introduce a lockdown to contain Corona?". Enumerator coding for "What do you think about the government's decision to impose a lockdown to contain Corona?" within each segment of NLP+sociolinguistic coding.

Figure 3.1.  Segmentation based on sentiment of responses to “What do  you think of the government’s decision to impose a lockdown  to contain Corona?”
Figure 3.1. Segmentation based on sentiment of responses to “What do you think of the government’s decision to impose a lockdown to contain Corona?”

Requesting Relief

  • Specified Items
  • No item specified
  • Hesitancies in Requesting Relief
  • Discussion

In this way, the enumerator's coding overestimates and oversimplifies the "good" position and underrepresents the width of shades in other positions. Interestingly, although the wording of the question clearly asks "besides food", many respondents (16%) say that "food" is their most needed help. This disregard for the intent of the questions may indicate the necessity of these responses for food aid.

In addition, respondents in rural areas requested the most food of the items listed, while respondents in cities requested company stock (Figure 3.9). Finally, we also found that day laborers and white-collar workers asked for goods across the range, but business owners were far more likely to ask for company stock (Figure 3.10). In fact, a number of respondents responded in exactly that way, saying that they cannot sit and wait for relief: "I work on a day-to-day basis and earn my money and food this way, how can I sit idly by and wait for some people come and help me and save me?”.

About 1/5th of respondents deny any hesitation to ask, either explicitly ("I have no problem asking for help" (11%)) or implicitly ("I don't need help" (9%)). More than half of the respondents reflected this internalized idea in their answer, either by explicitly saying that they felt embarrassed to ask for relief (34%) or by a more general confirmation of the question (19%). There is some alignment between the NLP results and the enumer results, for example, "Cash" from the NLP results is aligned with "Financial Assistance" from enumer results, which are respectively at 45% and 42% of notifications come in.

First, even though the question states "Apart from food items...", 16% of respondents mentioned food and two-thirds of those respondents mentioned specific food items such as rice, lentils, potatoes or oil. Although they have been asked to give different answers, they take the opportunity to "bring home" the importance of food. A total of 12% of the time, tellers only coded these types of long-term solutions, such as "arrangement of new income-generating activity" or "loan assistance."

In the NLP results, we show that respondents mention "business loans" and "jobs and work" almost a third (29%) of the time. Of those respondents who did not specify an item, enumerators coded their responses 77% of the time as a request for “financial assistance,” s. While there is some equivalence between NLP results and enumerator coding, the results are largely different and paint a very different picture.

Table 3.3.  Response categories to what kinds of relief would be   most beneficial
Table 3.3. Response categories to what kinds of relief would be most beneficial

Enumerator Discourse Analysis

  • Talking Amount
  • Talking Over
  • Post Talk
  • Discussion

Figure 3.13 shows the relative amount of speech enumerators on average per question. This means that a score of 50% would mean that the enumerator and respondent spoke for the same amount of time and had the same proportion of the conversation. Anything above 50% means the enumerator was talking more than the respondent and therefore probably dominating the conversation.

A possible explanation for this is that questions 79.9 and 79.10 are more complex questions and require a greater degree of personal reflection than 72.3. We see that very few recruiters approach a fully fair share of the conversations on average. First, we look at what percentage of the respondent's answer is spoken by the enumerator.

Here, the higher the percentage, the stronger the claim of power by the teller i. Post Talk is measured from the moment the respondent starts talking after the teller finishes asking the question. First, Figure 3.17 indicates the average amount of post talk that each enumerator engages in across all questions.

Here we see that a quarter (25%) of the survey on average consists of the teller's talk. Said differently, this number reflects the amount of potential response time the teller takes up and the amount of research they take after asking the question. The higher the percentage in this measure, the greater the risk that the counter will influence the respondent's answer. of survey that is post talk.

Specifically, in this exercise, feedback like "yes", "hmm" or "sure". sometimes called "back-channeling") will be classified as Post Talk. This form of Post Talk is not worrisome and instead affirms the respondent's power and encourages responses. In other words, the extent to which counters have any of the above behaviors is not systematic.

Figure 3.13.  Talking amount increases in later questions
Figure 3.13. Talking amount increases in later questions

Conclusions

Gambar

Figure 3.1.  Segmentation based on sentiment of responses to “What do  you think of the government’s decision to impose a lockdown  to contain Corona?”
Figure 3.2.  Proportion of respondents identifying as poor vs those   who do not
Figure 3.3  shows the sentiment breakdown of those who ID as poor. Overall,  we find that 98% of those who ID as poor express negative opinions of the  lockdowns.
Figure 3.4.  Distribution of the two types of positive responses 3.1.2.2.  Strong Negatives
+7

Referensi

Dokumen terkait

The objective of the study is to investigate whether the use of cartoon movieis more effective than story text in teaching vocabulary to the eleventh grade