Polling is one way, among many, to actively listen to the people we serve – in this case, a representative sample of all Coloradans. By having 20-minute conversations with a sample of people who look like the population of our state, we can better understand the priorities and attitudes of a wide range of Coloradans. The Colorado Health Foundation’s annual poll aims to take the pulse of Coloradans on a range of important health issues each year to inform policy far into the future.

Through Pulse: The Colorado Health Foundation Poll, we interview more than 2,000 Colorado adults each year, a large sample that allows us to dig into differences in perspectives based on race, ethnicity, income, geography and a host of other factors. In fact, one of the upsides of this kind of quantitative research is that it allows us to discern patterns in what different groups of Coloradans are thinking or experiencing.

Through the poll, we learn a lot about how people are feeling and what priorities they have – and we share those with policymakers, community leaders, the media and anyone else who is interested. These findings also inform our work to advance health equity.

Polling has its limitations, too – namely that we only have about 20 minutes with each Coloradan we interview. That’s why there are other research methods we use to understand even more about Coloradans’ views and what drives those. For example, we’ve used qualitative research like focus groups to dive deeper into topics with smaller groups of people so we can really understand the mindsets of Coloradans.

Each year Pulse gathers responses from two samples.

The primary way in which we get responses to the poll is through a probability sampling methodology. That means that participants were randomly selected from a list of all Colorado adults with addresses with every adult having the same chance of being selected. Addresses were matched to phone numbers and email addresses, and participants were contacted by phone call, text message, email and mail with an invitation to complete the survey. We use data from the U.S. Census Bureau to ensure that the sample of people we interviewed matches the characteristics of the entire population of adults who call the state “home.” The secondary sample uses a convenience sampling methodology, where respondents come from a group of people easy to contact or to reach. We added this component to our methodology to increase the number of Native Americans we’re able to interview. Since Native Americans living in Colorado make up less than 2% of the population, it is difficult to achieve a large sample size through our probability sampling methodology. To increase the sample size of Native Americans interviewed – and thus have more confidence in what we learn from the data – we partner with community organizations that serve Native Americans living in Colorado. Those organizations sent an invitation to participate in Pulse out to their networks via email, social media and text. Because of these partnerships, we were able to increase interviews with people who identify as Native American, American Indian and Indigenous.

Importantly, we specifically choose to conduct this research with adults – not just voters – living in Colorado. This decision follows our commitment to listen to all Coloradans, not just those who are registered or likely to vote. It also distinguishes our approach from much of the other polling that is conducted in our state on behalf of candidates or issue campaigns. Our purpose is not to predict the outcome of elections, but rather to listen to the people of our state and accurately reflect what Coloradans are thinking and experiencing.

People experiencing homelessness and housing insecurity are among some of the most challenging folks to include in quantitative research projects. With an address-based sample, it’s likely we didn’t reach many people who have been chronically homeless. This is a limitation of our methodology, and it’s one that challenges us to use other methodologies to listen to people experiencing homelessness in order to understand their opinions and experiences. Even with the limitations of this methodology, each year Pulse finds high levels of worry among Coloradans about the rising costs of housing – a reminder that concerns about housing affordability and stability are not limited to those currently experiencing homelessness.

Through Pulse: The Colorado Health Foundation Poll, we seek to hear from all Coloradans, even population groups that make up only a small percentage of residents of our state overall. That’s why we oversample African American/Black adults, Asian Americans and Pacific Islanders, and Native Americans. Importantly, because Hispanic Coloradans make up more than 20% of the state’s total population, we are able to achieve a large number of interviews without oversampling.

Here’s an example of how oversampling works: Black people make up around 4% of Colorado’s population; in a purely representative sample, that means they’d make up roughly 4% of all Pulse respondents. Given our commitment to improving the health of communities of color, we interviewed more Black Coloradans than just 4% of our sample so that we’d better understand their experiences, priorities and opinions.

It can be confusing to understand how oversamples like this don’t skew a poll’s overall results away from being representative of Colorado’s population as a whole. But they don’t – and that’s because of an additional step to the analysis process called “weighting.” According to the American Association for Public Opinion Research, “Weighting adjusts the poll data in an attempt to ensure that the sample more accurately reflects the characteristics of the population from which it was drawn. Weighting is used to adjust the relative contribution of the respondents, but it does not involve any changes to the actual answers to survey questions.”

In short, we do more interviews with populations that are mission-centric for us so that we’re better able to understand their opinions when we look at how they responded. This allows us to have greater confidence that we’re accurately reflecting the views of these groups of Coloradans when we report data about their opinions and experiences as a group. Then, when we’re reporting data about Coloradans overall, the weighting process balances everything out so that the overall numbers aren’t skewed by our oversamples.

Pulse: The Colorado Health Foundation Poll asks participants if they were born inside of or outside of the United States. The full results show how those two sub-groups compare in their answers to each question. The poll does not ask about documentation status for people not born in the U.S. In our current political environment, immigrants – both documented and undocumented – may live in fear of deportation or other consequences. Asking about immigration status could increase that fear and likely make many immigrants decline to participate in the poll. While we would love to know the priorities and experiences of undocumented immigrants living in Colorado, we do not want to create additional fear by asking about immigration status in our poll. We are challenging ourselves to listen to and explore the perspectives of people who are undocumented in other ways.

A poll the size of Pulse allows for a number of geographic breakdowns. You can see the data divided by region: Eastern Plains, Colorado Springs, Pueblo, Larimer and Weld, Denver Metro Area, and Western Slope. In addition to regions, county-specific data is available for any county where we gathered more than 100 responses. Those counties include: Adams, Arapahoe, Boulder, Denver, El Paso, Jefferson, Larimer, Pueblo, and Weld.

When conducting a poll, we only talk to a sample of the population, and while the sample represents the population, we’d get a truer result if we interviewed every single person in the population. The margin of error describes how close we can reasonably expect a survey result to be to the true population value if we were able to hear from every single person. A margin of error of plus or minus 3 percentage points at the 95% confidence level means that if we fielded the same survey 100 times, we would expect the result to be within 3 percentage points of the true population value 95 of those times.

In a poll, the most commonly cited margin of error applies to the entire sample that includes all respondents. When we look at sub-groups within the population – such as Hispanic/Latino adults living in Colorado – the margin of error gets bigger because the sample size gets smaller. While there are a number of factors that influence the margin of error, the predominant factor is sample size – the bigger it is (i.e., the more people you hear from), the smaller the margin of error (i.e., the closer you’ll be to representing the results if you talked to every single person).

While we aim to maintain consistency, there are instances where we may include additional questions in the online survey to prevent phone surveys from becoming overly lengthy. Typically, these added questions are open-ended and may require more context. We strive to maintain a balanced demographic response online.

Pulse: The Colorado Health Foundation Poll doesn’t aim to predict the results of any election. Interviews are conducted with Colorado adults, not just voters, because our goal is to know more about the experiences, concerns and priorities of all Coloradans to help us bring health closer in reach.

We understand that the experiences of recent polling trying to predict election outcomes and failing to do so has caused a lot of people to feel skeptical. The American Association for Public Opinion Research (AAPOR) has evaluated electoral polling from 2016 and 2020 to advise pollsters on how to avoid errors in the future. You can learn more about that work here.

Pulse: The Colorado Health Foundation Poll takes a number of steps to ensure that we’re accurately reflecting the opinions of Coloradans. First, Pulse is a multi-modal survey that reaches and interviews respondents on landlines, via texts and calls to cell phones, and through the mail. Second, Pulse is in the field with interviews for a month, allowing more time to get a representative sample of Coloradans to participate. Third, we interview respondents in both English and Spanish so the poll is more accessible. Finally, we ask Coloradans about the issues that are impacting their lives – not partisan politics – so they can trust the questions we’re asking and are more willing to participate.

To be sure, one of the most significant challenges facing the polling industry is the reality that people often don’t answer their phones. Pulse seeks to mitigate this challenge by contacting Coloradans via multiple modes. We send emails, text messages and postcards – in addition to calling – in an effort to reach a representative sample of our state.

Pulse: The Colorado Health Foundation Poll is committed to transparency; that’s why we release everything – every question we asked and all the results collected. The poll is also conducted as a partnership with both Democratic and Republican pollsters, and we work hard to make sure our questions are presented in a balanced way. The poll has no predetermined outcome, and the data speaks for itself.

Respondents were able to complete the poll in two languages: English and Spanish. According to U.S. Census Bureau data, English is the most spoken language in the state, and Spanish is the second most spoken language. We conduct the poll in these two languages to ensure we reach a representative sample of adults in Colorado. The results of the poll are also available in both English and Spanish.

Although our sample is representative, we know that we aren’t able to include Coloradans who don’t speak English or Spanish. German and Chinese (including Mandarin, Cantonese) are the next two most-spoken languages in the state – each with fewer than 24,000 speakers. Because we want to ensure that Pulse reflects the priorities of all Coloradans, we are considering what additional languages may need to be integrated into the various ways we listen to Coloradans.