Models and Scores 101
- what do the ranges mean?
- what is "rank choice"?
from SV -- what is a model?
In short, a model is a way to tell us something about an individual on the file. This may be whether or not someone will vote (Vote Propensity), whether or not someone will volunteers (Activism), or if someone will support an issue.
Models take many different data points including vote history, census information, publicly available information, and data from other organizations contacts to create a score. The score is a number which results from an equation all the data points are put into. That score tells us how likely someone is to take action, support an issue, and so on. In many cases, each individual will have a score attached to their name that can be searched depending on the project your organization is working on.
Many of the model scores may be in rank order and won’t mean a probability of support or action. It will show where that person is compared to others in the database. Rank Order puts everyone in a lined based on their score in numeric order from lowest to highest. It is not a percentage but it does indicate how likely someone is to action in relation to other people scored on the file.
For example, the vote propensity model is scored from 0 to 100. You will see the score range in create a list to the right of the name and criteria boxes. If someone is scored as 80 on the vote propensity model, that does not mean they are 80% likely to vote. It means though that someone who is an 80 is more likely to vote than someone is scored and 79. That person who is 79 is more likely to vote than a 78, and so on.
Models are very good, but they are never perfect. Because these models are the result of an algorithm and are subject to statistical analysis that created them, there is always going to be a margin of error when using them. The general rule for most models will be the higher the score, the better. For instance, if you were to use the general activist model to find petition signers, and were to go door to door without a model for targeting, after talking to 10 people on a street, maybe 4 will slam the door on your face. Using a model will increase your odds of success. Instead of those 4 people slamming the door on you, maybe only 1 will. Using the models will put you in front of the best possible people to speak with and put in the best possible place to succeed, but, again, it’s never perfect.
Models are also as much about finding the right people to speak with as they are about excluding the people you don’t want to speak with. This targeting of voters will find the right people to include in your contact universe. Targeting is the act of selecting which voters to speak with. This can be by geography (precinct, city), race, age, gender, or by using the models. When using the models, or any other criteria for targeting, the right number will have certain guidelines on what scores to use, however your final contact list may also be determined by how many contacts your budget allows you to make. Targeting will help you find the right number.
When using models and especially when engaging in any voter engagement work, please consult with an attorney on the appropriateness of the targeting criteria used. Certain models carry rules that may make timing or usage inappropriate depending on the program you are planning to use them for.
Before you begin, here are some other notes for using the models in VAN:
- The naming scheme for the models in VAN is [Year]:[Creator/Owner]:[Name of Model]_[Version if applicable]
- Many models are re-validated each year. The notes do not change much from year to year in how they are created or expected to be used but the year may be updated.
- Some models are propensity or probability and some are known as rank order. A probability or propensity model should indicate an approximate probable likelihood of someone taking an action. If someone is an 80 on the voter propensity score, they should have an approximately 80% likelihood of voting. If the score is rank order, voters are put in a line from highest to lowest and the higher they score on the model, the more likely are to take action relative to other people in line. For instance, someone who scores an 80 on the General Activism model is more likely to volunteer than someone who is a 79 and so on. It does not mean 80% likely to volunteer.
- No model is perfect. If these are working properly, they should be very good.
- Some models have restrictions on when and how they can be used.