It is human nature to want to give positive feedback and to avoid the negative, and the same principle has carried over to cause grade inflation in many classrooms. Nobody wants to give a D or an F or to risk the push-back that comes when having to justify a less than stellar ranking.
The editors of the famous Chicken Soup for the Soul series had guest readers who rated each of the submissions before they were included in a book. To accommodate for some of the poor ranking hesitation, they changed the scale. Readers could rate the story as a 7, 8, 9 or 10. Somehow giving an entry a "7" felt much better than giving it a "1" even though it accomplished the same end. And by eliminating a middle score, it forced reviewers to pick a side and make some sort of explicit judgment as to the merit of the piece.
Think about your rating scale and what barriers inherently come along with it. Can you modify your language to make the distinctions more palatable? Is there a way to shrink the options -- or to expand them -- to gain the measurements you are seeking? How can you create a ranking that makes the reviewers comfortable enough to place priority on some but not on others? There is no magic number.
Reported by Laura Brown in How to Write Anything