Weird Bug With Ratings

Hi, I have stories where the ratings now appear like the following. Even though the cum rating is higher than the hot rating, it shows without the white aura while the hot has a white aura.

25

Thanks for notifying.

Actually this makes a bit of sense in the new way the icons are calculated.

The order of the icons is determined by the overall average of ratings for that story.

But the display (normal, silver or golden) is controlled by the fact, whether the rating is in the top 10% or top 5% of all stories for that rating.

So in this case, ā€˜cummingā€™ was actually rated higher than ā€˜hotā€™ but its hot rating put it into the best 10% of all hot ratings, while the ā€˜cummingā€™ rating just isnā€™t in the top 10% of all cumming ratings.

Does that make sense?

:smiling_face:

2 Likes

Yeah, it makes sense. Did you by any chance change the way the ratings are calculated or the very/exceptional/outstanding categories? Some of my stories ratings changed all of a sudden. Nothing big, but I was curious. It may also be new ratings.

Yes, Iā€™ve changed it a couple of days ago, as Iā€™ve also noted in the Site Update thread :slight_smile:

Your ratings as an author are astonishing by the way. Just wanted to say. Hats off!

Thanks! I had fans on other sites that I recommended come here to see the new stories (I havenā€™t put the new ones consistently on other sites). I also see that after 17 years of inactivity, new stories led to people rereading my older stories here.

For some reason, I missed your post on Site Updates.

Thanks for bringing new users to GSS! Thatā€™s always very appreciated!

Hi Martin,

Iā€™m sure youā€™re not looking for something more to do, but just in case you find the following suggestions interesting, here you go:

  • I reiterate my idea that instead of averaging all ratings, you could instead do ((average of ratings by users with an account) + (average of ratings by anonymous submission)) / 2. This would give more weight to ratings by users with an account and could be used to sell the idea to get one. Most of my ratings would go down with such a system, but I donā€™t care. (The formula might need to change based on special cases, but close enough)

  • Since you decided to use the deviation from the population mean instead of the actual rating average to determine outstandingly, exceptionally and very, why not use that everywhere? Normalize the results and show the normalized rating. You could still show the raw ratings in the popup. This would also solve the bug I mentioned if you sort the icons by normalized rating instead of raw ratings.

Another idea for ratings is to do it like they do in some sports competitions and the like: drop an equal number of the highest and lowest ratings, then only work on the remainder of the data set. This has the tendency to discard anomalous low and high ratings, leaving the average (or whatever) being closer to most peopleā€™s votes.

First of all, donā€™t be afraid to be imposing with your suggestions. Iā€™m always happy if people get involved with what Iā€™m doing with the site, it means that itā€™s not just in vain. That doesnā€™t mean I have to agree with everything, but I want to hear out and discuss everything!

  • Iā€™ve already confirmed that I want to give ratings made by logged on users more weight. But I also have to make it more attractive to use accounts, since otherwise those few votes coming from logged on users still wouldnā€™t make much of a difference. As Iā€™ve written in the other posting, I have several ideas how to make accounts more attractive.
    One of them is to reward people who rate a lot with achievements, which would in turn increase the weight of their ratings
  • The normal user ONLY sees those icons, so for them it wouldnā€™t make much of a difference. Only the author himself gets to see the numbers. Still, I could add a value to the stats popup to show in which percentile a rating is. Would that satisfy your request? This would still only be shown to the author, though (and the all-mighty admins of course, which is quite a boon in this case :slight_smile: ).
    The problem with the order of the icons could be easily adapted, itā€™s just something that didnā€™t seem to be that important to me. However, lately I see more stories where the order is confusing, starting with a big silver icon and followed by two smaller golden onesā€¦

@RobinHood70
Thatā€™s also an idea I was playing around with, especially since it was frustrating the hell out of me, that the ratings of my latest, otherwise very appreciated story was ruined by two very bad votes.

But Iā€™m afraid it would feel like a cheat. Since thereā€™s usually a much higher number of high ratings and only few low ones, stripping a number of ratings from both ends feel a bit like I want to silence the harsh critics. If someone really doesnā€™t like the story, his vote also DOES matter, even though it hurts and frustrates the hell out of the author.

Any system that gives more weight to users will satisfy me. If there is a way to see the percentile and you sort the icons by percentile, thatā€™ll be totally fine.

But I do like @RobinHood70ā€™s idea a lot, and I second it. My arguments are:

  • If you get 1,5,5,5,5,5,5,5,5,5 as ratings, do you really deserve a 4.6? Yes, giving a 5 in this case even though one person out of 10 hated your story is not perfect either, but itā€™s a hell of a lot more fair.
  • Dropping 1 or 2 ratings at both ends does not mean that the people who gave the dropped ratings were silenced. If there are 3 people who vote 1, there is still a 1 that is counted, whereas if the person whoā€™s rating was dropped had not voted, that third one would not have been counted because there would have been only 2 ones. You see? It just means that it takes more votes for harsh reviews to be counted, because they have such an impact on the total score. If you get 1,1,1,1,1,5,5,5,5,5, you still get a 3 average, whether you drop 0,1,2 or 3 ones and fives. So all those who voted 1s contributed to the final score. They were not silenced.

Iā€™ll consider it :slight_smile:

Thanks Martin! I think the new popup has alerted a lot of writers to the frequent presence of a single 1 that has a lot of impact.

But in any case, none of these things discussed here are totally essential, the system works well as is.

1 Like

I think itā€™s too easy to get hung up on the ratings and I donā€™t think they matter too much. But itā€™s still only the author that sees them, isnā€™t it? In that case, I think showing the mean would be a really good idea.

For the ratings themselves, instead of making it users, you could tie it to the comments. If a user rates and comments, itā€™s weighted higher. Or you could ask a user via a pop up that gives ratings under 1 to leave a comment explaining their rating.

Nice ideas, but one think to consider is: I have to be able to fit the calculation into an SQL statement as well! I know thatā€™s a bit of techie for most of you, but it is a challenge, to write that and to get it to perform well.

Iā€™ve just added the percentile calculation to the SQL view, which was easy enough. To use weighted ratings could make it REALLY difficult though. I want to let the DB do the calculations, though.

Whatever, if someone is curious and understands a bit of SQL, hereā€™s the current code (a materialized view) to calculate the ratings statistics for all stories (only for one of the rating types, the rest is joined using UNION)

CREATE MATERIALIZED VIEW public.rating_statistics
AS SELECT concat(s.id, '_ratingCum') AS id,
    s.id AS story_id,
    s.name,
    'ratingCum'::text AS rating_type,
    date_part('month'::text, s.publish_date) AS publish_month,
    date_part('year'::text, s.publish_date) AS publish_year,
    count(NULLIF(r.rating_cum, '-1'::integer)) AS num_ratings,
    avg(NULLIF(r.rating_cum, '-1'::integer)) AS average,
    row_number() OVER (ORDER BY (avg(NULLIF(r.rating_cum, '-1'::integer))) DESC) AS rank_overall,
    row_number() OVER (PARTITION BY (date_part('month'::text, s.publish_date)), (date_part('year'::text, s.publish_date)) ORDER BY (avg(NULLIF(r.rating_cum, '-1'::integer))) DESC) AS rank_month,
    row_number() OVER (PARTITION BY (date_part('year'::text, s.publish_date)) ORDER BY (avg(NULLIF(r.rating_cum, '-1'::integer))) DESC) AS rank_year,
    percent_rank() OVER (ORDER BY (avg(NULLIF(r.rating_cum, '-1'::integer))) DESC) AS percent_overall
   FROM grails_story s
     JOIN rating r ON s.id = r.story_id
  GROUP BY s.id, s.name
 HAVING count(NULLIF(r.rating_cum, '-1'::integer)) >= 10
--- 
WITH DATA;

So letā€™s see what I can do, but no promises so far!

All right. Youā€™ve just ruined this Saturday for me ā€¦

No, just kidding. I started to think about how to do it and a couple of hours later, I had the following changes implemented:

  • Thereā€™s a new ā€œweighted averageā€, the only difference currently is that the worst and the best votes are stripped.
  • That also means that you now need at least 12 voted to get a rating icon
  • The database directly calculates the percentile. It is also shown together with the weighted average in the pop up. Note that the value listed in the table is still the normal average.
  • The icons are sorted properly using the percentile. The thresholds remain at 1.25% for gold, / 5% for silver / 20% for normal icons.

Iā€™d like to ask you to test this, because itā€™s a complicated and a lot of changes in the background.

And another remark: Since ALL stories now got a better weighted average, you might not even see any improvement for your rankings or percentile at all. As a matter of fact, my own story, ā€œCommunity Serviceā€ basically crashed as a result, it dropped into mediocrity.

Thatā€™s so my life. I try to accommodate someoneā€™s request and get screwed myself in the processā€¦

:stuck_out_tongue_winking_eye:

1 Like

This looks fantastic, Martin. Thanks for your hard work! It doesnā€™t change my ratings much, maybe dropped them a little for the reason you mentioned (all others got bumped up), in my case itā€™s because my new stories got lots of ratings (100+), but for people with less votes it will make a significant difference (prevent them from sinking from just one bad vote). Also, the order of the icons seem to work perfectly. Sorry about your Saturday, but you made the Saturday of all other authors better, I think.

Very nice!

Iā€™m impressed you got all that into a SQL query! If you find it starts lagging, you may want to either move the calculations to the front end, if you can, or start using some stored procedures. I canā€™t speak for Oracle (I think thatā€™s what youā€™re using, based on syntax), but in most database software, using a stored procedure can speed things up for complex stuff, simply because the database will have the chance to optimize it in advance rather than figuring it out on the fly as each query is issued.

Thanks for the advice, @RobinHood70 :slight_smile:

Iā€™m using a materialized view, so the calculation is only done once (until I explicitly recalculate it, which is done after 4 hours). And I love to do the stuff on the DB, itā€™s simply MUCH faster than front-end code. And Iā€™m really surprised how fast PostgreSQL is.

Iā€™m simply such a nerd for enjoying this stuff, ainā€™t I?

@Mafisto Glad youā€™re happy with it!

2 Likes