Announcement

Collapse
No announcement yet.

Ranking Points

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Ranking Points

    Does the scoring system of qualifying points and ranking points seem broken to anybody else? I like the qualifying points because it makes sense to have team rankings primarily based on match wins and losses. However, I think that a better tie-breaker is needed than ranking points. I understand the purpose behind ranking points: they are supposed to show the strength of the competition you beat. However, they have a major flaw: they don't necessarily reflect on the strength of your team. Just because the teams that you beat were low scorers, that doesn't mean that you can't also beat higher scoring teams. Furthermore, the amount of scoring that occurs in a match is not the best indicator of how good the teams involved in that match were. I believe that teams who win low-scoring and high-defense matches can be just as good as teams who win high-scoring and low-defense matches.

    But, enough about my views. What do other people think? Do you like the scoring system that FTC uses or would you like to see some kind of change?
    Burning Lights Programming
    FLL Team 341 Brick Chick'N Boys - Programmer (2009-2010)
    FLL Team 263 Brainy Bricks - Programmer (2010-2011)
    FLL Team 5028 Fellowship of the Brick - Youth Mentor (2011-2012)
    FTC Team 6100 Chariots of Fire - Programmer (2012-2013)
    FTC Team 7468 Blue Chariots of Fire - Programmer (2013-2014)
    FTC Team 7468 Blue Chariots of Fire - Mentor/Coach (2014-2017)

  • #2
    There is no combination of metrics that can perfectly rank teams. If you look at FRC, though the event uses a similar ranking system to FTC, there are a variety of metrics that teams have invented to rank teams. A quick look through Chief Delphi will show a lot of different ways to measure the offensive and/or defensive efficacy of a team. As a scout, one should look at all of these metrics, and combine them in a way you see fit to judge teams.

    The purpose of Ranking Points (I believe) is to encourage even matches that do not involve simply crushing your opponent. In previous years, it was advisable to score for your opponent(if you knew you were going to win) to increase your own RPs.
    Max Bareiss

    FTC #248 Fatal Error (2009-2013)
    FRC #3142 Aperture (2009-2012)
    FRC #1302 Team Lionheart (2012-2013)
    ZRHS #89 Team K├╝hlschrank (2011-2013)
    ZRAC #40 Catcher in the Skye (2012)
    ISR 12: Umptysquatch 6
    Rowan University Baja SAE

    And mentoring for life.
    --
    11 seasons of FIRST in 6 years, as a student. Many more as a mentor.

    Comment


    • #3
      Originally posted by Skinkworks View Post
      In previous years, it was advisable to score for your opponent(if you knew you were going to win) to increase your own RPs.
      That may be, but it doesn't jive with the way the rules were set up last year; the only way you could score for your opponent was to push any of their rings that were loose on the floor onto the center floor goal. Furthermore, I don't see anything wrong with blow out matches. I think a match where you blow away your opponent says more about how good your team is than a match where you artificially inflate their score by scoring for them.

      The primary issue that I have with ranking points is that it doesn't give a specific idea of how many points you can score, unless you lose every match. Instead, it shows that over the course of the day, you have scored at least x number of points. You could have scored that amount exactly, or you could have scored twice as many. You simply don't know what a team is capable of doing by the Ranking Points number.
      Burning Lights Programming
      FLL Team 341 Brick Chick'N Boys - Programmer (2009-2010)
      FLL Team 263 Brainy Bricks - Programmer (2010-2011)
      FLL Team 5028 Fellowship of the Brick - Youth Mentor (2011-2012)
      FTC Team 6100 Chariots of Fire - Programmer (2012-2013)
      FTC Team 7468 Blue Chariots of Fire - Programmer (2013-2014)
      FTC Team 7468 Blue Chariots of Fire - Mentor/Coach (2014-2017)

      Comment


      • #4
        Originally posted by FTC Ringer View Post
        That may be, but it doesn't jive with the way the rules were set up last year
        The FTC game hasn't been set up this way in a number of years. I don't know why; could anyone from FIRST chime in?

        Originally posted by FTC Ringer View Post
        I think a match where you blow away your opponent says more about how good your team is than a match where you artificially inflate their score by scoring for them.
        Hmm, I would perk my ears up if I was a scout and heard the announcer say "now look at team xxxx scoring for their opponent!" It shows that you are so confident in gaining 2 QPs that you can focus on gaining as many RPs as possible.
        Max Bareiss

        FTC #248 Fatal Error (2009-2013)
        FRC #3142 Aperture (2009-2012)
        FRC #1302 Team Lionheart (2012-2013)
        ZRHS #89 Team K├╝hlschrank (2011-2013)
        ZRAC #40 Catcher in the Skye (2012)
        ISR 12: Umptysquatch 6
        Rowan University Baja SAE

        And mentoring for life.
        --
        11 seasons of FIRST in 6 years, as a student. Many more as a mentor.

        Comment


        • #5
          This year we can actually score for our opponents, though. But it'd be more risky, since it's illegal during the entire 30 seconds of end game, and the other team still has a chance to (double?) climb.
          Lead programmer for FTC team #6424, the 'Oly Cow.
          https://www.facebook.com/pages/The-O...60019847470634

          Comment


          • #6
            There are two words that are the mainstay of FTC that definitively explain why high RP's are so important (and should be sought after). This is also why the game rules have evolved and been honed over the years to honor the expertise and technological sophistication of the robots FTC teams are bringing to the field:

            Gracious Professionalism

            Beat the other alliance on your own merits, not by preventing them from accomplishing what they have spent an entire season trying to achieve. Last year's challenge allowed you to thwart your opponent by stealing their peg. It was a challenging strategic game that escaped far too many teams. I will say that this year's challenge is much more straight forward. I look forward to seeing how it plays out. GOOD LUCK TO ALL TEAMS!

            Teams with a high GPA (Gracious Professionalism Attitude) will typically earn the highest RP. Watch for it.

            Comment


            • #7
              [QUOTE=FTC5666;7753]Beat the other alliance on your own merits, not by preventing them from accomplishing what they have spent an entire season trying to achieve.QUOTE]

              With all due respect, I believe that defense is a perfectly legitimate strategy in FTC. Indeed, smart and well executed defense can go a long way towards winning shootout matches. Consider Game Manual Part 2 <G9>: "FTC Games are highly interactive and Robot-to-Robot contact and defensive Game play should be expected." (emphasis added) If the GDC warns teams that defensive gameplay should be expected, then that probably means they support it as a legitimate strategy. Now granted, they obviously don't want teams to play unfettered defense because of the trapping, pinning, and end-game rules. But on the whole, the way the rules are written restricts defense to reasonable levels, while still making it an allowed part of the game.

              As to ranking points, the number of ranking points you get is primarily determined by your opponents. Even if you score blocks for them, you can't do anything for their end-game score and autonomous score, which could be the majority of the points scored in many matches. If you balance their pendulum for them, but in the last 30 seconds they unbalance it, there is nothing you can do about that. Furthermore, you cannot raise their flag more them, and you most certainly cannot hang their robots for them. The main point is, the number of ranking points you get is usually determined by a random lottery of the teams you happen to face during the day. And as FTC continues to grow and the number of teams at each competition expands, the sample size of the teams that you face decreases. Ranking points simply involve too many variables that are outside of your control. I think a system involving the score of your alliance would be a better measure, which, while also having variables outside of your control, has the main variable of what YOU can do.
              Burning Lights Programming
              FLL Team 341 Brick Chick'N Boys - Programmer (2009-2010)
              FLL Team 263 Brainy Bricks - Programmer (2010-2011)
              FLL Team 5028 Fellowship of the Brick - Youth Mentor (2011-2012)
              FTC Team 6100 Chariots of Fire - Programmer (2012-2013)
              FTC Team 7468 Blue Chariots of Fire - Programmer (2013-2014)
              FTC Team 7468 Blue Chariots of Fire - Mentor/Coach (2014-2017)

              Comment


              • #8
                Originally posted by FTC Ringer View Post
                Even if you score blocks for them, you can't do anything for their end-game score and autonomous score, which could be the majority of the points scored in many matches. If you balance their pendulum for them, but in the last 30 seconds they unbalance it, there is nothing you can do about that. Furthermore, you cannot raise their flag for them, and you most certainly cannot hang their robots for them.
                Another point to bring up: It is completely possible that if an alliance is losing really badly (and has no chance of winning), they could intentionally "not score", by not raising the flag when they could, or not hanging, or unbalancing their pendulum. Which certainly is neither gracious nor professional, if the motivation to score points is so. I mean, you'd essentially be scoring points for the other team by continuing to score points for yourself.
                Last edited by Ernest314; 11-03-2013, 03:40 PM. Reason: grammar
                Lead programmer for FTC team #6424, the 'Oly Cow.
                https://www.facebook.com/pages/The-O...60019847470634

                Comment


                • #9
                  Originally posted by Ernest314 View Post
                  Another point to bring up: It is completely possible that if an alliance is losing really badly (and has no chance of winning), they could intentionally "not score", by not raising the flag when they could, or not hanging, or unbalancing their pendulum. Which certainly is neither gracious nor professional, if the motivation to score points is so. I mean, you'd essentially be scoring points for the other team by continuing to score points for yourself.
                  An excellent point. This could potentially come into play with teams that have sister-teams. Picture this situation: Sister-team A is fighting for a spot in the top four with Team X. Both of these teams have equal ranking points. Sister-team B is way down in the rankings, with no mathematical possibility that they could get in the top four. Sister-team B goes against Team X. About halfway through the match, it becomes clear that the alliance that Sister-team B is on cannot beat the alliance with Team X. So, sister-team B proceeds to do everything they can to keep their alliance's score low, thus lowering the ranking points of Team X and giving Sister-team A a better shot at a spot in the top four. This is most definitely neither gracious nor professional, but it is still a very real possibility.
                  Burning Lights Programming
                  FLL Team 341 Brick Chick'N Boys - Programmer (2009-2010)
                  FLL Team 263 Brainy Bricks - Programmer (2010-2011)
                  FLL Team 5028 Fellowship of the Brick - Youth Mentor (2011-2012)
                  FTC Team 6100 Chariots of Fire - Programmer (2012-2013)
                  FTC Team 7468 Blue Chariots of Fire - Programmer (2013-2014)
                  FTC Team 7468 Blue Chariots of Fire - Mentor/Coach (2014-2017)

                  Comment


                  • #10
                    been talking about this for years

                    Over the years ... I have been involved in many discussions about RP with many people.

                    The critical time where RP matters is figuring out which teams will become alliance captains and what order they will get to select alliance partners.

                    My personal opinion is that the losing alliance should get the lower score as RP, and the winning alliance should get the average of the higher and lower score. This gets a bit more twisted this season due to the way penalties work.

                    For a tie, both alliances should get the lower pre-penalized score as RP, which is exactly what happens.

                    When it is not a tie, there is a chance that the winning alliance might have a lower pre-penalized score than the losing alliance. The scoring system follows what the rules dictate, which is that all teams receive the losing alliance's pre-penalized score. In my opinion this would be better if all teams receive the lower pre-penalized score. Whatever.

                    A much better approach would be like this:
                    For a tie, all teams should get the lower pre-penalized score, which is the same as what happens now.
                    When it is not a tie, the losing alliance should get the lower pre-penalized score, and the winning alliance should get the average of the lower pre-penalized score and their own pre-penalized score. In the case where the winning alliance actually had the lower pre-penalized score, they would get the same RP as the losing alliance. In the case where the winning alliance had a higher pre-penalized score, they would get the average.

                    In cases where there are no penalties, the RP is much easier to understand.

                    My favorite way to illustrate the problem with the current RP approach is to imagine two robots. Robot A wins every match by a score of 20 to 10. Robot B wins every match by a score of 200 to 0. Assuming these are the only undefeated teams, at the end of the day Robot A will be ranked higher and will get first pick of alliance partners. Somehow this doesn't seem like the best approach. Ask anyone in the audience which is the stronger robot, and they will all point to the robot that scores consistently high all day long.

                    The current RP model only takes into consideration the strength/performance of the losing alliance.

                    In my opinion it would be better to also consider the strength/performance of the robot itself, meaning that any time a robot is on the winning alliance it would gain RP which averages the losing alliance and the winning alliance. Just saying.

                    At least this year it is easy to score for the other alliance, which mitigates the issue somewhat. Last year it was extremely difficult to score for the other alliance. This varies from year to year.

                    I think I understand the idea of why RP works the way it does ... but imagine how much better RP could be if a consistently high-scoring robot could gain some RP based partly on its own scoring ability and not just the scoring ability of random opponents.

                    Years ago during my first season of FTC I saw a robot shoot a few balls into the other color basket (Hot Shot). At the time it just seemed strange. Eventually I understood why they did it. Even now I still think that a game where it makes sense to score for the other side is a strange game.

                    Robotics is great. FTC is great. The RP calculation is good but could be better. In my opinion.

                    Comment


                    • #11
                      Strenght of Schedule

                      I think you're missing the point of ranking points. They are an attempt to factor in strength of schedule or how tough your opponents are. FTC is both an offensive and defensive game. I would argue that the team A that won all their matches by 20 to 10 had a much harder set of opponents than team B that had an easy time scoring 200 points. In all likelihood, team A and team B shared few common opponents. With only score to go by, I would assume that defense was teams A's primary strategy. I would assume that offense was team B's primary strategy. Why would you reward offense over defense in ranking points? Personally, I'd bet on team A defeating team B, absent any additional information.

                      I fully agree that the current ranking point system is a poor indicator of both strength of schedule and determining which team should be ranked higher in the event of a tie in qualifier points. I think that's part of the reason they have been abandoned by FRC for several years now.

                      No arguments with qualifier points. They don't weight one strategy over another.

                      A way to make ranking points truly reflect the strength of schedule for team A and team B would be to base it on the cumulative records of their opponents. Assume 5 matches and a total of 10 opponents per team. If team A's opponents had a cumulative record of 10 and 40, they'd get 10 RP. If team B's opponents had a cumulative record of 35 and 15, they'd get 15 RP. I think everyone would agree that team B had the tougher schedule in this scenario and should be ranked above team A.

                      This would put the focus back on winning matches rather than gaming the system by scoring for your opponent. I don't care if you adopt an offensive strategy, defensive strategy or are flexible based on you opponents. It would be about winning matches, not how. Winning matches gets you QP. Opponents winning matches gets you RP.

                      --Scott

                      Comment


                      • #12
                        Originally posted by FTC4140 View Post
                        A way to make ranking points truly reflect the strength of schedule for team A and team B would be to base it on the cumulative records of their opponents.
                        Hmm, interesting idea there. I hadn't thought of that, but I like it.
                        Burning Lights Programming
                        FLL Team 341 Brick Chick'N Boys - Programmer (2009-2010)
                        FLL Team 263 Brainy Bricks - Programmer (2010-2011)
                        FLL Team 5028 Fellowship of the Brick - Youth Mentor (2011-2012)
                        FTC Team 6100 Chariots of Fire - Programmer (2012-2013)
                        FTC Team 7468 Blue Chariots of Fire - Programmer (2013-2014)
                        FTC Team 7468 Blue Chariots of Fire - Mentor/Coach (2014-2017)

                        Comment


                        • #13
                          Given the limited/catalogued number of teams, it's actually feasible to have an Elo-like system to rank teams.

                          EDIT: Here are some Wikipedia articles (don't worry, they're pretty short):
                          http://en.wikipedia.org/wiki/Sports_rating_system
                          http://en.wikipedia.org/wiki/Chess_rating_systems

                          Pros of using a more complex rating system:
                          • more accurate; fairer
                          • tried-and-true

                          Cons of using a more complex rating system:
                          • (more) difficult for individual teams to calculate their own rankings
                          • may require implementation of a system-wide database, or could just be on a per-match basis
                          • may include arbitrary weightings, can't really be worse than the current system though, which also has arbitrary weightings.
                          Last edited by Ernest314; 11-08-2013, 10:18 PM. Reason: Expanded post with pros and cons, etc.
                          Lead programmer for FTC team #6424, the 'Oly Cow.
                          https://www.facebook.com/pages/The-O...60019847470634

                          Comment


                          • #14
                            Originally posted by Ernest314 View Post
                            Given the limited/catalogued number of teams, it's actually feasible to have an Elo-like system to rank teams.

                            EDIT: Here are some Wikipedia articles (don't worry, they're pretty short):
                            http://en.wikipedia.org/wiki/Sports_rating_system
                            http://en.wikipedia.org/wiki/Chess_rating_systems
                            If you were trying to come up with a system that replaced both QP and RP, I could see using one of these systems. But I don't thing that's the goal. QP is perfectly fine for the first order of determining the best team(s). Now all we need is a way to break the ties in QP. Cumulative win-loss records of each team's opponents provides this.

                            If you want to get more complicated, and attempt to factor beyond Alliance scores, you would need to factor in the win-loss record of your alliance partners. I haven't completely thought through this, as I'm not sure it's required. But I can see an argument that team A had alliance partners with winning records and team B had alliance partners with losing records. If both teams had the same QP and the same cumulative win-loss record, I'd argue that team B had the tougher challenge to overcome and should be ranked high than team A.

                            --Scott

                            Comment


                            • #15
                              These are all very interesting points.

                              Based on my own experience, most of the robots that play extremely aggressive defense do so because they lack the ability to score. It's a good thing we have the pinning penalty, or some matches would devolve into two minutes of pinning.

                              A certain amount of defense is expected, and even encouraged.

                              As far as I can tell, the Game Design Committee tries very hard to reward teams who build excellent robots, with part of the definition of an excellent robot being the ability to score based upon the game rules (which change each year).

                              It's difficult to build a tower. It's easy to knock it down.
                              It's difficult to put objects into containers. It's easy to knock them out.

                              See where I'm going?

                              The QP system is excellent. It rewards the winning alliance.

                              The RP system is decent.

                              I understand that it tries to capture the difficulty of the match based upon the losing alliance's pre-penalized score.

                              Let's be clear: The RP only matters at the end of the day, and only matters for just a few teams.

                              I don't think there is any benefit to switching to any sort of complex system used by other sports. Also the RP are only valid during the course of a single event, so there is no point in trying to drag them out across multiple events.

                              There are some teams/robots that score a large number of points in every match. Often this consistency persists regardless of alliance partner or opposing alliance members. The most common complaint I have heard (and I agree) is that such a team is misrepresented as weaker than some other teams.

                              In my classic scenario ... the team that always scores 200 points always scores 200 points, and the team that always scores 20 points always scores 20 points. When they finally play each other during the elimination round, the team that scores 200 points wins, because 200 points is more than 20 points. I just think that raw ability to score points should be rewarded in some way.

                              Comment

                              Working...
                              X