|
|
Post by lessthanbread on Feb 18, 2020 14:20:56 GMT -5
I'd be curious to know how a player only would go about judging a course for a contest. But Mcconnell makes a good point that you really want expert opinion from both sides and Wes makes another good point that many of the top designers are also excellent players so it really is the fact that it's tough to get the best judging panel because they best judges available are in the contest. And I disagree there is judging bias based on the fact that it's designers judging other designers.. that's how any contest works, subject matter experts are used to judge the subject they are experts in and they don't care who the contestants are I am a player only, and I am probably the most senior judge having judged practically every official contest... Now that's dedication.. So with that, how do you go about judging courses for contests? I'm really just curious on the perspective. |
|
Deleted
Deleted Member
Posts: 0
|
Post by Deleted on Feb 18, 2020 14:56:12 GMT -5
You know that some of the best designers are also really good players too, right? It is not like you choose one side of the game to play or the other. Indeed, and there is a limit in how many volunteers you get anyway (and so it is important to not shun players away from volunteering - as I said, from what I read; I do not feel any pull towards volunteering, rather a push from not). I would never recommend to set criteria on how good a player you are for judging. The best is to have several players on different skill levels to judge, because a good course for a very good player, may not be that for a less skilled player and vice versa. Anyway, this was just a chime in from the outside with perhaps some new glasses and perspective. |
|
|
|
Post by jwtexan on Feb 18, 2020 15:04:36 GMT -5
No, I completely appreciate the perspective. I don't think there should be any kind of "test" or that the judging pool should lean too heavily towards players or designers. I am incredibly average at both but enjoyed my time judging even if it lead to disagreements.
|
|
|
|
Post by b101 on Feb 18, 2020 15:04:38 GMT -5
You know that some of the best designers are also really good players too, right? It is not like you choose one side of the game to play or the other. Indeed, and there is a limit in how many volunteers you get anyway (and so it is important to not shun players away from volunteering - as I said, from what I read; I do not feel any pull towards volunteering, rather a push from not). I would never recommend to set criteria on how good a player you are for judging. The best is to have several players on different skill levels to judge, because a good course for a very good player, may not be that for a less skilled player and vice versa.Anyway, this was just a chime in from the outside with perhaps some new glasses and perspective. See, this is where it gets dicey in my opinion. Because there was no difficulty level on this contest, so why should difficulty really matter? For example, if you play Beginner clubs on Kaiuma from the back tees, well, you're gonna be unable to carry any hazards and will hate every second of it. It's designed to play at around KF/CC-A level giving an advantage to players using Master clubs off the tips. Likewise, if a PGA guy shoots -17 on it, great - it should have rewarded his good shots. The difficulty is not the point of this contest - we have two others for that. |
|
|
|
Post by cephyn on Feb 18, 2020 15:11:16 GMT -5
I'd be curious to know how a player only would go about judging a course for a contest. But Mcconnell makes a good point that you really want expert opinion from both sides and Wes makes another good point that many of the top designers are also excellent players so it really is the fact that it's tough to get the best judging panel because they best judges available are in the contest. And I disagree there is judging bias based on the fact that it's designers judging other designers.. that's how any contest works, subject matter experts are used to judge the subject they are experts in and they don't care who the contestants are a player only would be a great judge of aesthetics and playability - who else to better understand risk/reward, difficulty (as in, overly difficult, unfair difficulty, etc) and options for play? the best players should be able to judge those things quite well. |
|
Deleted
Deleted Member
Posts: 0
|
Post by Deleted on Feb 18, 2020 15:13:37 GMT -5
Indeed, and there is a limit in how many volunteers you get anyway (and so it is important to not shun players away from volunteering - as I said, from what I read; I do not feel any pull towards volunteering, rather a push from not). I would never recommend to set criteria on how good a player you are for judging. The best is to have several players on different skill levels to judge, because a good course for a very good player, may not be that for a less skilled player and vice versa.Anyway, this was just a chime in from the outside with perhaps some new glasses and perspective. See, this is where it gets dicey in my opinion. Because there was no difficulty level on this contest, so why should difficulty really matter? For example, if you play Beginner clubs on Kaiuma from the back tees, well, you're gonna be unable to carry any hazards and will hate every second of it. It's designed to play at around KF/CC-A level giving an advantage to players using Master clubs off the tips. Likewise, if a PGA guy shoots -17 on it, great - it should have rewarded his good shots. The difficulty is not the point of this contest - we have two others for that. The best courses are good for all kind of players. The best courses challenges and is fun for the bogey player, the par player and the birdie player. Is it difficult to please all? Oh yes. But that is what you as a designer are charged to accomplish  |
|
|
|
Post by cephyn on Feb 18, 2020 15:18:39 GMT -5
I have spent the better part of three years helping to build this design community to a great place for EVERYONE, whether rookie designer or experienced, and appreciating the player-only point of view. My contributions to teaching newer designers on my streams and doing tutorials for you guys was genuine. The fact that I’m hearing the word “elitist” thrown around behind my back cuts me to my core, and demonstrates that my willingness to help and my service as the TGCTours staff member overseeing official contests over the last three years was completely misunderstood. If you think I’m being a sore loser because I didn’t WIN this contest, you are simply wrong, and totally missing the point. I think too highly of this field to ever assume a victory. To Reeb and Justin: thank you for stepping up to run this contest, I appreciate you guys. To the judges: thank you for dedicating the requisite time to play these courses, I know it isn’t easy. To group E: I apologize if my disappointment overshadowed your joy, that was not fair of me, good luck in the next rounds. With that being said, you won’t hear anything further from me on this, as I’ll be stepping away for a while. Being so misunderstood, having my character questioned, and being secretly disparaged is just too much to take right now. May the best course win! This just shows me - the system is broken. Period. <shrug> I hope TGCT staff and admins take a step back and understand the legitimacy of these contests is completely trashed. Going forward, there may need to be greater transparency from the judges (#1) and definitely some sort of agreement as to who the judges are prior to the contest, from the contestants. Something. More accountability is needed. I think having a "how to judge" tutorial will be really important, but it risks groupthink if there's just one accepted way to judge. I know the judges are all volunteers (again, I was one last WCOD) - but I really think there needs to be more accountability around the judging. When a large number of the community sees the results and just goes, wtf....i dunno. something's wrong and broken. The judges need to be respected by the competitors. the competitors need to be respected by the judges. if we don't have that, then this all becomes a sham. |
|
|
|
Post by catcherman22 on Feb 18, 2020 15:25:14 GMT -5
I am a player only, and I am probably the most senior judge having judged practically every official contest... Now that's dedication.. So with that, how do you go about judging courses for contests? I'm really just curious on the perspective. I think it was pretty much stated below.. but mainly on the play ability aspect... also as a scheduler I look for courses that would be good fits on tour... mostly courses that would be successful on any tour... but just fitting on a tour in general is good.... But the main thing is using the rubric provided to us... this one was pretty specific in what we were looking for. |
|
|
|
Post by b101 on Feb 18, 2020 15:27:44 GMT -5
I'd be curious to know how a player only would go about judging a course for a contest. But Mcconnell makes a good point that you really want expert opinion from both sides and Wes makes another good point that many of the top designers are also excellent players so it really is the fact that it's tough to get the best judging panel because they best judges available are in the contest. And I disagree there is judging bias based on the fact that it's designers judging other designers.. that's how any contest works, subject matter experts are used to judge the subject they are experts in and they don't care who the contestants are a player only would be a great judge of aesthetics and playability - who else to better understand risk/reward, difficulty (as in, overly difficult, unfair difficulty, etc) and options for play? the best players should be able to judge those things quite well. Ehh. Look in a PGA thread and you might change this opinion. Some do, some don’t. |
|
|
|
Post by Crazycanuck1985 on Feb 18, 2020 16:13:00 GMT -5
Ok, here's my thoughts from another designer probably being called "elitist" as well.
I honestly don't care that my course didn't make it through, it just sucks on the how. In the matchup that made or broke my chances, one judge had basically had to flip a coin because he thought our courses were so close (I can live that), and the other judge...I obviously have a few questions for.
Anyway, that being said, a huge thank you to Reeb and the judges. I apologize if I was rude or passive aggressive in any way, but I was frustrated last night and this morning. You guys worked hard and gave us feedback as best you could.
That being said, I think judging and contests in general may need to be tweaked in the future. I think the big issue here was lack of experience from some judges and trying to give feedback to designers with (sometimes) hundreds more hours of experience than they have with design and the game in general, makes it a daunting task for judges.
Also, as said countless times before, these contests are almost entirely subjective now. Sure, we try to take into account lots of things, but if judges aren't weighting things equally, why bother? Why bother even having a rubric? Sometimes I wonder.
So I propose 2 different directions we can take from here on out.
1. We can continue to move ahead as is for the most part, but maybe have 3 or 4 more "experienced" designers or judges to oversee the judging. In other words, if they see something that is WAY WAY off (Hallstatt, cough cough) they could add in their two cents and maybe have judges re-evaluate with a fresh set of eyes. Now I can see people thinking these experienced judges are "influencing" the competition, blah,blah, but having a few "anchors" (most of us play every course in the competition anyways) might help minimize huge outliers. Again, I can see problems with this for a number or reasons. Or we can keep judging the same, but ensure we have more experienced judges. Maybe those that have entered every single contest the past few years, can take one off and volunteer to judge. I don't do many contests, but I haven't really judged, so I need to step up myself in that regard.
2. We could back off the gas and make the judging a bit more of a casual affair. We could just have a group of people (could be restricted to just players, designers, everyone in the community, fellow competitors), and just have them vote on the course or courses they like the best. Yes, there won't really be any feedback, but for those experienced designers, do you really need feedback at this point? For many of us, it really comes down to personal taste. Yes, this sucks for newer designers who might get very few votes with no reason as to why, so that is a definite downside. On the upside, this could open up design comps with lots of different fun formats....and isn't this what this is all about? Fun?
Anyway, there is no solution and neither of these options aren't without their flaws. We've been running design comps for a few years now and this is the first time where I have seen a number of truly head-scratching decisions where I might think a tweak is in order.
TLDR: Let's tweak stuff....but I'm not really sure how.
|
|
|
|
Post by lessthanbread on Feb 18, 2020 16:36:53 GMT -5
If contests have been judged better in the past, can we get to the root of why this contest was different? Typically you see the quality of a product start low and increase as lessons are learned and kinks ironed out, so, what happened in this instance that made the quality go down?
(These are all just questions to consider even if the answer is obvious or unrelated to the issue. I do this for a living)
Was the standard process followed for selecting judges?
Does a standard process exist for selecting judges?
Do we understand the qualities a good judge based on positive experiences in the past?
Did the selected judges follow the rubric *DRINK* given to them for judging?
Does the judging rubric *DRINK* need to be modified?
Do we think this problem will happen again or is this likely a one time occurrence?
Is there a simpler solution to the problem we already know will work instead of going through all these questions looking for solutions?
These are not the types of questions for the members of the community to respond to here... these are just examples of questions for the leaders of this community to consider if they are going to put their heads together on this matter
|
|
mayday_golf83
TGCT Design Competition Directors
Posts: 2,279 
TGCT Name: Jeremy Mayo
Tour: Elite
|
Post by mayday_golf83 on Feb 18, 2020 18:10:54 GMT -5
OK, been following along from work most of the day, but haven't had a lot of time to devote to chiming in. I've got about hour now, which may or may not be enough time, but here goes.
The biggest "flaw" so to speak in WCoD is that because there are so few judges/contest in the group stage, it lends itself to have an outlier have a greater chance of affecting an outcome. It's luck of the draw and if you happen to draw a judge that does not like your course for whatever reason -- right, wrong or indifferent -- you're almost assured of losing two points, if not three in your group. That's just simple math.
There are ways you could go about changing this -- larger pools, eliminating head-to-head in group stage, more judges/group -- and I'm sure these will be discussed going forward, but that's the nature of things as we have them right now. I also think that this format is harder to win now than ever. Last year's WCoD was collectively the strongest collection of courses we have ever had in a contest, until this year's WCoD that surpassed it. I felt, out of the 32 courses, there were at least 25 that were deserving of advancing out of group stage. When there are only 16 spots, that math doesn't work and there are going to be great courses hitting the showers early. When I look back at contests two, three years ago, I felt it was a bit easier to sort the wheat from the chaff. While there were still some surprising upsets, for the most part, the best courses ended up winning.
I think, overall, the biggest thing that's needed is an understanding of where everyone is coming from and some mutual respect among all parties. I've been on both sides of this. I've had times where I've felt a certain judge(s) has/have not assessed my course correctly. And it's natural to feel like that when you've seen 100s of hours of work go up in smoke in a blink of an eye. I've also had people question the assessments I've made as a judge after putting in a similar amount of hours dissecting the courses at hand. When you're on the receiving end of a perceived slight, it's natural to get a little ornery. But, the key is to try to take things with a grain of salt and to try to understand where the other party is coming from. Designers put a full faith effort into their courses and expect a full faith effort back from the judges -- and I do believe 99.99% of the judges do the best they can to deliver the result they feel is correct. Is that always the popular result, no? Can guys disagree with it, yes? But it makes it so much more tolerable when you're at least able to hear the other side out.
To bring up the Hallstatt-Mulligan matchup, it was not so much that Judge 1 picked Mulligan Quarry (not what I would have chosen, but to each his own), it's there were, initially, no comments to back it up. Again goes back to that full faith-full faith thing. The rationale is easier to digest when you get a glimpse into the thought process behind it.
Judges don't have it easy, either. There's a lot of pressure in those war rooms and, to Reeb's point earlier, his observation was that some of the newer judges may have been trying "too" hard to get it right. I get it, I was a first-time judge once and I did go above and beyond because I figured I may get questioned by some of the more experienced judges or contestants who did not agree with my assessment, and I wanted to make sure I had hard evidence to back up my assertions.
I know, when I was a first-time judge I would have LOVED to have a video from someone like Reeb dissecting what he's looking for in a course when judging. Now would that make me a Reeb clone when judging? No. While I trust his eye, he's got ways of viewing and doing things that differ from me. Neither's more right or wrong than the other, just two different paths to reach a solution. No matter how "standardized" you try to make the judging, there's always going to be at least some degree of subjectivity and personal preference.
Now, to those who claim this design community is broken because of this, hogwash. It's like this, judgement day will ALWAYS lead to hurt feelings. It's inevitable. Designers and judges a like are proud of their work and, things tend to get a little testy right afterward. It's the whole red mist of war thing. Privately, and some of my friends on here can attest, I was SEETHING when I found out Necedah didn't make the top 5 of the CC. But what good was going on here and throwing a royal conniption fit going to do? Nothing, but make me look like an ass. I learned of what the perceived flaws in the course were. Some I could understand, some I didn't, I processed and moved on.
I think that's what we need to do here. We haven't built the perfect mouse trap yet in terms of a design competition, probably never will. But we must learn from our experiences and grow from them. If we can do that -- and do so in a respectful manner -- then this community will continue to thrive.
TLDR: We're not perfect. Designing is hard. Judging is hard. Being put down for one or the other is hard too. Let's just take a breath, respect the work everyone's trying to do and we'll be all good.
|
|
|
|
Post by warhawk137 on Feb 18, 2020 18:18:29 GMT -5
If contests have been judged better in the past, can we get to the root of why this contest was different? Typically you see the quality of a product start low and increase as lessons are learned and kinks ironed out, so, what happened in this instance that made the quality go down?
Truth be told while this one is probably on the more contentious end I wouldn't describe it as exceptional. There's a long history of blow-ups in these things.
|
|
TCRBrad
Amateur Golfer
Midwest Ope-n Champion
Posts: 180
TGCT Name: TCRBrad
Tour: Challenge Circuit
|
Post by TCRBrad on Feb 18, 2020 18:34:22 GMT -5
This might be too on the nose...but why don't we do what TGC does with "rivalries" in the career mode and go head to head with specific criteria. Give the nod to one course over an other based on 5-7 "things" If you want to keep the ties, then use an even number of categories. Otherwise, use an odd number. Pick something fairly "tangible" when choosing categories (I'll use two courses from our Group...mine and Hallstatt) ---------- Immersion - Hallstatt (much better backstory and good lord look at the scenery in the background. Scawt was good, Hallstatt was better) Theme - Hallstatt (took much more advantage of the Swiss theme than Scawt did of the Links theme) Playability for all - Scawt (easier for players of all club types, Hallstatt is a bear for pro clubs and less) Cleanliness - Hallstatt (planting, sculpting, texture of fairways and greens WAY cleaner. Scawt was a bit more rugged) Fun - Scawt (close call, but Hallstatt just flat out kicks your teeth in compared to Scawt which has more "qwirk") Winner - Hallstatt over Scawt (3 - 2) ---------- Just a thought. I'm still "kinda" new here. EDIT: Thoughts Crazycanuck1985 lessthanbread mayday_golf83 |
|
|
|
Post by lessthanbread on Feb 18, 2020 18:55:42 GMT -5
I thought that’s kind of what judges are supposed to do already, focus on comparing different categories. Might be wrong though. I’ve never looked close at the judging guide lines
|
|
