Purpose
In spring of 2020, a gymnastics judges’ survey was conducted to compare judging educational systems and mental processes across disciplines, levels, and nationalities. The survey also included questions regarding sources of pressure, stress, and the judge’s perception of their own judging bias. To the researcher’s knowledge, the survey was the broadest and most in-depth survey of its type to date. 853 responses were included for analysis with judges participating from 54 countries around the world. Given the richness of data from the survey, the analysis of survey results will be broken up into two parts. This first report will focus on the participant demographics and an overview of the usefulness of specific educational tools.
Method and Participant Demographics
The survey collected responses from an online convenience sample distributed through gymnastics judges’ social media groups and via direct e-mail within several national judging associations. Participation was voluntary and anonymous. A total of 853 responses were included for analysis: 590 Women’s Artistic Gymnastics judges (WAG), 230 Men’s Artistic Gymnastics Judges (MAG), 17 Rhythmic Gymnastics Judges (Rhy), and 16 from other judged disciplines (Other) which included Tumbling & Trampoline, Acro, Team Gym, and judges who judge multiple disciplines. (See Figure 1)
The survey had participation from a broad range of judging levels. 151 International Gymnastics Federation (FIG) rated Brevet judges responded (51 WAG, 91 MAG, 9 Rhy), 623 judges were classified as National/Optional (Nat’l) level judges (488 WAG, 128 MAG, 7 Rhy), 54 judges were classified as Regional/Compulsory (Comp) level judges (42 WAG, 11 MAG, 1 Rhy), and 9 of the respondents were retired WAG judges (all USA). 16 judges from multiple and other disciplines (Acro, TeamGym, and Trampoline and Tumbling (T&T)) were not sorted by judging level as certification systems varied greatly among those disciplines, and judges who judge multiple disciplines often had differing levels of certification for each discipline. These classification groupings allowed for the closest comparison across disciplines and nationalities with different educational systems and levels of certification.
Nationality
54 nations were represented in the survey responses. The highest number of responses were from USA MAG and WAG judges (482 WAG, 116 MAG). To prevent over-representation of USA data, much of the analysis will present results for USA MAG and WAG judges separately from the judges of other nations (labeled “Int’l”). There was also considerable participation from AUS (16), CAN (38), GBR (36) and NED (24) allowing for a more thorough analysis of their responses, which will be presented separately when found to be of significance. 15 (1.76%) of the participants declined to identify their nationality. (See Figure 2)
Gender
615 (70.81%) of respondents were female and 231 (27.20%) were male, while 7 (1.00%) preferred not to say. As expected, the vast majority of judges align to the gender of their discipline with 94.24% of the WAG judges listing their gender as female and 85% of the MAG judges listing their gender as male. (See Figure 3)
Age
Participants varied in age from 16 to 86 years old, with 21 participants whose age was not determined. The mean respondent age was 47.13 and the median was 47.00.
Compulsory/regional judges tend to be younger than FIG and national/optional judges. These differences are both statistically significant at alpha=0.05. (See Figure 4)
The age judges began judging varied significantly, starting as young as 10 years old to as old as 65 years old. Some judges commented that the starting age they reported was the age they began to learn the judging process, not their age of their first judging assignment. The mean reported starting age was 26.22 and the median was 23.00. Most judges (71.39%) start judging under the age of 30. FIG judges tend to start at an earlier age (mean=21.33) than optional/national judges (mean=27.32). This difference is statistically significant at alpha=0.05. The oldest starting ages for FIG judges were ages 36 WAG and 38 MAG (See Figure 5). It was observed that female MAG judges generally started judging later than their male counterparts. The average female MAG judge started judging at age 33 and the average male MAG judge started judging at 22. However, the reverse is not of statistical significance; the average female WAG judge started at 27 and the average male WAG judge started at 29.
Respondents have been judging for a mean of 21.09 years and a median of 19.00 years. Unsurprisingly, FIG judges have the most experience on average. 49.65% of FIG judges earned their brevet rating within their first two cycles of judging, and 27.97% earned their brevet rating after three cycles of judging. Compulsory and Regional certifications are simpler levels of certification to obtain in most countries; therefore, those judges typically have fewer years of judging experience. (See Figure 6)
Background
Formal Education
The process of gymnastics judging has been described as a “complex information-processing task, surpassing the limited human capacity to process information” (Plessner, 1999). As such, it is not surprising that gymnastics judges generally have a high degree of formal education, with 79.13% of the judges earning a bachelor’s or post-graduate degree, as shown in Figure 7.
It should be noted that 31 (41.3%) of the participants age 25 and under indicated they are currently students in the process of attaining a University degree.
Highest Level Attained as a Gymnast
The influence of personal motor experience (experience as an athlete) on officiating expertise is an area of current research and debate in many sports. A study by Pizerra (2012) measured the judging performance of gymnastics judges who can execute a specific skill they were evaluating and found that judges who had personal experience performing an element tended to be more accurate in their evaluation of that specific element. In contrast, Heinen, Vinken & Velentzas’s (2012) study concluded that judging in gymnastics can be facilitated by either the personal motor experience or by specific visual experience (past judging experience). Dosseville, Laborde & Raab’s (2011) study of judo referees took the examination of embodied cognition a step further, accounting for judging experience and sequential factors which are present when judging in live competitions. Their study found that recent personal motor experience in aspiring judges had a negative influence on decision-making compared to decisions made by more experienced judges, because the judges were more likely to judge based on the assumed intentions of the athlete, rather than an objective evaluation of the executed actions. This underlines the value of actual judging experience over a reliance on personal participatory experience within the sport. There would be great value to have a study of similar ecological validity conducted for gymnastics judges.
In the current analysis of survey participants, former international elites were most likely to be FIG judges; 58.12% of international elites who responded are now FIG judges, while just 37.83% are national/optional judges and fewer judge at lower levels. However, 71.33% of the FIG judges who responded did not compete at the international elite level. (See figure 8)
Roles Other than Judge
Many judges hold, or have held, other roles within the gymnastics community (See Figure 9). Coaching is the most common complementary role judges play. 89.40% of FIG judges are coaches or have coached in the past; compared to 82.37% of national/optional judges and 83.02% of compulsory/regional judges. MAG judges are also more likely to have coaching experience than WAG judges (88.70% and 82.03% respectively). While many judges noted their coaching background, only 25% of the respondents listed “Coach” as their current primary occupation. Even so, multiple judges cited their coaching experience as helpful to their development as a judge, with comments such as: “As a coach, I am mentally judging all the time” (MAG Nat’l) and “Coaches are discerning judges” (MAG FIG). Other gymnastics roles mentioned were: leotard company representative (WAG Nat’l), gymnastics commentator (MAG FIG), and sport researcher and manager (MAG FIG).
Educational Tools
Judges use many educational tools to help them develop their craft. Most of these tools focus on the technical aspects of judging, such as identifying elements, memorizing skill values, learning rules, and the practical application of that knowledge. Of the educational tools which judges indicated they used, respondents gave the highest helpfulness ratings (“Very helpful” or “It’s the best” ratings) to: live practice with other judges (85%), in-person judging courses (84%), and competition videos with D & E scores (83.9%). Live practice with other judges received more “best” ratings (333) than any other category. (See figure 10)
Most Helpful Tools
Judges cited that live practice with other judges is an invaluable educational experience. Even with the increased availability and quality of videos on the internet, live practice is still viewed as a more valuable learning experience. “Video is OK for practice but it's never the same as judging live” (WAG FIG). “More practice session with judges live discussing views, deductions, value parts, bonus” (WAG Nat’l). Previous studies by Dallas & Kirialanis (2010) and Ste-Marie (1999) have emphasized the importance of experience and live practice in the development of gymnastics judging expertise. Live practice gives judges the opportunity to practically apply their book-learned (or video-learned) knowledge and transform it into the ability to actually perform the task. It is the process of transforming declarative “book and rule” knowledge into the procedural “how to” knowledge necessary for accurate evaluations on a competition floor. “Knowing the rules and applying the rules are very different. There are lots of judges who are bad test takers who are great judges and vice versa” (WAG Nat’l). “What we judge and what we are asked on the test isn't always reflective of real-life judging” (Rhy Nat’l). “It is not just being able to pass a written test that makes you a good judge” (T&T Nat’l). Other written-in comments cited that live judging practice is more effective when it is done in small groups with higher level (FIG) judges present and “Practice with the senior-level panels” (MAG FIG). “I love practice judging with a small group and we can share why we took certain deductions… Judging routines with high level judges in a practice setting is helpful to see things from another perspective” (WAG Nat’l).
Even FIG judges cited that opportunities to exchange opinions with other judges, “judging with other judges and discussing the scores” (MAG FIG), is important to continue developing their judging expertise. Such opportunities can be limited in competition settings, but post-competition discussion with colleagues can help judges reflect on unexpected presentations which had occurred. Judges shared that learning from their live experiences judging on panels, having secretarial roles at higher level competitions, and judging alongside other judges at competitions is beneficial to better understanding their own judging tendencies. “As someone who almost always judges alone, I think more judging with others (whether in practice or in competition) is enormously helpful. You will only be made aware of errors you are making when others see you making those errors, which probably won't happen unless you're judging alongside others” (USA MAG Nat’l). “I usually go back after the meet to reevaluate and ask colleagues their thoughts so I can learn for future reference” (WAG Nat’l).
Some specific judging courses were found to be helpful:The FIG Intercontinental Course was specifically cited as particularly helpful, and MAG courses given for CAN, GBR & NED judges along with GBR WAG courses also received high helpfulness ratings.Judges cited having regional and national courses target different levels of judging, and having a positive learning atmosphere as things that make in-person courses more effective. Suggestions from the survey include: “The live course should be broken into two groups: beginner judges - basic concepts and rules, then seasoned judges - review of updates and standards… As a beginner, I found they glazed over concepts they think everyone just knows and it's intimidating to ask because of this "boys club" feel in the room” (MAG Comp).Both MAG and WAG USA courses seemed to vary by the region in which they were given.“I think clinics are very helpful. Each year at our summer clinic I learn new techniques that help me to be more accurate” (USA WAG Nat’l).
Judges frequently mentioned that practice videos are far more effective when there is a breakdown of what a correct evaluation could be. Individual practice with videos often lacks feedback to realize the errors one has made. Knowledge of the D & E scores which were awarded at the competition and “being able to watch lots of scripted videos both for E and D” (MAG Nat’l), brings more value to the drill of practicing with videos at home. “I feel much more comfortable when I … can compare my scores to other judges. Because of this, I actually feel more comfortable judging optionals because there are far more scored optional routines online than for compulsory” (MAG Nat’l). Additionally, evaluations which delineate the elements and the deductions which should be taken are exceptionally helpful for judges to realize exactly where their errors may have occurred. Plessner and MacMahon (2013) cite the importance of immediate feedback in video training to verify the correctness of judgements as a necessary feature to increase the accuracy of future judgements. Many judges wrote about specific characteristics which lend greater feedback value to practice videos, such as: “Videos with scores and explanations of deductions,” “…how elements should look like to get credit,” “videos of elements complete from start to finish with comments about form and execution… to learn what it is supposed to look like and actually see perfect compared to not perfect” (WAG Nat’l). Judges also had suggestions on video resources and features that would best help them: “more video practice that judges can do at home to discern differences between the deductions” (WAG Nat’l), “Videos with annotated deductions shown in regular and slow motion” (MAG Nat’l), “More interactive education with notations, angles, timers... I prefer where there is a single skill or family of skills with varying degrees of execution shown... Poor with multiple deductions, poor, medium, good, excellent” (MAG FIG). All of these video tools help judges to bring greater meaning and learning to their individual practice.
YouTube was frequently cited as a source for gymnastics videos, particularly the USAG and FIG YouTube channels. Judges also found the video links from the Road To Nationals website helpful for locating videos of specific NCAA performances linked to the scores they received in competition. It should be noted that videos from the judges’ seating perspective of every routine from every session at all World Championships and Olympics since 2014 are available on the Web-STS (FIG Education) website, along with a frame-by-frame multi-speed video player.
Helpful, but Underutilized Tools
It can be said that there are two helpful educational tools which are underutilized, in that many people reported never using it, but those who have used them gave high ratings. Web-STS, the official FIG learning platform, had the most striking response. A strong majority (63%) of the judges who have used it gave it a “Very helpful” or “It’s the Best” rating. In an even greater testament, of the FIG rated judges who have used STS, 85.6% gave it a “Very helpful” or “It’s the Best” rating, making it one of the most useful tools for the highest rated judges. In addition to the high FIG ratings for STS, multiple judges wrote in comments emphasizing the important role STS plays in their judging preparation, particularly for the FIG Continental and Intercontinental courses. However, just over 50% of the respondents reported they had never used it or didn’t give it any evaluation, making STS the most under-used of the most helpful educational tools.
The other somewhat underutilized, yet very helpful tool, is that of a one on one mentor. Many judges (66.4%) indicated that they learned to judge with the help of a current judge, however only 22.32% would classify themselves as having learned from an official mentor. And while 76.64% of the judges who had a formal mentor rated their experience as “Very helpful” or “It’s the Best”, nearly 16% of judges reported that they never had any kind of mentor. It seems as if very few countries have formal judging mentor programs, and judges must independently seek out their own personal judging mentor to help them in their learning process. “It would help to have more mentoring when starting out, I feel there is a lot of pressure to already know what you are doing and have a handle before even starting. I did not receive much guidance and was on my own for most of the process” (WAG Nat’l), “It is very hard to start being a judge if you aren’t actively coaching...there’s no good process to learn/practice/connect with mentors or other judges. And after you pass your exam, there’s no real guidance what to do next” (WAG Nat’l), “Some attention to solidly training/developing the lower level judges so they ultimately can feel comfortable progressing to higher judging rankings would be appreciated” (WAG Nat’l), “A mentor is a must for non-gymnast judge” (WAG Nat’l).
The judges who are fortunate enough to have an experienced judge they can go to for guidance highly value that relationship: “I have learned so much from being with more experienced judges” (WAG Nat’l), “I like being with a more experienced judge. They can talk about what they saw vs what you saw. Their insight is very helpful!” (WAG Nat’l) and that it helps to be “judging with veterans who explain hard situations” (WAG Nat’l).
Tools Receiving Mixed Responses
There were mixed responses for Pre-competition judges’ meetings and National Junior Program Manuals dependent upon region. Although 56.26% rated Pre-competition judges’ meetings as only “somewhat” or “not very helpful”, others rated them much higher and wrote in comments that the judges meetings at FIG competitions were extremely helpful. “I find the judges briefings at World Cups and Continental Champs extremely educational… Also the presentations of World Champs made available on STS” (WAG FIG). It was cited that by having practice “video judging reviewed by all judges and panels together” (Rhy Nat’l) during the judges’ meetings, judges are able to calibrate for the expected level of competition. An in-meeting review of specific elements which are likely to be performed was also cited as beneficial for judges to fine-tune and align their judging expectations for the competition, along with the value of practice judging with their panel (when permitted and known in advance) prior to a competition.
The helpfulness of National Junior Program Manuals understandably varied by country. Some Junior Program Manuals are little more than a list of elements to be performed, whereas others are extended documents which include philosophies on program direction, skill development, and desired technique. Some nations have video examples with guidance for judging parameters directly linked to the documents themselves, making those types of manuals more valuable tools to encourage uniformity in judging across the community.
The helpfulness of the FIG Code of Points as an educational tool also varied by region and level. For example, 65.12% of judges rated the FIG Code of Points as “very helpful” or “it’s the best”, however, 182 (38.15%) compulsory and national USA WAG judges responded that they never use the FIG Code of Points, as the USA women’s program has their own Junior Code of Points with a scoring system still based on a 10.0 scale.
Educational PowerPoints received of mix of helpfulness ratings. Again, this is based on how well they are done. PowerPoints which just list the rules without video examples are seen as less helpful; whereas others stated: “I enjoy the presentations where routines are scored and deductions gone over after that. Those educational materials help me to see deductions accurately and help judges be more on the same page with their deductions” (WAG Nat’l).
PowerPoints which include videos with clear examples demonstrating the subtle differences in the execution of elements were cited as the most helpful. PowerPoint Videos which freeze the frame and use drawing tools to show lines of importance along with offering graphic overlays of how angles are measured are great learning aids. PowerPoints created with these features can help judges learn where to quickly focus their attention. Bard, Fleury, Charrière, & Halle (1980) studied the gaze patterns of judges and determined that expert judges focus on information relevant to evaluation faster than novice judges. This means that experienced judges have more effective visual search patterns. Novice judges have more visual fixations and spend more time scanning between fixations in processing “dead time”, during which no meaningful evaluation occurs. During the scanning process, novice judges take in a large quantity of trivial and redundant information leaving them with less time to analyze errors. According to Ste-Marie’s (2000) study of expertise in women’s gymnastics judging, novice judges also spend less time looking at the gymnast and more time looking at their papers than their more experienced counterparts, further limiting the time spent taking in meaningful information. PowerPoints which demonstrate where and how angles are measured help judges learn exactly what they should focus on during specific phases of each element and develop their ability to rapidly home in on the most relevant information. As novice judges learn from these tools, they become more experienced and, like expert judges, can spend their time more efficiently analyzing rather than searching.
Many of the same concepts which make at-home video practice more effective also apply to judges’ Educational PowerPoints: “Being able to see a skill done with perfect technique either by a gymnast or biomechanically to see how a technically correct skill should look. Videos with angular deductions shown with line markers to increase easy recognition of the applicable deduction” (MAG FIG), “More examples of actual great skill technique or a comparison break down of execution deductions”(WAG Nat’l), “FIG help desk also helps alot” (WAG FIG) were all comments of things judges liked to see or would like to see more of in judging educational PowerPoints.
Online judging group challenge/contests garnered mixed results dependent on the individual judge’s experience. The concept of an online challenge was summed up by this WAG Nat’l judge: “Pair up judges from different geographic areas and have them judge together virtually. Maybe have a contest to see which pair or group can get closest to the base score, or to a national average.” The USA National Gymnastics Judges Association (NGJA) has run such a challenge contest in preparation for the past three MAG NCAA seasons. Several judges emphasized the educational and calibration benefits of the MAG NCAA Challenge: The “NCAA Challenge has significantly helped to reduce bias in NCAA judging” (MAG FIG). However, similar to the efficacy of small judging groups, challenge teams were cited to be more effective when there was a variety of levels represented on the team, so that less experienced judges have the opportunity to learn alongside more seasoned judges. It was stated that when a group is comprised entirely of less experienced judges, the learning during the discussion phase is limited.
Tools with Low Helpfulness Ratings
One would think with so many judges citing the importance of working and learning through discussion with other judges that social media judging groups (such as Facebook groups) would also have high ratings. But that is not the case. Social media-based judging groups (such as Facebook groups) were seen as the least helpful educational tool, garnering the fewest “best” and “very helpful” ratings and the most “not very helpful” ratings of any listed category. There is an etiquette among judges on many social media pages not to critique specific scores which were given in competition, as circumstances on the competition floor are different than those when viewing a video on social media. This philosophy of respect limits the usefulness of public social media forums for open, informed discussions among judges. It is unknown if there are closed judging groups, which could offer more psychological security for judges to openly participate, which have seen more success. The only Facebook groups mentioned by name as being helpful were the PAGU Facebook Groups and the Referee Gym pages on Instagram and Facebook.
Judging Websites, Apps, and Other Written-in Responses
Other website-based educational tools and judging apps received mixed ratings, with many judges not making use of them at all. Respondents were asked to specify which websites and apps they find helpful. Those mentioned often divide along MAG/WAG lines in their use, as most are very discipline specific for technical knowledge. Respondents were also able to write in additional tools which they find helpful. The most common written-in responses are listed in Figure 11.
Several judges wrote in comments regarding the value of watching gymnasts practice. As we saw in the demographic information, many judges are also coaches and count their coaching time as something that helps them develop as judges with statements such as: “In my head, I judge almost every routine I see, even as a spectator” (MAG Nat’l), and “Coaching athletes is the very best tool that I have found” (WAG Nat’l). However, some coach-judges voiced more technical concerns: “As a coach, I have felt some of the judges were not up to date on skills and technique” (WAG Nat’l) and the need for judges “…staying involved in the coaching process so they are more up to date with current skills and how they are taught” (WAG Nat’l). “From a coaching perspective, I feel like if I have an older panel, judges typically want different shapes than what the modern code ask for, so my athletes get deducted for doing what I want them to do” (WAG Nat’l).
Even judges who are not currently coaches stressed the importance of getting into a gym to watch live practice stating: “In gym experience is irreplaceable. Knowledge of the skill and how it should be performed” (WAG Nat’l), “Live gymnast training not video. Eyes need to be trained for technique” (WAG Nat’l), and the importance of “Coaches working with judges and learning from each other” (WAG Nat’l). Similar to watching podium training, being present at gymnast practice sessions allows judges to see a variety of errors, giving them an opportunity to prepare, and not be caught off-guard by unusual presentations which sometimes occur in competition. A judge’s experience in the gym allows them to cultivate a better understanding of the composition of gymnastics elements, varying techniques which may be used, and technical challenges which gymnasts face in execution. The benefit was stressed of “having coaches explain/demonstrate different techniques better helps judges understand,” and that there should be an “open dialogue with coaches and judges working together for better understanding” (WAG Nat’l).
In a study of characteristics of judging expertise by Diane Ste-Marie (1999), it was noted that expert and novice judges differ in their ability to read and interpret biomechanical information as a gymnast performs, allowing expert judges to more accurately anticipate gymnastics elements. The increased accuracy in anticipation of elements likewise resulted in a more accurate evaluation of the execution of the elements. The capacity to quickly and correctly anticipate what element will be performed translates to the judge rapidly attuning to the most relevant areas of focus, less time spent scanning, fewer gaze fixations, and more available processing time to evaluate the element being performed. Survey respondents noted the importance to develop their understanding of the biomechanics of gymnastics and how to apply it to their judging by “Spending more time with coaches and other judges to talk about the human body and what is possible and the techniques” (WAG Nat’l).
Judges also mentioned the value of crafting their own reference guides (a.k.a. “cheat sheets”), video analyses, and study guides. Judges deepen their understanding of the rules and evaluation during the creation process, and others find value in the finished products when they are shared.
The gymnastics judges’ survey was written early in the 2019-2020 competitive season, before COVID-19 became a global pandemic, and the survey was launched early in the worldwide confinement periods. Therefore, there were no questions pertaining to the efficacy of Zoom and other similar online learning platforms which showed increased use during this period. Like much else, the judging community had a steep learning curve to figure out how to effectively use this tool. Zoom and other similar platforms were used as a substitute for live judges’ courses and training and as a medium for international practice judging opportunities with FIG judges from other countries. Many of the recommendations for effective PowerPoints and video practice tools can be applied to Zoom learning environments. Judging associations continue to explore and tweak how to most effectively use these platforms and overcome obstacles such as video buffering issues, and this will be an area of continued study.
Conclusions and Limitations
Conclusions
Effective judges’ education is the cornerstone for accurate scores at competitions. When done well, structured educational courses can provide a great foundation for a judge’s development. A judge’s fundamental knowledge base is then refined and reinforced through small group study, live judging experience alongside other judges, and mentorship by experienced judges. Pre-competition judges’ meetings serve to review and refine the evaluation of specific judging challenges which are anticipated to be seen on the competition floor and provide a calibration mechanism for judges to align their evaluation.
It is seen from the survey that judges also rely heavily on independent study to prepare for judging assignments. As such, it is necessary for judges to have at-home preparation tools which go beyond the FIG Code of Points and national junior program manuals. Tools which provide feedback, such as videos with D and E scores and a delineated notation of errors bring meaning to at-home video practice. Notwithstanding, some of the web-based tools, such as Web-STS are under-utilized by lower level judges, even though the highest rated judges find them incredibly helpful.
It is not sufficient for judges to practice judge with videos. Expert judges have a broad understanding of the biomechanics of the performance of gymnastics elements. They augment their judging expertise through live observation, whether it be through coaching or intentional in-gym practice opportunities. Indeed, effective judging comes from a blend of structured instruction, individual practice and live evaluation experiences.
Limitations
There are limitations inherent with surveys as a method of research, from which this survey is not immune. For example: Analysis of data is correlational in nature, therefore, no definitive statements about causality can be made. Even though survey distribution on social media was targeted to gymnastics judging populations, the fact that there was some distribution on social media, limits the control over who volunteered to participate and their responses may not be generalizable to the entire judging population. It is important to note that the survey was an international survey presented in English. As such, most of the respondents were from primarily English-speaking nations. However, there were a number of participants for whom it was apparent that English was not their native language. For ease of reading, and with respect to their thoughtful contributions, some English spelling and minor grammatical errors have been corrected in participant quotes.
The next report will focus on the judges’ learning process and specific technical challenges of judging.
Author Note
Acknowledgements
I would like to thank all of the judges who participated in the survey, particularly those who helped with follow-up interviews and the people who gave the survey a wide distribution within their organizations. The survey was quite extensive; even so, nearly 85% of the judges elected to do the additional set of questions, which is a testament to the value of the inquiries. Further analysis will continue to be done pertaining to sources of pressure, financial compensation and bias. I would also like to give a special thanks to Brina Seidel at Score for Score for partnering to crunch the data, Danae Rupp for additional analytics and insight, and Darby Summers for editorial assistance.
Address correspondence to Kathi-Sue Rupp. E-mail: FlippedDecisions@gmail.com
Works Cited
Bard, C., Fleury, M., Charrière, L., & Hallé, M. (1980). Analysis of gymnastics judges' visual search. Research Quarterly for Exercise and Sport, 51 (2), 267-273.
Dallas, G., & Kirialanis, P. (2010). Judges' evaluation of routines in men artistic gymnastics. Science of Gymnastics Journal, 2 (2), 49-57.
Dosseville, F., Laborde, S., & Raab, M. (2011). Contextual and personal motor experience effects in judo referees' decisions. The Sport Psychologist, 24, 67-81.
Heinen, T., Vinken, P. M., & Velentzas, K. (2012). Judging performance in gymnastics: A matter of motor or visual experience? Science of Gymnastics Journal, 63-72.
Pizzera, A. (2012). Gymnastic judges benefit from their own motor experience as gymnasts. Research Quarterly for Exercise and Sport, 83(4), 603-607.
Plessner, H. (1999). Expectation biases in gymnastics judging. Journal of Sport & Exercise Psychology, 21, 131-144.
Plessner, H., & MacMahon, C. (2013). The sport official in research and practice. In D. J. Farrow, Developing Sport Expertise: Researchers and Coaches put Theory into Practice (pp. 71-95). New York, NY: Routledge.
Ste-Marie, D. M. (1999). Expert-novice differences in gymnastic judging: An information-processing perspective. Applied Cognitive Psychology, 13, 269-281.
Ste-Marie, D. M. (2000). Expertise in women's gymnastic judging: An observational approach. Perceptual and Motor Skills, 90, 543-546.