Warlock Analytics: TES:L Tournament Survey Report
About three weeks ago, BradfordLee and I sought to collect some data for the benefit of the community-driven TES:L Champion Series tournament series as well as the slowly growing competitive scene in TES:L. With our powers combined, I felt confident we could arrive at some meaningful conclusions to share with the community. This article highlights key findings from that survey and discusses potential implications from these results. Also, here’s a quick shout-out to Twitch user beanofsize for winning the $20 gift code raffle! Be sure to check your Twitch messages.
Before we talk about the results, here’s a quick disclaimer akin to one I gave in my previous article on the first TES:L Community Survey. Because the survey was solicited primarily to individuals on Reddit and in various TES:L streams, there may be a slight sampling bias. What this means is that the opinions of the respondents may be more extreme than those who casually play TES:L. For several items on this survey, however, this is a non-issue. I hope to have the capacity in the future to distribute surveys to a wider range of the overall TES:L player base.
That all being said, let’s get started!
Setting the Tone: Tournament Mindset
The first section of the survey asked respondents to indicate how often they watched and participated in CCG (collectible card game) tournaments, which included variant questions asking specifically about CCG tournaments held online rather than in-person. These items were primarily meant to prime respondents and get them in the mindset of competitive CCGs. In the interests of brevity, these non-essential items will not be reported extensively here. While not part of the main analyses, the results fell in such a way that both online participation and online viewership scored higher than items that asked about online and offline participation/viewership combined. Keep this in mind as we continue to TES:L-specific items.
TES:L Tournament Insights
After engaging participants to think about CCG tournaments in general, I then steered them toward TES:L tournaments and competitive play. The first item had respondents indicate how often they watched community-driven TES:L tournaments on a scale of 1 to 5, where 1 represented “Never” and 5 represented “Very Frequently.” I opted to use this specific language so that this question can be asked again in the future once Bethesda has solidified an official tournament scene for TES:L. The results, as displayed below in Figure 1, illustrate that most of the responses clustered toward the middle and upper end of the distribution. The mean score for this distribution ended up being 3.52, to further illustrate the point.
The second item was a bit more complicated. In an effort to gauge brand identification with a new hashtag that the TES:L Champion Series had adopted on Twitter (#TESLCommunityCup) versus their original name, I had participants rank the extent to which they identified certain names to TES:L competitive play. Specifically, respondents were asked to rank TES:L Champion Series, TES:L Community Cup, TES:L Open Swiss Cup, TES:L Weekly Series, and the TES:L Turning Point Series from 1 to 5, where 1 represented the name they most identified with competitive play in TES:L and 5 the least identified. It is important to note here that the last three were added as a means of concealing the true comparison I wanted to draw between the TES:L Champion Series and the Community Cup.
Brace yourselves, statistics talk is coming. In the spirit of true empiricism, I first ran what is known as a repeated measures analysis of variance (ANOVA) test to determine if there were any significant differences between the rankings of the five tournament series. This sort of procedure is known as an omnibus test – it assesses whether or not a difference exists between multiple comparisons. Further analyses are required to then determine which difference(s) were detected. It’s a bit like a home security alarm that triggers when an intruder enters, but it doesn’t tell you which entrance they used. Thankfully, modern software for statistical analysis (in my case, SPSS) includes pairwise comparisons between all possible permutations. The result of the comparison between TES:L Champion Series and TES:L Community Cup was staggering, to say the least. In a paired-samples t-test with 361 valid responses, participants significantly identified the Champion Series as more indicative of competitive play than the Community Cup brand (for the statisticians out there, t = -.327, p < .005).
So what does this mean? The short version is that participants clearly didn’t identify with the Community Cup brand, but that would be like me saying it didn’t rain in California today because my home in the Inland Empire didn’t see rain (this happens way more often than I’d care to admit, much to the chagrin of my tomato plants). As with most metrics, it’s difficult (and downright incorrect) to draw causal claims from self-report surveys, let alone singular items. This does give us an interesting talking point when discussing community tournaments in TES:L, though. Does this mean the Champion Series name carries a significant amount of weight, or did the Community Cup name just not stick? Additional data at multiple time points can help provide the truth (or, more accurately, our closest approximation of the truth).
But, hey, let’s go back to pretty pictures. In Figure 2, I illustrate the next item that tried to catch a glimpse of social media engagement with the TES:L Champion Series. This wasn’t an excellent indicator of tournament health, but it may also help to explain why the Community Cup brand never really took off.
This next one, however, was a bombastic result. When asked to rate their overall feelings toward the TES:L Champion Series on a scale of 1 to 5 where 1 indicated that they greatly disliked it and 5 indicating that they greatly liked it, the mean result was an excellent 4.40 across 394 valid respondents. As I explained to BradfordLee when discussing these results, this is roughly equivalent to a product having 4.5/5 stars on Amazon with 394 reviews. I know I would feel comfortable buying that. To solidify the point, feast your eyes on Figure 3:
When asked to comment on the rating they gave, participants had a wide variety of positive comments to give. Many complimented the casting quality, both for the mainstays of BradfordLee and Schwiddy, but also for the various guest casters such as CVH, Techandjunk, and xGymClassHero. This commentary, according to the comments, helped educate players on nuances of the game that they overlooked or had never seen before. They also lauded the professional quality of the tournament stream. Another common mention was the excitement of seeing the decks that would invariably shape the TES:L metagame for that week. The few negative comments were mostly doom-and-gloom concerns about the competitive future of TES:L as a whole, but these comments were in a very small minority.
The last two items regarding the TES:L Champion Series asked participants to indicate how satisfied or dissatisfied they were with the rate at which the tournaments increased prize support and featured popular players, including guests who don’t regularly play TES:L. These two items are illustrated in Figures 4 and 5, respectively.
The mean score for the prize support item was a positive 3.87 on a scale that ranged from 1 (Very Dissatisfied) to 5 (Very Satisfied), so there seems to be a clear appreciation for the rate of prize support increases, especially after August’s funding. September’s funding schedule has yet to be determined, but I personally have faith that it will be highlighted soon and continue this upward trend.
In terms of the popularity of competitors, the distribution of ratings was very similar to that of prize support with a mean score of 3.88 using the same 1-5 scale. Taken together, Figures 4 and 5 bolster the already resounding positive regard the Champion Series has with the overall approval rating.
The final two items worth mentioning on the survey were demographic in nature. Since I neglected to gather demographic information in my first Community Survey, I made sure to collect age and gender information here. Figure 6 highlights the gender breakdown among the respondents to the tournament survey. Participants were able to report their gender as male, female, other with the option to specify, or indicate that they prefer not to say.
Figure 7 is a condensed distribution of the age item, since illustrating every single age category would have caused unnecessary visual clutter. The most commonly reported age was 27, which was also the average for the distribution as a whole. Ages reported ranged from 14 years old to 65 years old.
As The Elder Scrolls: Legends enters its 14th month since being unleashed from Closed Beta, the question in the minds of many is, “Will there be a future for the competitive scene?” Speculation runs rampant in several forums and streams, but if spending so many hours with the data presented here has given me any insight, it’s this: the community is ready for a competitive scene. More importantly, I also believe representatives of Bethesda when they say that they are taking their time to make everything right. By throwing support behind the TES:L Champion Series, we as a community have already created a weekly phenomenon that both significantly impacts the game itself and inspires positive attitudes from its fan base. This alone gives me great hope. When an official tournament scene does eventually emerge, you can bet that I’ll be there, glued to my screen with a drink in one hand and a voodoo doll of Supreme Atromancer in the other (too soon?). Until then, however, I’ll see you all somewhere between the lanes!
New to TESL (5 days), but veteran magic player that left that excellent game almost a year ago. I feel ready to commit to a game, and one day stream and all the good things. I’ll be eagerly awaiting opportunities to get into competitive play, and look forward to learning much more. Thanks for a great article.