PSA - Don't be a Luddite

Elaborate on the merits of specific tournaments or have general theoretical discussion here.

Re: PSA - Don't be a Luddite

Postby Periplus of the Erythraean Sea » Sat Oct 21, 2017 9:35 pm

CPIGuy wrote:Y'know, Thursday's great and all, and if we'd been more on top of things we could perhaps have mitigated the 30-minute delay that happened because staffers didn't know how to use the system (although I sent out the instructional video to all of our rotating staffers a week in advance) -- but what would have been even better would have been a few weeks' notice, since you clearly knew about this for a really long time, and yet you didn't really indicate it anywhere in the tournament announcement (at least not that I saw). That way we could have actually planned on having an appropriate number of staff, rather than realizing four rounds in that we had insufficient staff and the tournament was going to be delayed. Even if we'd realized this as soon as we'd gotten your email, it would have been pretty difficult to completely restructure our staffing on two days' notice.


Perhaps this is me making too many assumptions, but this strikes me as a rather disingenuous assertion given that you seem to both a) post fairly often and b) maintain a reasonably robust Discord presence. This, and the fact that several Michigan people played the online mirror, during which we told people we were using an advanced stats system, makes this really seem like you're playing dumb to pile on the bandwagon and score some points here, when in reality you played four fucking house teams, including a number of experienced moderators. Colin Parks appears to have identified the problems on both sides accurately at least.

Furthermore, you insisted on almost certainly delaying things by rotating your staff, when one of the big advantages of the whole stat system is that the mods shouldn't even have to leave the room to keep the tournament running. So, the folks running the tournament compounded whatever problems that may have existed due to lack of familiarity with a new system (which many people have commented is quite easy to get used to after a few rounds) by effectively eliminating the ability of mods to get used to it during the day.

The fact is that the people who were coordinating logistics and sending out the emails are juggling day jobs, set editing (to make the set as high quality as you insist it was), and a number of other quizbowl responsibilities, and still managed to respond to forums feedback to improve the system. In fact, Ophir went out of his way to spend time making the system even easier this week after already making a number of improvements to the system in response to previous criticism - most notably, a feature where you can just type in three ones or zeroes (instead of having to go into each box and write a 10 or 0). We provided comprehensive instructions, a demo video, suggestions as to how to best use the system, and more. The only errors on our part were things that were already acknowledged.

Here is the list of skills it requires to use the system efficiently as a solo moderator:

1) Viewing a YouTube video / Showing one to all your staff (I was informed that staff were never shown a video)
2) Tabbing between browser tabs
3) Knowing how to use arrow keys / shift / keyboard shortcuts, to move around easily (this is something that you start to figure out and get used to more and more)
4) Efficiently using time (enter the tossup value after you've read the first bonus part and the other team is collaborating, things like that - more stuff you pick up over time)

I would have sympathy for people who aren't digital natives / used to spreadsheets who are introduced to this system. For a team that seems to be fairly young and composed of digital natives, however, these do not strike me as particularly difficult. In particular, in my experience it's a faster to type "10", push an arrow button a few times, then write "101" to indicate a team getting Part A, missing Part B, and getting Part C than it is to write all these down on paper. The buzzpoints just require a click on the packet to use (and you're already in the packet anyway) and only need to be copied at the end of the game. Thus, I have reason to suspect that many folks who were taking 45 minutes without a scorekeeper might have been taking almost as long with a traditional system, and that this would almost certainly be true if the instructions we took care to provide before the tournament were followed.

Again, there were communication issues which I apologize for, but I resent the consistent implication that this is some sort of onerous requirement, when a) it is clearly not and b) in this case, the people implying this seem to have actively contrived to prevent people from getting used to the system. In fact, we would have said "yes" to a request to just use paper scoresheets (and we let Minnesota do so) had you asked, but there was no email communication after we sent out information Thursday night, aside from some supplemental instructions from Ophir.
Will Alston
Bethesda Chevy Chase '12
Dartmouth College '16
"You are beyond parody" -Auroni Gupta
User avatar
Periplus of the Erythraean Sea
Auron
 
Posts: 1454
Joined: Mon Feb 28, 2011 11:53 pm
Location: Falls Church, VA

Re: PSA - Don't be a Luddite

Postby CPiGuy » Sat Oct 21, 2017 10:28 pm

Periplus of the Erythraean Sea wrote:
CPIGuy wrote:Y'know, Thursday's great and all, and if we'd been more on top of things we could perhaps have mitigated the 30-minute delay that happened because staffers didn't know how to use the system (although I sent out the instructional video to all of our rotating staffers a week in advance) -- but what would have been even better would have been a few weeks' notice, since you clearly knew about this for a really long time, and yet you didn't really indicate it anywhere in the tournament announcement (at least not that I saw). That way we could have actually planned on having an appropriate number of staff, rather than realizing four rounds in that we had insufficient staff and the tournament was going to be delayed. Even if we'd realized this as soon as we'd gotten your email, it would have been pretty difficult to completely restructure our staffing on two days' notice.


Perhaps this is me making too many assumptions, but this strikes me as a rather disingenuous assertion given that you seem to both a) post fairly often and b) maintain a reasonably robust Discord presence. This, and the fact that several Michigan people played the online mirror, during which we told people we were using an advanced stats system, makes this really seem like you're playing dumb to score pile on the bandwagon and some points here


Looking back at what I wrote, I was certainly too harsh -- it's probably the case that no matter how much pre-knowledge we had, nothing short of being told that the system would require more staff than usual (by you or by another site) would have mitigated the problems. I certainly wasn't "playing dumb" intentionally, but uh I was wrong to blame our failure to provide two staffers per room on your failure to give us lots of notice (and I'll add an addendum to that post as soon as I finish this one). I do maintain, however, that for TDs who aren't as "plugged in", and who are perhaps more prescient than myself with regard to the staffing requirements of Ophirstats, could certainly have benefited from some real advance notice. You seem to think that as well, though, so unless you actually, like, run another tournament and fail to notify TDs in advance, no further complaint in this regard.

Periplus of the Erythraean Sea wrote:you played four fucking house teams, including a number of experienced moderators. Colin Parks appears to have identified the problems on both sides accurately at least.

Furthermore, you insisted on almost certainly delaying things by rotating your staff, when one of the big advantages of the whole stat system is that the mods shouldn't even have to leave the room to keep the tournament running. So, the folks running the tournament compounded whatever problems that may have existed due to lack of familiarity with a new system (which many people have commented is quite easy to get used to after a few rounds) by effectively eliminating the ability of mods to get used to it during the day.


We played four house teams in order to ensure that the brackets would be evenly balanced and teams wouldn't have to have byes. We made the decision to play four house teams (rather than, like, two or zero) based on the knowledge that this would require eight staffers, and we knew we would have six staffers from the pool of "people not active Michigan players who haven't played the tournament". Rather than making two people staff the entire day, we made the decision to use the rotation system in order to let everyone play. Again, this was predicated on the assumption that one staffer per room would be sufficient. As I think I made clear, that assumption was false, and I take full responsibility for that. However, I think that assumption was entirely reasonable, as evidenced by the fact that you and several others continue to argue that it should be the case. I also think that, given that assumption, the staff rotation model was entirely reasonable, and were this tournament held on paper scoresheets I think there would have been virtually no problem with the staff rotation model.

Periplus of the Erythraean Sea wrote:The fact is that the people who were coordinating logistics and sending out the emails are juggling day jobs, set editing (to make the set as high quality as you insist it was), and a number of other quizbowl responsibilities, and still managed to respond to forums feedback to improve the system. In fact, Ophir went out of his way to spend time making the system even easier this week after already making a number of improvements to the system in response to previous criticism - most notably, a feature where you can just type in three ones or zeroes (instead of having to go into each box and write a 10 or 0). We provided comprehensive instructions, a demo video, suggestions as to how to best use the system, and more. The only errors on our part were things that were already acknowledged.


I don't disagree with any of this, and I don't think anything in my previous posts suggests I disagree with any of this, so I don't know why this is relevant. I (completely unsarcastically) agree that your stats system is awesome, and you've done a lot of good work to put it together.

Periplus of the Erythraean Sea wrote:Here are is the list of skills it requires to use the system efficiently as a solo moderator:

1) Viewing a YouTube video / Showing one to all your staff (I was informed that staff were never shown a video)
2) Tabbing between browser tabs
3) Knowing how to use arrow keys / shift / keyboard shortcuts, to move around easily (this is something that you start to figure out and get used to more and more)
4) Efficiently using time (enter the tossup value after you've read the first bonus part and the other team is collaborating, things like that - more stuff you pick up over time)

I would have sympathy for people who aren't digital natives / used to spreadsheets who are introduced to this system. For a team that seems to be fairly young and composed of digital natives, however, these do not strike me as particularly difficult. In particular, in my experience it's a faster to type "10", push an arrow button a few times, then write "101" to indicate a team getting Part A, missing Part B, and getting Part C than it is to write all these down on paper. The buzzpoints just require a click on the packet to use (and you're already in the packet anyway) and only need to be copied at the end of the game. Thus, I have reason to suspect that many folks who were taking 45 minutes without a scorekeeper might have been taking almost as long with a traditional system, and that this would almost certainly be true if the instructions we took care to provide before the tournament were followed.

Again, there were communication issues which I apologize for, but I resent the consistent implication that this is some sort of onerous requirement, when a) it is clearly not and b) in this case, the people implying this seem to have actively contrived to prevent people from getting used to the system. In fact, we would have said "yes" to a request to just use paper scoresheets (and we let Minnesota do so) had you asked, but there was no email communication after we sent out information Thursday night, aside from some supplemental instructions from Ophir.


I was not intending for my post to be read as "Will/Ophir/the EFT people made an onerously difficult stats system, and it's their fault our tournament sucked." In fact, before today, I think I would have agreed with you that this stats system should not cause much more time than paper. However, after seeing the actual evidence from during the tournament, it is clear that a) rooms in which there were two staffers ran as quickly as most rooms in most tournaments, while b) rooms with only one staffer often ran much slower, even when that staffer was known to be experienced and good. (I think that, for example, Libo Zeng is probably capable of getting through a round in considerably less than 45 minutes when he doesn't have to spend 30 seconds to a minute between every tossup and bonus switching between tabs and clicking on stuff. Libo was unable to get through Round 10 in 45 minutes, so it clearly wasn't a result of initial struggles.) It is probably the case that the most experienced staffers (you, Cody, etc) are actually capable of getting through a round solo without much delay. I also anticipate that as the stats system becomes more popular and people get more experience, more and more staffers will be able to run a room solo on this system. However, that is not the case currently, and the "growing pains" seem to last at least one full tournament's length.

So, in summary:

-- I'm not trying to attack you, and I'm sorry if/that I did
-- You are almost certainly suffering from "i'm very good at this thing therefore most people are" bias, which is reasonable, but the evidence suggests you are wrong.
-- The fact that our tournament ran late is entirely our fault. We made an assumption that advanced stats would require the same resources as paper scoresheets, which was reasonable but ultimately false. Therefore,
-- I advise that in the future, if the editors of a tournament want to use advanced stats, they notify TDs well in advance to ensure that the TDs have enough time to make adjustments to their staffing expectations. That way, people can have the best possible experience playing the tournament, and TDs can have the smoothest experience running it.

Edit to add: perhaps there could be some sort of public "dummy tournament" so that anyone can go on and practice using the stats system with real questions, in real time? that might make things go more smoothly, since many people have a hard time learning something solely from a video about it.
Last edited by CPiGuy on Sat Oct 21, 2017 11:01 pm, edited 1 time in total.
Conor Thompson
Bangor HS (Maine) '16
Michigan '20
CPiGuy
Rikku
 
Posts: 284
Joined: Wed Nov 16, 2016 8:19 pm

Re: PSA - Don't be a Luddite

Postby armitage » Sat Oct 21, 2017 10:44 pm

Rather than making two people staff the entire day, we made the decision to use the rotation system in order to let everyone play. Again, this was predicated on the assumption that one staffer per room would be sufficient.


I don't have any comment on the rest of your post - I can appreciate that you and the other staff were doing your best at damage control, and I'm sure you've learned a lot for future TD-ing. This point stuck out at me in the post, though, and I'd like to offer my opinion.
It's generally bad practice, however common it is, to rely on single-staffed rooms; that's why no national tournament lets this happen, even though they all run on paper scoresheets.
I was fortunate to be surrounded by experienced TDs in undergrad who sanity checked personnel decisions and made sure all of our tournaments, save a notable few, ran very well, and my impression is this may not have been the case for your club, so I hope pointing this out now is helpful. In the future, even for paper tournaments, I would strongly recommend doubling up staff on nearly all rooms before you consider forming house teams.
Richard Yu
User avatar
armitage
Wakka
 
Posts: 189
Joined: Sun Nov 28, 2010 11:52 pm

Re: PSA - Don't be a Luddite

Postby Periplus of the Erythraean Sea » Sat Oct 21, 2017 11:02 pm

CPIGuy wrote: (I think that, for example, Libo Zeng is probably capable of getting through a round in considerably less than 45 minutes when he doesn't have to spend 30 seconds to a minute between every tossup and bonus switching between tabs and clicking on stuff. Libo was unable to get through Round 10 in 45 minutes, so it clearly wasn't a result of initial struggles.)


Again, this is probably something that becomes easier with experience - at least half my job involves manipulating spreadsheets and slide decks - but you shouldn't actually be clicking that much. Click on the buzz point, tab to a new sheet, then click on the row where you'll enter the tossup score and the bonus points. I reasonably can't see that taking more than several seconds after the changes we implement unless there's lag (which apparently there can be if someone else is in the stat sheet - apparently me looking at some of the sheets to see what was up slowed a few things down, for which I sincerely apologize). This really is not very much clicking, and shortcuts like this can and should be explained in pre-tournament meetings as a best practice in the future.
Will Alston
Bethesda Chevy Chase '12
Dartmouth College '16
"You are beyond parody" -Auroni Gupta
User avatar
Periplus of the Erythraean Sea
Auron
 
Posts: 1454
Joined: Mon Feb 28, 2011 11:53 pm
Location: Falls Church, VA

Re: PSA - Don't be a Luddite

Postby Charbroil » Sun Oct 22, 2017 12:02 am

I actually disagree that using the electronic scoresheets requires having two staffers in every room. Most rooms at our mirror only had 1 staffer, and almost all rooms still finished in around 40 minutes. This is true even though two of our moderators had never moderated a college tournament before. (Admittedly, those moderators did have scorekeepers to help them for part of the day).

Obviously, 40 minutes per round isn't great, but it is fairly common among the college tournaments I've attended. In that context, I agree that the ideal would be to have 2 staffers in every room at every tournament. However, I don't think the electronic scoresheets make this ideal dramatically more necessary to run an effective tournament. (I do think that a few scorekeepers should be on hand, though, to help any moderators who are particularly struggling with the new program, but then again, I would think that you'd want to keep a couple of scorekeepers on hand at any event just in case.)
Charles Hang
Francis Howell Central '09
St. Charles Community College '14
Washington University in St. Louis '1x (President, 2017-18)

Owner, Olympia Academic Competition Questions, LLC
Question Writer, National Academic Quiz Tournaments, LLC and National History Bee and Bowl
Member, Missouri Academic Coaches Association
Charbroil
Auron
 
Posts: 1085
Joined: Fri Jun 09, 2006 11:52 am
Location: St. Charles, MO

Re: PSA - Don't be a Luddite

Postby Short-beaked echidna » Sun Oct 22, 2017 4:36 am

We used this at Oxford yesterday and most rooms didn't have two moderators and the tournament ran to schedule. Many people (including me) preferred using this system to paper scoring, and I think it is probably faster. I don't think anyone but me had used it before, and no one looked at how to use it before maybe 10pm the night before the tournament, and yet we had no issues with the scoring system itself.
Joey Goldman
Oxford '17
User avatar
Short-beaked echidna
Wakka
 
Posts: 172
Joined: Thu Dec 22, 2011 9:59 pm

Re: PSA - Don't be a Luddite

Postby tommykl » Sun Oct 22, 2017 1:18 pm

Short-beaked echidna wrote:We used this at Oxford yesterday and most rooms didn't have two moderators and the tournament ran to schedule. Many people (including me) preferred using this system to paper scoring, and I think it is probably faster. I don't think anyone but me had used it before, and no one looked at how to use it before maybe 10pm the night before the tournament, and yet we had no issues with the scoring system itself.

I'll temper that statement by saying that a couple of people modding had used it before, but most (including me) hadn't. The scoring system is easy to use and intuitive, and although we didn't track buzz points for the most part, it seemed pretty intuitive.

There are still ways for it to be misused, such as using the wrong scoresheet, etc., but the main issue I have with it (and it's a minor quibble) is that the advanced stats don't have all the details of SQBS. Essentially, if we want simple stats with win-loss records and so forth, it all has to go through SQBS again for publishing purposes, and whether the data comes from Excel spreadsheets or hand-filled scoresheets, the process is the same.

Another minor point I noticed is that while the number of points scored seemed accurate, the number of games played seemed erratic at best. I'm not sure if this is down to input errors from mods and scorers or a coding mistake, but it affected the PPG data.
Thomas De Bock - Oxford Brookes University
User avatar
tommykl
Lulu
 
Posts: 43
Joined: Tue Jan 26, 2016 7:58 pm

Re: PSA - Don't be a Luddite

Postby CPiGuy » Sun Oct 22, 2017 3:33 pm

tommykl wrote:Another minor point I noticed is that while the number of points scored seemed accurate, the number of games played seemed erratic at best. I'm not sure if this is down to input errors from mods and scorers or a coding mistake, but it affected the PPG data.


This happened to us as well.
Conor Thompson
Bangor HS (Maine) '16
Michigan '20
CPiGuy
Rikku
 
Posts: 284
Joined: Wed Nov 16, 2016 8:19 pm

Re: PSA - Don't be a Luddite

Postby Periplus of the Erythraean Sea » Sun Oct 22, 2017 3:36 pm

CPiGuy wrote:
tommykl wrote:Another minor point I noticed is that while the number of points scored seemed accurate, the number of games played seemed erratic at best. I'm not sure if this is down to input errors from mods and scorers or a coding mistake, but it affected the PPG data.


This happened to us as well.


This is an input error, not a system error. Mods routinely forget to enter TUH - unsurprising since it's at the bottom of the scoresheet. Definitely made this mistake a few times myself at UVA.
Will Alston
Bethesda Chevy Chase '12
Dartmouth College '16
"You are beyond parody" -Auroni Gupta
User avatar
Periplus of the Erythraean Sea
Auron
 
Posts: 1454
Joined: Mon Feb 28, 2011 11:53 pm
Location: Falls Church, VA

Re: PSA - Don't be a Luddite

Postby The Stately Rhododendron » Sat Jan 20, 2018 5:38 pm

Thanks to electronic scoresheets regs is still in prelims at 430! Thanks technology!

- quizbowl's pre-eminent anti-revisionary luddite
Oakland Mills '14 - The Most Unpredictable Team in Quiz Bowl
Yale '18 - Majoring in X Studies
"Field Commander" Isaac KD, "The Savior of the Forest"
User avatar
The Stately Rhododendron
Rikku
 
Posts: 405
Joined: Sun Oct 16, 2011 7:18 pm
Location: Heart's in the woods

Re: PSA - Don't be a Luddite

Postby Borel hierarchy » Sat Jan 20, 2018 6:16 pm

electronic scoresheets are very easy to use thanks to the fact that I've ever used a computer mouse. Thanks Ophir!
Andrew Wang
Illinois 2016
User avatar
Borel hierarchy
Tidus
 
Posts: 567
Joined: Sun May 23, 2010 10:03 am

Re: PSA - Don't be a Luddite

Postby RexSueciae » Sat Jan 20, 2018 7:57 pm

4:30 is crazy late.

I'd like to report that while one bracket of the Virginia Regs site got delayed and took longer to get through than the other, Ophir's stats system allowed us to keep moving so that after those issues were ironed out we were able to conclude seven prelim rounds and release all teams for lunch before 2PM. Our lads on stats were also on point, as everything got done before lunch ended so that we were able to quickly distribute prelim book prizes and commence the playoffs.

What sort of problems exactly did the Northeast site face with the electronic scoresheets? If a tournament hasn't finished its prelim rounds until nearly the evening, my first thought would be structural issues beyond the stats system. (I don't see an entry for the Northeast Regs site up on the database yet, is that related?)
Vasa Clarke
Maggie Walker '14
Virginia '18
RexSueciae
Rikku
 
Posts: 261
Joined: Sun Dec 29, 2013 12:24 am

Previous

Return to Collegiate Discussion

Who is online

Users browsing this forum: The Lutes of Dependency and 3 guests