The Quizbowl Resource Center

(This topic has been split from this thread --Management)

I have no real new thoughts on any of the concrete ideas here (I obviously agree on the alphabetical thing, I have a range of thoughts on the "carrying a prelim loss" thing that I don't want to address here). I just wanted to say that I think Will's point about tone is correct. I don't know if "gratitude" is necessarily the right word, but yeah there's like an assumption in Conor's original post that NAQT is brutally incompetent or deliberately, gleefully malicious. Using words such as "holy fuuuuck this so stupid," "complete and utter bullshit," and "fucking bush league" seem unnecessarily confrontational to me (to be clear, I am not being a prude or saying they are never acceptable, just that they seemed unnecessary in this context at this moment). Starting a conversation with this invective, as Will pointed out, seems a clear way to enrage the people you're talking to or get them to be defensive.

This is also particularly unfortunate because Jonah's explanation makes sense: the policy was just overlooked during the format change and because it didn't seem to create an issue last year, it seemed fine this year. That's a mistake and they should definitely change it, but come on now, couldn't we just try to figure out what happened or what the reasoning behind this policy was before we start throwing words around like "fucking bush league" and "fucking stupid"? (If you want to argue the prelim-loss thing could use this invective, you may have a point as that has been talked about before.)

In light of what we talked about in the "How We Treat Each Other" thread, I'm kind of disappointed with this. Just because NAQT is a company doesn't mean it's not run by people, people that we know have a sincere desire to promote good quizbowl. Let's try to assume some good faith and believe that there might be reasons under than malicious or deliberate incompetence that sometimes suboptimal outcomes happen.

(This probably could go in another thread)

I'm aware of the tone rule--I tended to interpret said rule when I was a moderator as basically being put in place to stop people from rejecting arguments or points out of hand because of tone ("Conor's wrong because he's rude") or to avoid the temptation of "reporting" people so they can face punishment for using bad words or making other people feel bad. In this case, I (and Will) were not rejecting Conor's arguments; we tended to actually agree with them, nor were we calling for him to face punishment or apologize or anything. We both just thought the invective he used in his post was unnecessary, and while I guess that is "complaining about tone," I would hope that we don't end up with a scenario in which total invective/assuming the worst is acceptable, while urging good faith/respect isn't. In any event, I'm willing to take this discussion elsewhere and even off forum.

jonpin wrote: ↑

Mon Apr 08, 2019 11:16 am

Just wanna put it out there that if you are not a member of the board staff, you should not be spending the bulk of your post telling someone you don't like the manner in which they are complaining about a quiz bowl thing. It is right at the top of the board rules:

As a result of the above, and of the need to reserve the privilege of moderating the forums for the designated staff members, anything that looks like telling other people not to discuss quizbowl is prohibited. This includes but is not limited to: telling someone that you do not like the “tone” of their post...

I don't care! This is one of the worst defenses I've heard for putting up with the sort of inhospitable behavior that serves to alienate core institutions in the quizbowl community from each other. As Mike pointed out, for a community where we spend so much time self-flagellating about how people often treat each other (which, incidentally, seems to result in these same people being more civil and conflict-averse in public, but nasty and gossipy in group chats), we sure as hell seem to have a double standard when treating the people who run the single most important organization in the spread of good quizbowl today and have allowed people to make an actual living promoting this game. These people have done a hell of a lot more for quizbowl than almost any of us have and yet we give them the harshest treatment, and I find this extremely wrong. We should show some goddamn respect.

(Brief pause, of course, to acknowledge my hypocrisy based on the rhetorical tone I've adopted)

jonah wrote: ↑

Mon Apr 08, 2019 8:47 am

I don't remember hearing any issues about the alphabetical aspect last year, which is why—to the best of my knowledge—we didn't even contemplate changing it.

In before this thread inevitably degenerates horribly to say that, at least as far as I was explained things by R. and others last year, the alphabet rule was not actually the way ties were broken out of the prelims last year. In 2018, when OSU was 4-1 (loss to Chicago B) and Berkeley B was also 4-1 (loss to OSU), we were told the tie was broken on ppg (Berkeley had a total of like, 30 more points than we did over the course of the prelims). I can assure you that, had I understood the tie to have been broken BY ALPHABET, my response would have been *rather different.*

I actually referenced this explicit issue (breaking on ppg when the 1/2 seed matters so much) in the post I made about ICT format leading up to this year, but it's not surprising to me that Jonah (and other folks at NAQT) didn't see it. They care what we think, but they also don't care *that* much. NAQT's corporate culture is awful about diligence in a number of ways, and while I do see they're making efforts, I also have tons of emails where R. Hentzel, or another senior NAQT executive, promises to get back to me (sometimes on something extremely important!) and then literally never follows up. I'm not trying to be vitriolic here; I'm certainly grateful for the work NAQT does, I've been particularly vocal about endorsing areas where they've listened and made great changes (the clock for SCT/ICT being an excellent example). But that doesn't change that they have a real issue with responsiveness in general and seriously listening to feedback about SCT/ICT in particular.

Periplus of the Erythraean Sea wrote: ↑

Mon Apr 08, 2019 11:33 am

I don't care! This is one of the worst defenses I've heard for putting up with the sort of inhospitable behavior that serves to alienate core institutions in the quizbowl community from each other. As Mike pointed out, for a community where we spend so much time self-flagellating about how people often treat each other (which, incidentally, seems to result in these same people being more civil and conflict-averse in public, but nasty and gossipy in group chats), we sure as hell seem to have a double standard when treating the people who run the single most important organization in the spread of good quizbowl today and have allowed people to make an actual living promoting this game. These people have done a hell of a lot more for quizbowl than almost any of us have and yet we give them the harshest treatment, and I find this extremely wrong. We should show some goddamn respect.

As far as the tone thing goes, both of you are wrong.

Will, the line about "we should show some goddamn respect" is dumb, rhetorically dubious, almost certainly against board rules, and makes me just a bit low-key worried about why you got that "Authoritarian personality" power at CO a few years ago. The reason Conor's post should be more respectful is because good-faith people deserve respect, not because NAQT has earned some right not to be critiqued for things they do wrong. That's a stupid right for people to earn, and is completely anathema to how people who the community built itself out of something far lesser than it is today (in fact, this sort of reasoning was consistently employed by the people who most centrally OPPOSED that project).

Conor, it shouldn't surprise you that people don't like the way you post on this board. It fucking sucks. Like, take-a-hard-look-at-yourself-in-the-mirror sucks. It's really, really bad. And understand that I say this as someone who regularly used to post reckless and unproductively confrontational shit (and still struggles with that balance). So like, dude, Chris Ray is telling you that you're being way too much of an asshole on the internet, so maybe think about that. And folks here are correct when they say that ranting is not a productive form of engagement.

That said, I think it's a fair to quite sincerely wonder what a productive form of engagement would be. NAQT's leadership regularly fails to return emails. They refuse to bring in people who could help them seed ICT (even when the new format so critically depends on seeding!!), even though this criticism has been going on for like a decade. [If I'm wrong and you did bring people in, all I can say is Jesus, find better people, because it sure looks suspiciously similar to a pure D-value ranking...] Despite Seth taking an active role in the discussion about ICT format, it doesn't seem like Cody's excellent points in the thread were listened to at all. Instead, we got Chad Kubichek making smarmy comments about how carrying over prelim losses against unshared opponents was the One True Way all tournaments should be run (come on, Chad Kubichek!).


I do understand that, from a strictly corporate perspective, ICT isn't a moneymaker (though I would absolutely argue that it is nonetheless an important project for NAQT, and that they end up deriving plenty of value from it - I completely reject the suggestion that it's an act of charity).
And I want to make clear that I DO think NAQT cares - Joel Gluskin, for instance, specifically sought me out to talk about the one fix NAQT did make to the schedule (adding a tiebreaker out of playoffs); NAQT does deserve recognition for doing that, because it was important (though I also got the impression that "it was self-evidently important" played more of a role there than "the community really wanted this").

And that's really what I'm saying: That, as a community, we really need NAQT to do a better job of caring in certain specific ways. Listening a bit more is first and foremost among them. The community did not feel listened to during the Matt Bruce episode (though I understand their were legal complications, I think it's fair to say the public face aspect could have been done MUCH better). We haven't felt taken particularly seriously over the years regarding things like the clock, aspects of SCT scheduling, etc. And we don't feel listened to now about ICT.

We, on the other hand, need to take ownership of the fact that plenty of our griping has been overly caustic (and even at times unfair), and that it's easy to lose perspective that NAQT has a much larger set of projects to oversee (which we all agree are worth supporting). We can also do better. And yet, I don't have much of a reasonable expectation that being more reasonable will have any impact on the at times blatant disregard R. and Chad and others at NAQT have shown the community. I hope that changes, because it seems like there's quite a united voice about the ICT issue. This is a golden opportunity for NAQT, and for us, to take a step forward. Let's hope we're both up to the task.

I think Chris is correct. I think he raises an interesting question about "what a more productive form of engagement would be." I assume in many cases NAQT's corporate structure may complicate responses and decisions--(I'm speculating here, I have no idea)--even simple responses or solutions require presumably lots of communication and ink spilled, which would probably explain why they produce so many boilerplate and "we'll get back to you" responses. I am not sure if like a community relations person might help, or that would simply result in more boilerplate statements. Ultimately, I think Chris' point that more firm, respectful, persistent, and transparent communications outlining complaints in a reasoned, organized manner would probably be the best idea.

DumbJaques wrote: ↑

Mon Apr 08, 2019 12:37 pm

Periplus of the Erythraean Sea wrote: ↑

Mon Apr 08, 2019 11:33 am

I don't care! This is one of the worst defenses I've heard for putting up with the sort of inhospitable behavior that serves to alienate core institutions in the quizbowl community from each other. As Mike pointed out, for a community where we spend so much time self-flagellating about how people often treat each other (which, incidentally, seems to result in these same people being more civil and conflict-averse in public, but nasty and gossipy in group chats), we sure as hell seem to have a double standard when treating the people who run the single most important organization in the spread of good quizbowl today and have allowed people to make an actual living promoting this game. These people have done a hell of a lot more for quizbowl than almost any of us have and yet we give them the harshest treatment, and I find this extremely wrong. We should show some goddamn respect.

As far as the tone thing goes, both of you are wrong.

Will, the line about "we should show some goddamn respect" is dumb, rhetorically dubious, almost certainly against board rules, and makes me just a bit low-key worried about why you got that "Authoritarian personality" power at CO a few years ago. The reason Conor's post should be more respectful is because good-faith people deserve respect, not because NAQT has earned some right not to be critiqued for things they do wrong. That's a stupid right for people to earn, and is completely anathema to how people who the community built itself out of something far lesser than it is today (in fact, this sort of reasoning was consistently employed by the people who most centrally OPPOSED that project).

Suggesting that NAQT has earned any sort of right to not be criticized is not what anybody is doing here. For me to do such would indeed be bizarre because, in fact, I have posted multiple criticisms of NAQT not just above, but in the past as well, and indeed specifically regarding the diligence issue Chris identifies! So, I don't exactly appreciate being labeled as some sort of dissent-crusher suggesting that we all bow down and worship at Jonah's feet. But I do find the community's collective hand-wringing over how people get treated to look pretty bad considering that multiple prominent members of the community were publicly declaring Conor's initial post to be excellent, when in fact it represents the absolute worst excesses of quizbowl rhetoric. And yes, I do think that people should be accorded more respect and good-faith consideration when they not only have a consistent pattern of contributions to the community, but in fact derive their entire livelihoods from its existence.

Aside from that, though, Chris is very correct that there ought to be more transparency, though obviously we can't expect to be privy to absolutely everything going on in NAQT's internal conversations. We can certainly have empathy for the fact that corporate communications do tend to result in issues like this getting lost in a flurry of emails while also maintaining and repeatedly articulating a firm stance that this is not really something that a professional organization dedicated to quizbowl should be doing.

Periplus of the Erythraean Sea wrote: ↑

Mon Apr 08, 2019 11:33 am

As Mike pointed out, for a community where we spend so much time self-flagellating about how people often treat each other (which, incidentally, seems to result in these same people being more civil and conflict-averse in public, but nasty and gossipy in group chats), we sure as hell seem to have a double standard when treating the people who run the single most important organization in the spread of good quizbowl today and have allowed people to make an actual living promoting this game. These people have done a hell of a lot more for quizbowl than almost any of us have and yet we give them the harshest treatment, and I find this extremely wrong. We should show some goddamn respect.

You have this backwards--this should make NAQT more accountable, not less. I have high expectations for NAQT because I respect them, and hold them in high regard. Sure, the tone of Conor's post may have been over the top, but don't you want the most influential organization that spreads good quizbowl to also be using best practices? And isn't it particularly disappointing when they don't?

Cheynem wrote: ↑

Mon Apr 08, 2019 12:50 pm

I assume in many cases NAQT's corporate structure may complicate responses and decisions--(I'm speculating here, I have no idea)--even simple responses or solutions require presumably lots of communication and ink spilled, which would probably explain why they produce so many boilerplate and "we'll get back to you" responses.

I am not an NAQT member, but I assure you, it does.

DumbJaques wrote: ↑

Mon Apr 08, 2019 12:37 pm

[...]
That said, I think it's a fair to quite sincerely wonder what a productive form of engagement would be. NAQT's leadership regularly fails to return emails. They refuse to bring in people who could help them seed ICT (even when the new format so critically depends on seeding!!), even though this criticism has been going on for like a decade. [If I'm wrong and you did bring people in, all I can say is Jesus, find better people, because it sure looks suspiciously similar to a pure D-value ranking...] Despite Seth taking an active role in the discussion about ICT format, it doesn't seem like Cody's excellent points in the thread were listened to at all. Instead, we got Chad Kubichek making smarmy comments about how carrying over prelim losses against unshared opponents was the One True Way all tournaments should be run (come on, Chad Kubichek!).


I do understand that, from a strictly corporate perspective, ICT isn't a moneymaker (though I would absolutely argue that it is nonetheless an important project for NAQT, and that they end up deriving plenty of value from it - I completely reject the suggestion that it's an act of charity).
And I want to make clear that I DO think NAQT cares - Joel Gluskin, for instance, specifically sought me out to talk about the one fix NAQT did make to the schedule (adding a tiebreaker out of playoffs); NAQT does deserve recognition for doing that, because it was important (though I also got the impression that "it was self-evidently important" played more of a role there than "the community really wanted this").

[...]

We, on the other hand, need to take ownership of the fact that plenty of our griping has been overly caustic (and even at times unfair), and that it's easy to lose perspective that NAQT has a much larger set of projects to oversee (which we all agree are worth supporting). We can also do better. And yet, I don't have much of a reasonable expectation that being more reasonable will have any impact on the at times blatant disregard R. and Chad and others at NAQT have shown the community. I hope that changes, because it seems like there's quite a united voice about the ICT issue. This is a golden opportunity for NAQT, and for us, to take a step forward. Let's hope we're both up to the task.

I'm disappointed that you (I may very uncharitably say of all people, although of course graduate students and corporations should be held to different standards) seriously have no sympathy that stuff sometimes just gets lost in the shuffle.

I'll give you an example. When Jeff made this post, I was miffed that they touted the Advocate program when I knew damn well that there had been no behind-the-scenes movement on improving it. So I emailed Emily Pike. When she said "get back to me in [x amount of time]," I literally made a note in my planner for that amount of time saying "bother Emily," and when that day was up I did so. We were eventually pretty productive on the issue. Should that have been necessary? Maybe not, but you have to realize that you're dealing with human beings who have many competing priorities. As has already been pointed out, college quizbowl is surely not at the top of NAQT's priority list. Being able to be gently persistent and on top of your stuff is a good life skill.

EDIT: Speaking of which, Conor, although I see that Jonah's already seen the thread, it would have also been productive to email NAQT with your feedback, given that they literally have a contact email for that purpose. Maybe you did that, I don't know. I understand that posting with righteous anger is fun (oh boy, do I understand), but that's generally how adults communicate their concerns with each other. They also know that 95% of their customers will never go on this site or know it exists, and their digital public face to those people is primarily through their regular social media.

Edit again:

Cheynem wrote: ↑

Mon Apr 08, 2019 12:50 pm

I am not sure if like a community relations person might help,

Shouldn't this already be the communications person's job?

Alex, I don't think "harshest criticism" and "harshest treatment" are the same thing. I agree that NAQT should be held to higher standards. This does not mean they should be subjected to our most abusive rhetoric.

Periplus of the Erythraean Sea wrote: ↑

Mon Apr 08, 2019 2:04 pm

Alex, I don't think "harshest criticism" and "harshest treatment" are the same thing. I agree that NAQT should be held to higher standards. This does not mean they should be subjected to our most abusive rhetoric.

Sure, and I appreciate your succinct clarification. I, too, have long been uncomfortable with peoples' incredibly bad faith while dealing with NAQT (since Chris brought up the Matt Bruce thing, I'll link the post that I made to that exact effect).

I think that NAQT should be held to a much higher standard because of its position in the community, and instead it is often let off the hook when changes that need to happen do not happen.

The discussions we are having about overall record should not be happening because NAQT should have changed its policy two years, six years, ten years, even fourteen years ago. A little heated rhetoric is more than justified when the premier quizbowl organization is running a national championship that unfairly jobs teams out of the finish they deserved. Instead, NAQT again used the cover of their summer meeting to avoid making necessary changes during the year when the issue came to a head. And, unfortunately, that's the status quo with NAQT that I've learned to (begrudgingly) accept as the best they'll do for these types of issues.*

And, yeah, the alphabet tiebreaker *is* fucking stupid. That doesn't mean the person or people that implemented it are stupid. It means they didn't think through the consequences of the policy, which is a major oversight on their part and which I'm sure someone feels bad about (because that's the natural response when you're invested in tournament direction). Fortunately, this particular policy probably didn't screw over anyone too badly this year -- but it will certainly do so if left in place for the future. (As Joel notes in the other thread, this is being changed and will not be an issue in the future.)

* I'm not certain I've said this publicly before, but there are different sides to the NAQT experience. And while I have vociferously (sometimes inappropriately so) disagreed with NAQT on any number of issues, they are a great organization to work with as a staffer, and always make you feel valued. So, they know how to excel in some necessary areas -- and I wish they did the same in others.

I just want to let the community know that I've read this thread (and the original ICT format one) and we have started internal discussion within NAQT about these (well-founded) community concerns. We won't have a full statement tonight (R. and Emily are on the road back to the Twin Cities; I am about to leave the Hyatt for O'Hare), but I will have more to say during the day tomorrow. Please feel free to contact me directly about any of the issues in any of these threads.

Edit: this will be Wednesday, not Tuesday.

I'm sorry if my tone came off too harsh to people, and especially if it detracted from actual discussion of the issues. I don't think it was unjustified, but I agree that it might have been pragmatically better, if only to avoid a discussion like this one overshadowing the actual discussion I was trying to start.

I tried to be reasonable and not speak too strongly about the carrying-over-prelim-brackets game (and re-reading my post, I think I achieved that goal) but the alphabet thing was honestly just so absurd I felt it was necessary to use that tone to highlight the degree of the absurdity. Like, we all joke about the alphabet being used as Tiebreaker Number 73 or whatever, but the fact that it actually happened at a national tournament was just mind-boggling.

[Edit to add, since I was going to post this in the other thread but this one seemed more appropriate:]

I also don't like the insinuation that just because NAQT forgot about something or overlooked it that it's less serious than if they had intentionally included it in bad faith. Of course NAQT is not going to deliberately use a tournament format that they believe to be inferior! They are, however, a major national organization with at least a double-digit number of members, who manage to successfully put on a (very, very well run!) 352-team tournament every year. I don't think it's inaccurate or unfair to describe an organization of that size failing to recognize that the two superplayoff brackets are not equal as "fucking bush league", when ten nobodies on the Discord figured it out as soon as someone posted the screenshot of the tiebreaking procedure -- because it's 100% bush league for an organization of NAQT's stature, size, and yes, organizational prowess, to overlook something that easy.

[Edit over.]

I also certainly didn't want people to get the sense that I thought NAQT as an organization was acting in bad faith or anything -- I don't think I ever really alleged that, and I deliberately left "NAQT" out of the title, instead focusing on "the ICT format". NAQT does a lot of good things and, while I might take issue with their chosen format, their actual in-tournament logistics consistently knock it out of the park (seriously, when was the last time an NAQT national was delayed by more than 15 minutes?). I think it's unreasonable for people to take my directed criticism of a specific facet of a specific function of NAQT and extrapolate that into my acting in bad faith towards NAQT as a whole.

DumbJaques wrote: ↑

Mon Apr 08, 2019 12:37 pm

jonah wrote: ↑

Mon Apr 08, 2019 8:47 am

I don't remember hearing any issues about the alphabetical aspect last year, which is why—to the best of my knowledge—we didn't even contemplate changing it.

In before this thread inevitably degenerates horribly to say that, at least as far as I was explained things by R. and others last year, the alphabet rule was not actually the way ties were broken out of the prelims last year. In 2018, when OSU was 4-1 (loss to Chicago B) and Berkeley B was also 4-1 (loss to OSU), we were told the tie was broken on ppg (Berkeley had a total of like, 30 more points than we did over the course of the prelims). I can assure you that, had I understood the tie to have been broken BY ALPHABET, my response would have been *rather different.*

Like Jonah, I can't remember hearing anything about alphabetical tiebreakers at the 2018 ICT (specifically I can't remember anyone pointing them out/complaining about them). I went back and looked up the 2018 ICT DI format. If you go to the "Round 6 Tiebreaker" section, you can see that the listed tiebreaker was in fact alphabetical order of team name used on the schedule. I don't know if R. realized when confronted with the 2018 OSU/Berkeley B prelim record tie that this was a bad idea and switched on the fly to using a statistical tiebreaker, but then didn't make a note to update the DI format sheet to reflect this realization, or what happened there.

DumbJaques wrote:I actually referenced this explicit issue (breaking on ppg when the 1/2 seed matters so much) in the post I made about ICT format leading up to this year, but it's not surprising to me that Jonah (and other folks at NAQT) didn't see it. They care what we think, but they also don't care *that* much. NAQT's corporate culture is awful about diligence in a number of ways, and while I do see they're making efforts, I also have tons of emails where R. Hentzel, or another senior NAQT executive, promises to get back to me (sometimes on something extremely important!) and then literally never follows up. I'm not trying to be vitriolic here; I'm certainly grateful for the work NAQT does, I've been particularly vocal about endorsing areas where they've listened and made great changes (the clock for SCT/ICT being an excellent example). But that doesn't change that they have a real issue with responsiveness in general and seriously listening to feedback about SCT/ICT in particular.

I actually did see this post (and the rest of that thread) back in February. In general my impression is that Andrew H., Jeff, Jonah, and I all pay attention to hsqb threads discussing NAQT in the context of collegiate quizbowl. I can't speak for whether any of them (or any other members) read through this thread at the time, but I'd pretty surprised if I was the only one.

Speaking for myself, I somehow did not manage to extract the important point Chris raised in that post regarding the importance of deciding who is first and who is second coming out of a prelim bracket. I guess I was focused on various other points that were being discussed in the thread at that time.

DumbJaques wrote:That said, I think it's a fair to quite sincerely wonder what a productive form of engagement would be. NAQT's leadership regularly fails to return emails. They refuse to bring in people who could help them seed ICT (even when the new format so critically depends on seeding!!), even though this criticism has been going on for like a decade. [If I'm wrong and you did bring people in, all I can say is Jesus, find better people, because it sure looks suspiciously similar to a pure D-value ranking...] Despite Seth taking an active role in the discussion about ICT format, it doesn't seem like Cody's excellent points in the thread were listened to at all. Instead, we got Chad Kubichek making smarmy comments about how carrying over prelim losses against unshared opponents was the One True Way all tournaments should be run (come on, Chad Kubichek!).

I know that NAQT has dropped the ball plenty of times on returning emails, but I nevertheless want to encourage people to incorporate "sending email to NAQT" into their portfolio of strategies for engagement with NAQT. Emails to various naqt addresses are almost certainly the best way of making sure several members see your message quickly. And you don't have to limit yourself to one mode of engagement! Feel free to send an email and post here, or discuss on the Discord and also send an email, etc.

Jeff has definitely consulted with community members regarding ICT seeding at least a few times in recent years. I don't know off the top of my head which years he has/has not done this; I don't know if he did it this year. I do know that this year's DI seeding was not a pure D-value ranking—I believe it was based on the most recent community poll, the D-values, and the A-values from ACF Regionals (to help a bit with comparisons for teams that hosted SCT and thus did not play at all or did not play at full strength).

I really apologize if I wasn't clear when I posted in the discussion about ICT format, but I meant to indicate that we are going to reassess SCT placement policy and ICT ranking policy this summer. I assure you that the points raised in that thread will be discussed as part of that reassessment. (And I will make sure to go back and read through the thread again when the time comes, to make sure I don't miss anything important that I didn't catch my first time through.)