Mix Challenge

Hello everyone,


today, I would like to talk about some upcoming changes regarding a core game mechanic of the Songwriting Competition. The biggest point of contention has always been the final step of the game - the voting process. Up until this point, all participants were asked to give feedback to each other and cast a vote for everyone, until we exceed 20 participants. This resulted in growing dissatisfaction, because of felt unfair behavior towards the "last spots" on the personal vote list. However, without casting a vote, there can not be a Winners Podium, and therefore no bonus licenses addressed.

After gathering a lot of feedback, and some testing behind the scenes, I might have found viable solutions. They will certainly need some adaption time and this will also not be free of flaws, as you can read in a moment. However, I am confident that this will result in positive feedback, and more engagement in the long run.

Let's get to it, shall we...




SONGWRITING COMPETITION


Changes to the Voting Process game mechanic

I would like to introduce a "Tiered System" on casting your vote, depending on the participant amount. Technically, we can introduce a "Top 3 vote" system even at low participation. Yet in order for this to work, we need at least 7 attendees. While testing with old score sheets, I realized however that some participants might be completely left out. So this is not ideal, and a point we could further discuss. Personally, I would like to skip this tier, and start with the "09 or more participant" one.

Tiered Voting System Idea:

• 01 to 06 participants -- no tiered voting system, everyone please casts a vote for all participants
• 07 to 08 participants -- vote for personal Top 3 entries (voting for 42,85% to 37,50% of the participants)
• 09 to 14 participants -- vote for personal Top 5 entries (voting for 55,55% to 35,71% of the participants)
• 15 to 20 participants -- vote for personal Top 7 entries (voting for 46,67% to 35,00% of the participants)
• 21 participants or more -- vote for personal Top 10 entries

On average, I try to introduce a participation amount that equals about one third to half the attendees. I am currently unsure if I will stick to the "10pts for 01st Place" point mechanic if we exceed 11 participants, like this has been the case for previous games. This deserves more testing with the tiered system.



Feedback mechanic

Another big point of criticism, was that giving feedback to everyone felt jarring. Some might not even be good at writing constructive criticism, and or they don't know what to do. Others might not feel comfortable, especially if they're new to the community. As somebody that joined the game myself to give feedback to everyone (as part of the "Virtual Client" special game mechanic), I can also confirm a certain time investment.

So to take the pressure out of the "final assessment", and thanks to plenty of feedback from private conversations, I came up with the following.

Feedback writing idea:

• Reflect on your experience with this game (in 2-3 paragraphs tops), then...
• either -- give some final thoughts on at least 3, but no more than 5 entries that stood out to you (general thoughts)
• or -- give constructive feedback for at least 3, but no more than 5 entries that could see possible improvements (more technical - especially for, in your opinion, lower ranked productions)

Final thoughts?

I am aware, that this endeavor might definitely result in certain drawbacks. For example: the final feedback won't cover everyone. And some fine-tuning will be needed. However, one can still participate in giving feedback during the initial submission process (late-stage WIP until final version), if you feel comfortable to do so.

I do plan to make both the feedback and voting process mandatory from this point forward. Previously, only the voting was mandatory. I am confident however, that this lowered the entry bar quite significantly. And I am also certain, that these changes will feel less jarring and/or frustrating overall, which in turn could also mean, that we see more participants in the the long run.



You might also be asking: "But what about the current Bonus Point Mechanic?"

Good question. I am considering adjusting the one or another thing. The 01pts per participant "Voting Process" does feel obsolete with the "Tiered Voting System". Especially considering that giving feedback and casting a vote will be mandatory, and will therefore be the only disqualification criteria. Previously, you could still make a statement, but only voting for half of the participants. However, not many people did that. The incentive was "feedback for all, stay high on the score sheet".

I am considering to at least adjust the bonus points for documenting ones production. With the last three Songwriting Competitions (SWC059/July 2022 to SWC061/September 2022), I noticed that there is a recurring debate on "what should even be listed?", which is a very good question actually. Like... should you really list each individual FX module that you used? Or if you utilized 30 different sample packages, then those as well? Some of us have a quite extensive setup at this point. Can we maybe simplify this, like with the Mix(ing) Challenge, that you should rather focus on one or two sounds you've enjoyed sculpting and how you got there? Or at least mention if you used "sampled instruments" vs "real instruments"? This is definitely something to talk about.

I still think that "credit where credit is due" is absolutely mandatory. If you collaborated with somebody, say for recording a part, vocals, lyrics, or even mixing, that should be pointed out. Something that is often ignored, or plain downright forgotten.

This is where a Rule Set Addendum would definitely help. Time management is an issue though.




We still got more than 14 days until the current running Songwriting Competition ends (SWC062 / October 2022). However, I'd already love to try this new mechanic with SWC062 October 2022, and then fine-tune things as we go.

Please let me know your thoughts down below.



Thanks for reading

I can't estimate how the change in the voting system would influence the game, so I'm neutral on that part.

Mister Fox wrote: ↑

Sat Oct 08, 2022 08:46 CEST

Feedback mechanic
...
So to take the pressure out of the "final assessment", and thanks to plenty of feedback from private conversations, I came up with the following.

Feedback writing idea:

• Reflect on your experience with this game (in 2-3 paragraphs tops), then...
• either -- give some final thoughts on at least 3, but no more than 5 entries that stood out to you (general thoughts)
• or -- give constructive feedback for at least 3, but no more than 5 entries that could see possible improvements (more technical - especially for, in your opinion, lower ranked productions)

Not having to comment on every entry would be a huge relief for me. I don't feel comfortable to comment on possible improvements yet. I am still learning the basics myself, so I'd prefer to point out what my favorite entries did well, what makes them stand out for me. I like this idea.

Mister Fox wrote: ↑

Sat Oct 08, 2022 08:46 CEST

Final thoughts?
You might also be asking: "But what about the current Bonus Point Mechanic?"
...
I am considering to at least adjust the bonus points for documenting ones production. With the last three Songwriting Competitions (SWC059/July 2022 to SWC061/September 2022), I noticed that there is a recurring debate on "what should even be listed?", which is a very good question actually. Like... should you really list each individual FX module that you used? Or if you utilized 30 different sample packages, then those as well? Some of us have a quite extensive setup at this point. Can we maybe simplify this, like with the Mix(ing) Challenge, that you should rather focus on one or two sounds you've enjoyed sculpting and how you got there? Or at least mention if you used "sampled instruments" vs "real instruments"? This is definitely something to talk about.

I still think that "credit where credit is due" is absolutely mandatory. If you collaborated with somebody, say for recording a part, vocals, lyrics, or even mixing, that should be pointed out. Something that is often ignored, or plain downright forgotten.

I think the documentation should not be too rigid. As long as there is some insight on the writing or production process of the participant, it should be considered as enough.

At the moment, I don't care too much about what plugin someone used on which channel. I'm more interested what their idea behind the song was, why they chose or discarded an instrument, how they built the structure of the song and what the lyrics are (if there are any). As long as one of these questions is answered, I'm satisfied. And of course I can learn from listed FX-Modules as well, especially when the participant explains why and where they used it.

Mister Fox wrote: ↑

Sat Oct 08, 2022 08:46 CEST

Changes to the Voting Process game mechanic
...
While testing with old score sheets, I realized however that some participants might be completely left out. So this is not ideal, and a point we could further discuss. Personally, I would like to skip this tier, and start with the "09 or more participant" one.

Just an afterthought:
When I started to enter this contest for the first time this May, I was prepared to be voted on the last place. The same goes for my entries in June and July. Through the votes and the feedback for those entries, I learned that the other participants often rated my entry higher than I would have rated it myself. This was a huge motivation to keep going.

So maybe - not as a rule, but as a kind of "best practice" - it could be established that participants who enter for the first time or have not gotten points or feedback for the last two times they participated get some mandatory encouraging feedback the next time they enter? Maybe there could be some bonus point mechanic for this?

Great afterthought. However, this is already encouraged with the way the game is set up, and now even more emphasized with the overhauled "writing feedback" idea.

I see the current idea as a way to simplify things. Participants can set a good example by commenting on entries that would usually not get a comment. This has always been the case, and this is why I stepped in to comment in a couple of recent games. Adding more bonus points would mean more rule adjustments, and then things are open to interpretation again.

Well thought out and executed adjustments. Definitely leans towards the improvment of what already is a great free flowing event.

I think the voting worked as intended.

For me (as a non-native speaker) it was easier to give feedback only to a few contestants. Is there a reason to limit the number of given feedback to a maximum of 5? As far as I can see, feedback is very much appreciated by most contestants, and participants who feel able to comment on more than 5 entries should be free to do so.

I don't know if I should write here, but maybe the these questions are belonghere:

I would like to comment on the new scoring system for the songwriting competition

I think the current 10-9-8-7-6-0-0-0-0 system is not really fair.

There are examples of fairer distribution from many competitions, e.g. Formula One back in the day:
8-6-4-3-2-1
9-6-4-3-2-1
10-6-4-3-2-1
Of course, it can be varied according to the number of participants.

I will show an example of the main error of the 10-9-8-7-6-0 system:
Player 1: 10, 10, 10, 10, 10, 10, 10, 10, 0 = 80
Player 2: 9, 9, 9, 9, 9, 9, 9, 9, 9 = 81

(imagine that someone wins every race, but fails to finish in the last one, so he is not the world champion - sounds pretty unfair. That's why in other sports give the winner (or the podium) more points. -- and the more bigger jump between the 5th and 6th place and the one behind them is also unfair.

In the example above, according to everyone, the best song was Player 1's (he won the competition outright), but if someone (for whatever reason) doesn't give him a point, that person has much more influence on the competition than in the previous (old) system.
the possibility of influencing is even greater due to open voting (which many did not like over the years) than in the past -
In the above case, for example, the majority decision does not apply.

That's why I think it is much fairer to reward the podium with higher points (like in other sports - olympic points method, or Formula One) and (at least as important) that there will not be a significant difference between those who score points and those who do not (6 pt vs 0 pt -- or 1pt vs 0 pt).

Staying with the above example, according to the scoring calculations I have proposed and already tried in other sports, the community will prevails better - it is fairer.

I need to think about this, do more tests.


However, two things already stand out.

1) so far we never had a "full row of 10s" or "full row on 9s" in a game, and considering the participation and individual opinions (music is subjective)/interpretation of the genre, I don't see that this will happen anytime soon

2) it would make the Bonus Point mechanic useless, as the top spots can basically ignore all rules regarding the filename template, loudness and documentation, yet could still win by running away with the top scores. In this case, there would be no learning aspect involved anymore.


I understand your concerns on "fairness", but with going a "Gran Prix Score System" route, this isn't that "fair" either.
But again, this needs further testing.

Mister Fox wrote: ↑

Sat Nov 26, 2022 10:51 CET

I need to think about this, do more tests.
However, two things already stand out.

1) so far we never had a "full row of 10s" or "full row on 9s" in a game...
2) it would make the Bonus Point mechanic useless, ...

Thanks for the reply, yes i understand your arguments too.

1) of course, this (all 10 and 9) is an extreme case, but I would like to point out that the new system does not reward the podium finishers, and mainly - the difference between the competitors who receive and those who do not receive points is too big, which can cause manipulations (even if for most of us this "competition" is a good game)

2) turn it around and look at it from the back: the total bonus point can be 10, but with two fifth places you can get 12. Ergo, it's more worthwhile to make loud (loud always sounds better ) songs now for one or two better positions than to do everything properly
if the distribution of points were, say, 7-5-3-2-1, then the 10 bonus points would count more (!) - people would pay more attention.
(and if someone gets many first places, he probably has already learned enough to be able to pay attention to, say, the true peak, lufs etc. )

Again, I understand your arguments, just I wanted to write my opinion about the new mechanism and that it is no coincidence that in other sports use other scoring systems in such cases.

I would like to follow up on this conversation and from Discord and earlier.

I've heard your feedback on "writing up feedback". I also have nothing against a slight alteration to simply state "Please give feedback to at least three participants (you can provide feedback to as many participants as you'd like)" rather than the current "three to five". However, the biggest topic was the "Score Sheet" mechanic, and that a system as used in sports might eventually be better. Turns out, I already have enough data to simulate various scenarios.

And I am sorry to say this, but I will not further alter the current used points mechanic.



TL;DR Version:

• Points mechanic based upon concepts form several other audio communities, in use since 2017
• rock solid, simplified for participants (voting mechanic being less dreadful)
• "bonus point" mechanic for more suspense remains, "tie-breaker" mechanic available, room for expansion in case of more participants (tiered system)
• vote manipulation not possible, due to no "outside votes"
• "Sports" game mechanics would merely change the mid-field and introduce "runaways"
• In Summary: Points Mechanic will not change in the foreseeable future

THE BASELINE

Before I even started the Songwriting Competition, I went through various communities (IIRC, I took a dive at at least 8 different ones), just to see how these games are set up, and how their game mechanics would work. In the end, I decided on something as seen on KVR Audio's "Music Cafe Songwriting Contest", but with and added mandatory "feedback for everyone" mechanic. The idea was to encourage interaction, and giving feedback for possible improvements, and the "voting mechanic" was the most essential I could find. The system was then fine-tuned to add more "bonus points" as incentive to adhere to the given extra tasks (loudness, documentation, etc). And in most cases, they can make or break a position on the podium.

The "new game mechanic" came up, after several recurring conversations throughout the months regarding the "unfairness towards users that would get the last spots". So I spent a couple of days to find a suitable solution, which resulted in the "tiered system" in the above mentioned post.



UNDERSTANDING THE CONCERNS OF POSSIBLE MANIPULATION

This can definitely always happen. Doesn't matter which system we would use. May it be from various "sports", may it be the one from KVR, may it be the one from community XYZ that also only gives points to top 3 entries. It is similar to the infamous "Eurovision Befriended Country Vote". And the only way to prevent that in the future, is to call it out.


With that out of the way, let's look at the "original concept, vs the new concept...

The old concept forced participants to give a rank for everyone involved. While I can confirm that this indeed often punishing, it however also made it possible for certain users to actually score "higher" instead of just getting a handful of votes. Whether or not this is "more fair", lies within the eye of the beholder.


A closer look...

This is the chart from SWC061 with the "Old System"
As you can see, this chart covers all participants including distributed bonus points based upon certain met criteria (proper filename, not exceeding the loudness specs, providing suitable documentation). Please note that I've also withdrawn -2pts for one user in this particular game, since the material was not downloadable (technical limitation, the participant agreed to this deduction). Else, known rules applied.

Please also note that the bonus points for the "Voting Process" still exist on that list. One user decided to only give feedback and vote for 8 entries, therefore he only got 8 points, while everyone else got 11.

This alone has a significant influence on the final results. If the user in question would have given a ranking for all entries, then he would have also had 85pts like EsteveCorbera. Then the tie-breaker mechanic would have taken over. In that particular case, "Dear David John" had 2x11pts/1x10pts and EsteveCorbera had 0x11pts/2x10pts listed -- "Dear David John" would have made 4th. Now add the missing points for the "filename/file download" and "loudness specs", then "Dear David John" would have gone against "thenanocats". However, this time "thenanocats" would have had the higher spot on the chart, due to the fact that they had more 10pts on the score sheet.

You get the idea - since we had this mechanic in effect several times at this point.


Now let's look a the chart from SWC061, but with the "New System"

A couple of things to note here:

The points mechanic is now based upon a "Top 5" system, with points going from 10 down to 6 (so: 10, 9, 8, 7, 6). Additionally, and not shown on this chart (since we didn't have that mechanic in effect yet), you would now loose points by not reflecting on your experience during this particular month, and/or not giving feedback for three to five entries that stood out to you.

We already see a huge shift due to this change alone. While one could argue, that "votes aren't less fair now", the focus is now really on "the best entries". While with the previous chart, so called "pity votes" (which were mandatory) could influence the field completely. If we were to have a Winners Podium of 5 positions (which can happen for special events, or if we exceed 15 participants), then things suddenly get more interesting.

Now let us please look at "Dear David John"'s scores again. Would he have made it on the podium? Let me virtually add the 5pts he lost out on, and suddenly his final score would be 59pts. Then the answer would be a clear: yes. Even if "aurelien.c" had all bonus points as well (to get from 55pts to 58pts), this particular podium would have looked different. A very close call.

As you can see... even with the "new mechanic", it is still balanced out and full of suspense. And it can scale accordingly. (I've also tried this with the chart from SWC031 / March 2020).



BUT WHAT ABOUT A SCORING MECHANIC, AS IS COMMON IN VARIOUS SPORTS COMPETITIONS?

I've invested the time and actually created additional charts, based upon the "
Driver's Points from Mario Kart 7 (2011, Nintendo Wii) and Mario Kart 8 Deluxe (2017, Nintendo Switch). Something that the "Formula 1" also employs. Just as an example, because there are so many iterations of this, and I only have xyz amount of hours available during the day.


This is how it would look like, once more for SWC061.

Please do take note of the upper middle field on the score sheet. Here you see this system having the most influence, while the "more points for higher position" also introduces in IMHO very glaring issue: runaways.

The "runaways" will legit be zero affected by withdrawn bonus points. So even with withdrawing all bonus points from the 1st position, the participant in question would still make it on top spot. Which is only emphasized by utilizing the points mechanic from "Mario Kart 8 Deluxe". Since "rule violations" (as technical term) are not punished like with the Mix(ing) Challenge, this can spiral out of control if a user realizes "oh, I've been doing good in recent months, why even pay attention to detail?!".

On top of the fact, that I have to wrap my head around noting down the correct points. Not only because no user adheres to the ranking template, but I also do not have access to an automated system (that just as a side note).



Now in comparison, let's look at SWC063, and how the points would look in this case?

Please notice the following here:

The "Voting Process" saw a reduction of -1pt for two users this game, because of the lack of properly reflecting on one's experience for the month in question. And as you can see, the most notable difference is once more in the "mid field" and "runaways". However, the last position on this chart this time around, came from a nigh unanimous decision, that the entry was not sticking to the given premise.





THE BOTTOM LINE

• We have been using the "Top 10" points system since 2017 - it has been proven to be "rock solid".
• The mechanic fits the purpose of our community, is easy to handle (for me)/confirm by other participants, and has additional mechanics for "added suspense" (bonus points) and balancing out "ties" (tie-breaker mechanic).
• The mechanic is build in such a way, that it can adapt/scale, the more participants there are (see tiered system).
• Notice that the Songwriting Competition does not allow outside votes. Therefore vote manipulation is impossible. If vote manipulation is obvious (favor votes), this will be called out.
• The criticism of "unfair final places" and "I don't want to give as much feedback" has been addressed and tested since SWC062 (to positive response).
• A scoring system like with sports (e.g. Racing Grand Prix of various types) would IMHO only change the mid-field and introduce "runaways"

I do not see extreme or fringe cases in the foreseeable future. I therefore see no need for adjustments of the points mechanic, but I will address the "feedback mechanic" once SWC064 / December 2022 has ended.



Thanks for reading.