The vi-c blinded violins shootout - POLL to determine next steps - HAVE YOUR SAY!

[Garry]

Ok, so we had the VI-C blinded violins shootout, the results of which are here. There were many contributions, and it generated a lot of interest - thanks to everyone who participated.

The question now is how we can best capitalise on this to really help each other in our purchasing decisions, and to keep it fresh to incorporate future libraries?

I have a suggestion: that we create a database comprising only 1 of each of the best representations of each library. This is not a contest of libraries, but we'll need a fair and transparent way to identify which entry should represent each library. We will of course iterate on the rules, using the input of forum memkmnbers, but here's a provisional overview:

RULES OVERVIEW:

• We will again use 'Saxer's Seven': a MIDI file which contains 7 melodic lines with different challenges for the library to meet
• Anyone (developers included) can submit a version of the library
• There will be 2 categories: library alone (can only include anything within the library), and library-plus (can include elements from other plugins, but all details must be stated)
• We'll take a poll to determine which entry is to represent each library. Note this will NOT be blinded: this is not the purpose this time. Also note: this is NOT a competition across libraries (also, not the purpose this time). The polling is only to get the majority opinion of which entry best represents a library and should enter the database. The winning entrant wins kudos and fame amongst the VI-C community beyond their wildest dreams!
• The library will then be stored within the database, and can be updated as newer libraries come along. Anyone can access the database at any time to listen to each library performing the same lines.
• Each entry to include a screenshot of the CC data to help others get an idea of working with that library (credit to @brek) and their general impressions of the library (strengths and weaknesses) (credit to @pipedr)

THE PURPOSE: the aim is to create a database with all major violin libraries, which allows users to review a library and compare it with another, on the basis of having the SAME melodic lines. This will address the concern that some libraries may not have been represented at their best in the blind shootout. Here, our aim is not to do this blind, but to generate the best possible representation. Then, if you already have library A, and are thinking of buying library B, you can go to this database, and hear how it sounds, being able to directly compare it performing the same lines - something that is currently not possible, with all developers having their own individual demos; these are useful too, but they make comparison difficult.

Of course, there are many factors that go into whether to purchase a library (workflow, number/quality of articulations, price, developer, etc), and this will NOT address these, nor is it intended to. It will however, give you an idea of the comparison of timbre between libraries: likely a key, (though not the only) factor in your purchase.

If you think this is useful, please vote in the poll above; based on the results of the poll, we'll go ahead and create the database, or not, as the forum decides.

Click to read more...

 

[brek]

I love what you've been doing with this Garry. Pesronally, I'd vote to just leave any competition out of it. Whether between developers or MIDI manipulators. If there are 3 versions of Saxers Seven from one library, I'd like to hear all of them. Maybe things could eventually get out of hand but that seems like a a bridge you cross upon arrival. Maybe we also request people include a screenshot the piano roll/CC lanes so the rest of the community can get an idea of working with that library.

Just my two cents.

 

[Garry]

I love what you've been doing with this Garry. Pesronally, I'd vote to just leave any competition out of it. Whether between developers or MIDI manipulators. If there are 3 versions of Saxers Seven from one library, I'd like to hear all of them. Maybe things could eventually get out of hand but that seems like a a bridge you cross upon arrival. Maybe we also request people include a screenshot the piano roll/CC lanes so the rest of the community can get an idea of working with that library.

Just my two cents.

I think that's a perfectly reasonable view, but I'd just add: isn't that what we just did? We now already have a database with all of the entries - but with 78 entries, there were 2 main criticisms:
- many people felt this was unwieldy
- without any quality control, and wide variability in experience/ability or just time available, some of the entries didn't represent libraries as well as others, and so could provide an unfair reflection of what a library is capable of.
So this is to improve on what we already have, incorporate the forum's feedback, and generate something that stands the test of time and is useful to the community.

For me, any further competition element is just for 2 mains reasons: (i) it should be fun - we're not trying to be definitive/scientific here, the competition element encourages people to submit, because there's a fun aspect to having 'your' entry chosen by your peers; and (ii) if we agree we need just 1 representative per library, then it's the only fair way of doing it. I appreciate you would want to hear all of them, but in this way, you WOULD hear all of them: I anticipate it going something like this:
- Library A: 3 entries: you hear all of them, poll to select the best: highest voted of the 3 goes into database.
- Library B: same,
- Library C: etc...

There are other ways of doing the selecting: e.g.:
- a panel decides, but this also creates many problems: who is on the panel, how do we decide who to include/exclude; this becomes not the VI-C database that we all own/share, but the preferences of select few; what to do when the panel don't agree; if this is done by voting, why not everyone vote?
- first one submitted per library (arbitrary, and could end up with poor entries)
Are there any other suggestions of how to select? We can go with other proposals if there are other equitable and transparent ways of doing it, but a simple open votes seems to me to be the fairest and most open.

Then, with the database narrowed down to just 1 entry each, it's something more usable: people can refer to it, with 1 entry per library, knowing that that entry has been vetted, and is viewed by the majority of the community to be the best example of it.

What do you think - does that sound useful/reasonable?

 

[Garry]

Maybe we also request people include a screenshot the piano roll/CC lanes so the rest of the community can get an idea of working with that library.

Great idea, thanks. I've added it to the opening description.

 

[Garry]

Also, to get us started, with the permission of the original entrants (on a case-by-case basis) we could use the existing entries (anonymised or not, as the authors prefer), we could use the entries we currently have. For CSS for example, we clearly already have an example of that library that many people liked. For some of the Spitfire libraries however, many feel these were poorly represented in terms of the quality that the libraries are capable of, so if we include as a voting option for each library 'none of the entries meets required standard', and this gets the most votes, then we know that for that library, we need additional entries before any can enter the database and have the library represented. That way, where we already have good entries, we can use them and quickly build up the database, and where none of the entries satisfied what people felt the library is capable of, we exclude these, and ask for additional submissions.

How does that sound?

 

[Garry]

For those that are voting ‘yes, and I would be likely to contribute an entry’, since this round is open, not blinded, please could you write in the comments which library.

 

[Garry]

Currently 70:30 in favour of us establishing the database. For those voting in favour, don't forget to indicate which library you might contribute.

 

[Garry]

More than 30 votes, still with 70:30 in favour, which is great, but still no one listing libraries they'll plan to cover (currently 16 people saying they're likely to submit an entry). Remember, this time around it will be an open entry, not blinded, so we need to see how many of each we'll have for each library, before we see if this is worth doing or not.

 

[JeeTee]

but still no one listing libraries they'll plan to cover

Hollywood Strings.

 

[awaey]

8Dio Anthology Strings + CineStrings CORE

 

[eli0s]

Garry, feel free to use any of the tracks I contributed in the previous test.
Thank you!

 

[pipedr]

I felt that the first 78 tracks were very useful, very educational—not just a shopping tool.

I would suggest we also give the composers the opportunity to leave some comments on how they made the track. How they approached keyswitches and articulation, CC data, layering (if done), effects, etc. Maybe their general impression of the library, strengths and weaknesses. That would be great info for others trying to learn from their experience with the library.

I think it’s also helpful to collate people’s reactions to the tracks. Maybe not so much to “rank” a library but to see what people think is a more or less successful interpretation of a phrase. With that in mind, I would also be interested to listen to lower rated interpretations even in the same library (for example the multiple Josh Bell submissions). Sometimes, something sounds good until you realize it can be done better.

Thanks for organizing this!

 

[yhomas]

Fair comparative listening without matched levels is not possible. It would be great if all the examples could get mastered to the same average loudness level.

 

[MatFluor]

The same principle would be nice for all other instruments (not just Violins).

If we want to do something like that, I happily create a website/webapp for these comparisons to host and make the database publicly accessible.

 

[Garry]

I felt that the first 78 tracks were very useful, very educational—not just a shopping tool.

I would suggest we also give the composers the opportunity to leave some comments on how they made the track. How they approached keyswitches and articulation, CC data, layering (if done), effects, etc. Maybe their general impression of the library, strengths and weaknesses. That would be great info for others trying to learn from their experience with the library.

I think it’s also helpful to collate people’s reactions to the tracks. Maybe not so much to “rank” a library but to see what people think is a more or less successful interpretation of a phrase. With that in mind, I would also be interested to listen to lower rated interpretations even in the same library (for example the multiple Josh Bell submissions). Sometimes, something sounds good until you realize it can be done better.

Thanks for organizing this!

That’s a great point to collate the responses. Once we start, I’ll create a separate thread for each library, and that way, a link to it will serve this purpose.

Note also that you WILL get to hear all versions submitted for all libraries, as before we poll for the best one, you’ll get to hear them to choose.

 

[Garry]

Fair comparative listening without matched levels is not possible. It would be great if all the examples could get mastered to the same average loudness level.

Is there an objective (and quick) way to do this? In the blinded competition, someone did normalize them all, but I don’t know the best way to do this, unless it’s just going through and adjusting individually?

 

[Garry]

The same principle would be nice for all other instruments (not just Violins).

If we want to do something like that, I happily create a website/webapp for these comparisons to host and make the database publicly accessible.

That would be awesome Matt, thanks. I’ll get back to you about that once we have all the files in.

Yes, the idea is to extend to other instruments, assuming the violins is successful, so this is hopefully just a starting point.

 

[Garry]

Great response!

After the first 50 votes, we have 78% in favor, so I think we go ahead. In addition, thank you to the 20 people so far who have indicated that they are likely to participate by submitting an entry. That should be enough to cover the main libraries, so thanks for your participation.

Later today, I’ll send out instructions as to how we get started.

 

[JeeTee]

Is there an objective (and quick) way to do this? In the blinded competition, someone did normalize them all, but I don’t know the best way to do this, unless it’s just going through and adjusting individually?

If I were doing this, I'd do it manually. It's quite possible that different submissions would have varying dynamic ranges. When normalised, those with smaller dynamic ranges might appear louder. So I think 'sounding comparable' (manual adjustment) might be the way to go.

 

[Garry]

Entries are now open - please see the new thread here.