Confirmed users
142
edits
Abhishek/metabrainz GSoC2016Proposal: Difference between revisions
Jump to navigation
Jump to search
Abhishek/metabrainz GSoC2016Proposal (view source)
Revision as of 17:28, 21 March 2016
, 21 March 2016more details added
(more details added) |
(more details added) |
||
| Line 100: | Line 100: | ||
</p> | </p> | ||
<p>'''Phase I : | <p>'''Phase I : Getting Features and index creation''' <br> | ||
Getting features via AcousticBrainz and index creation for extracted features will be done in this phase. In AcousticBrainz have a lot of annotation data, including all of the information in MusicBrainz, which we can be accessed via the MusicBrainz IDs which are used in both projects, and also the generated annotation data. These features could be annotated texts, tags, descriptions, labels etc. It would also include low level and high level metadata. </p> | |||
<p> Some specific list of descriptors that would be a good idea to include in a search index: | |||
* bpm - could be used for finding songs with specific bpm. Also, can be used in making groups. | |||
* artist - user can search for a song/music based on artist name | |||
* tempo - used for searching. some user likes songs with hight tempo while other with low. | |||
* instrumentation - searching songs based on instrument type. Say 'guitar' | |||
* mood - searching songs based on mood. Get songs which have 'sad' or happy 'mood' | |||
* release - get songs based on release info, date etc. | |||
* track name - get music based on name. say 'scuse me' | |||
* key - get songs or music based on key. Can be useful for some musician users. | |||
* gender - get songs sung by male/female singer | |||
* speech/non speech - can be useful for grouping. | |||
* type of file - search by file type. get mp3 format of some track. Helpful in serving field queries | |||
* length - search by total length or duration of a music. Can be useful in identifying duplicates (will be discussed later) | |||
* language - get all 'english' songs | |||
* genre - search songs based on genre. 'rock', 'classic' etc. | |||
<br> | |||
If we have these features then it would simplify many searching and grouping tasks. It would also help in data exploration task. It would be able to support search for a track by its name, or even | |||
search for a release or artist to add all matching recordings to a class. search could also match recordings which match certain tags. For example, there are tags on recordings in MusicBrainz which look like genres. Would we be able to say "find all recordings tagged with 'rock' | |||
and add them to the rock class in our genre dataset". </p> | |||
<p>[[freesound.org|Freesound]] seems to be good platform for downloading and creating datasets. They also provides annotated data and low level and high level descriptors along with videos. These features wil be used in building and testing our search system. </p> | |||
<p> A demo code for previewing audio from freesound. </p> | <p> A demo code for previewing audio from freesound. </p> | ||
| Line 124: | Line 144: | ||
</source> | </source> | ||
<p> Features | <p> Features retrieved form AcousticBrainz server will be stored. These features will be indexed in the elastic search database.</p> | ||
<p> A sample code to store indexes in elastic search </p> | <p> A sample code to store indexes in elastic search </p> | ||
| Line 201: | Line 221: | ||
<p> Given a query audio, learn the vector representation of it and then find the similarity between the query vector and indexed audio vectors. More the similarity more is its relevance. Cosine similarity measure can be used to calculate relevance score. Features for audio could be tempo, bpm, average volume, genre, mood, fingerprint etc. </p> | <p> Given a query audio, learn the vector representation of it and then find the similarity between the query vector and indexed audio vectors. More the similarity more is its relevance. Cosine similarity measure can be used to calculate relevance score. Features for audio could be tempo, bpm, average volume, genre, mood, fingerprint etc. </p> | ||
<p> This could be done by using Gaia software already present. </p> | |||
<u> '''Spectrum analysis(Fingerprint) model''' </u> | <u> '''Spectrum analysis(Fingerprint) model''' </u> | ||