Exploring the World of Music Technology with AI

Studying Aims

By engaged on this mission, we stand to realize new technical abilities and an understanding of how AI algorithms may be carried out to construct revolutionary purposes. By the tip of this mission, we are going to:

1. Achieve an understanding of how artificial intelligence is employed in creating music. We are going to study the basic ideas and strategies used to coach AI fashions for music composition.
2. Learn to acquire and put together related musical information for AI mannequin coaching. We are going to uncover methods to collect .mp3 recordsdata and convert them into MIDI recordsdata, using instruments comparable to Spotify’s Primary Pitch.
3. We will even perceive the steps concerned in constructing an AI mannequin for music era. Additional, we are going to study concerning the mannequin structure appropriate for this process and its relevance and achieve hands-on expertise in coaching the mannequin, together with figuring out the variety of epochs and batch dimension.
4. We are going to spend time to find strategies to guage the efficiency of the educated mannequin. Then we are going to discover ways to analyze metrics and assess the standard of generated music items to gauge the mannequin’s effectiveness and establish areas for enchancment.
5. Lastly, we are going to discover the method of utilizing the educated AI mannequin to generate new musical compositions.

This text was revealed as part of the Data Science Blogathon.

Challenge Description

The aim of this mission is to discover the intriguing area of music era utilizing AI. We purpose to research how synthetic intelligence strategies create distinctive musical items. By leveraging machine studying algorithms, our goal is to coach an AI mannequin able to producing melodies and harmonies throughout numerous musical genres.

The mission’s focus is on gathering a various vary of musical information, particularly .mp3 recordsdata, which is able to function the muse for coaching the AI mannequin. These recordsdata will bear preprocessing to transform them into MIDI format utilizing specialised instruments like Spotify’s Basic Pitch. This conversion is important as MIDI recordsdata present a structured illustration of musical components that the AI mannequin can simply interpret.

The following part entails constructing the AI mannequin tailor-made for music era. Prepare the mannequin utilizing the ready MIDI information, aiming to seize underlying patterns and buildings current within the music.

Conduct the efficiency analysis to evaluate the mannequin’s proficiency. It will contain producing music samples and assessing their high quality to refine the method and improve the mannequin’s capacity to provide artistic music.

The ultimate end result of this mission would be the capacity to generate authentic compositions utilizing the educated AI mannequin. These compositions may be additional refined by way of post-processing strategies to counterpoint their musicality and coherence.

Drawback Assertion

The mission endeavours to deal with the problem of restricted accessibility to music composition instruments. Conventional strategies of music creation may be laborious and demand specialised data. Furthermore, producing recent and distinct musical ideas can pose a formidable problem. The purpose of this mission is to make use of synthetic intelligence to bypass these obstacles and provide a seamless answer for music era, even for non-musicians. By means of the event of an AI mannequin with the potential to compose melodies and harmonies, the mission goals to democratize the method of music creation, empowering musicians, hobbyists, and novices to unleash their artistic potential and craft distinctive compositions with ease.

A Transient Historical past of Music Technology Utilizing AI

The story of AI in making tunes goes again to the Nineteen Fifties, with the Illiac Suite for String Quartet being the primary tune made with a pc’s assist. Nevertheless, it’s solely in the previous couple of years that AI has actually began to shine on this space. At the moment, AI could make tunes of many varieties, from classical to pop, and even make tunes that duplicate the type of well-known musicians.

The present state of AI in making tunes could be very advance within the latest occasions. Not too long ago, Meta has introduced out a brand new AI-powered tune maker referred to as MusicGen. MusicGen, made on a powerful Transformer mannequin, can guess and make music components in the same approach to how language fashions guess the following letters in a sentence. It makes use of an audio tokenizer referred to as EnCodec to interrupt down audio information into smaller components for straightforward processing.

One of many particular options of MusicGen is its capacity to deal with each textual content descriptions and music cues on the identical time, leading to a clean mixture of creative expression. Utilizing a giant dataset of 20,000 hours of allowed music, ensuring its capacity to create tunes that join with listeners. Additional, corporations like OpenAI have made AI fashions like MuseNet and Jukin Media’s Jukin Composer that may make tunes in a variety of types and kinds. Furthermore, AI can now make tunes which might be virtually the identical as tunes made by people, making it a powerful device within the music world.

Moral Issues

Discussing the moral facets of AI-generated music is essential when exploring this discipline. One pertinent space of concern entails potential copyright and mental property infringements. Prepare AI fashions on in depth musical datasets, which might end in generated compositions bearing similarities to present works. It’s critical to respect copyright legal guidelines and attribute authentic artists appropriately to uphold honest practices.

Furthermore, the arrival of AI-generated music might disrupt the music trade, posing challenges for musicians in search of recognition in a panorama inundated with AI compositions. Putting a steadiness between using AI as a artistic device and safeguarding the creative individuality of human musicians is an important consideration.

Knowledge Assortment & Preparation

For the aim of this mission, we are going to attempt to generate some authentic instrumental music utilizing AI. Personally, I’m a giant fan of famend instrumental music channels like Fluidified, MusicLabChill, and FilFar on YouTube, which have wonderful tracks for all types of temper. Taking inspiration from these channels, we are going to try to generate music on comparable strains, which we are going to lastly share on YouTube.

To assemble the mandatory information for our mission, we deal with sourcing the related .mp3 recordsdata that align with our desired musical type. By means of in depth exploration of on-line platforms and web sites, we uncover authorized and freely obtainable instrumental music tracks. These tracks function invaluable belongings for our dataset, encompassing a various assortment of melodies and harmonies to counterpoint the coaching technique of our mannequin.

As soon as we’ve efficiently acquired the specified .mp3 recordsdata, we proceed to remodel them into MIDI recordsdata. MIDI recordsdata symbolize musical compositions in a digital format, enabling environment friendly evaluation and era by our fashions. For this conversion, we depend on the sensible and user-friendly performance offered by Spotify’s Basic Pitch.

With the help of Spotify’s Primary Pitch, we add the acquired .mp3 recordsdata, initiating the transformation course of. The device harnesses superior algorithms to decipher the audio content material, extracting essential musical components comparable to notes and buildings to generate corresponding MIDI recordsdata. These MIDI recordsdata function the cornerstone of our music era fashions, empowering us to control and produce recent, revolutionary compositions.

Mannequin Structure

To develop our music era mannequin, we make the most of a specialised structure tailor-made particularly for this objective. The chosen structure contains two LSTM (Lengthy Quick-Time period Reminiscence) layers, every consisting of 256 models. LSTM, a sort of recurrent neural community (RNN), excels in dealing with sequential information, making it a superb selection for producing music with its inherent temporal traits.

The primary LSTM layer processes enter sequences with a set size of 100, as decided by the sequence_length variable. By returning sequences, this layer successfully preserves the temporal relationships current within the musical information. To stop overfitting and enhance the mannequin’s adaptability to new information, a dropout layer with a dropout charge of 0.3 is included.

The second LSTM layer, which doesn’t return sequences, receives the outputs from the earlier layer and additional learns intricate patterns throughout the music. Lastly, a dense layer with a softmax activation operate generates output chances for the following word.

Constructing the Mannequin

Having established our mannequin structure, let’s dive straight into constructing the identical. We are going to break down the code into sections and clarify every half for the reader’s sake.

We begin by importing the mandatory libraries that present helpful functionalities for our mission. Along with the same old libraries required for normal ops, we will likely be utilizing tensorflow for deep studying, and music21 for music manipulation.

import numpy as np
import os
import tensorflow as tf
from tensorflow.keras.fashions import Sequential
from tensorflow.keras.layers import LSTM, Dropout, Dense
from tensorflow.keras.utils import to_categorical
from music21 import converter, instrument, stream, word, chord
from google.colab import recordsdata

Loading and Processing MIDI Information

Subsequent, we outline the listing the place our MIDI recordsdata are situated. The code then goes by way of every file within the listing, extracts the notes and chords, and shops them for additional processing. The ‘converter’ module from the music21 library is used to parse the MIDI recordsdata and retrieve the musical components. As an experiment, we are going to first use only one MIDI file to coach the mannequin after which examine the end result through the use of 5 MIDI recordsdata for coaching.

# Listing containing the MIDI recordsdata
midi_dir = "/content material/Midi Information"

notes = []

# Course of every MIDI file within the listing
for filename in os.listdir(midi_dir):
    if filename.endswith(".midi"):
        file = converter.parse(os.path.be a part of(midi_dir, filename))

        # Discover all of the notes and chords within the MIDI file
        strive:
            # If the MIDI file has instrument components
            s2 = file.components.stream()
            notes_to_parse = s2[0].recurse()
        besides:
            # If the MIDI file solely has notes (
            # no chords or instrument components)
            notes_to_parse = file.flat.notes

        # Extract pitch and length data from notes and chords
        for factor in notes_to_parse:
            if isinstance(factor, word.Word):
                notes.append(str(factor.pitch))
            elif isinstance(factor, chord.Chord):
                notes.append('.'.be a part of(str(n) for n in 
                factor.normalOrder))

# Print the variety of notes and a few instance notes
print("Whole notes:", len(notes))
print("Instance notes:", notes[:10])

Mapping Notes to Integers

To transform the notes into numerical sequences that our mannequin can course of, we create a dictionary that maps every distinctive word or chord to a corresponding integer. This step permits us to symbolize the musical components in a numerical format.

# Create a dictionary to map distinctive notes to integers
unique_notes = sorted(set(notes))
note_to_int = {word: i for i, word in 
enumerate(unique_notes)}

Producing Enter and Output Sequences

With the intention to prepare our mannequin, we have to create enter and output sequences. That is completed by sliding a fixed-length window over the checklist of notes. The enter sequence consists of the previous notes and the output sequence is the following word. These sequences are saved in separate lists.

# Convert the notes to numerical sequences
sequence_length = 100  # Size of every enter sequence
input_sequences = []
output_sequences = []

# Generate enter/output sequences
for i in vary(0, len(notes) - sequence_length, 1):
    # Extract the enter sequence
    input_sequence = notes[i:i + sequence_length]
    input_sequences.append([note_to_int[note] for 
    word in input_sequence])

    # Extract the output sequence
    output_sequence = notes[i + sequence_length]
    output_sequences.append(note_to_int[output_sequence])

Reshaping and Normalizing Enter Sequences

Earlier than feeding the enter sequences to our mannequin, we reshape them to match the anticipated enter form of the LSTM layer. Moreover, we normalize the sequences by dividing them by the whole variety of distinctive notes. This step ensures that the enter values fall inside an acceptable vary for the mannequin to study successfully.

# Reshape and normalize the enter sequences
num_sequences = len(input_sequences)
num_unique_notes = len(unique_notes)

# Reshape the enter sequences
X = np.reshape(input_sequences, (num_sequences, sequence_length, 1))
# Normalize the enter sequences
X = X / float(num_unique_notes)

One-Sizzling Encoding Output Sequences

The output sequences representing the following word to foretell will convert right into a one-hot encoded format. This encoding permits the mannequin to know the likelihood distribution of the following word among the many obtainable notes.

# One-hot encode the output sequences
y = to_categorical(output_sequences)

Defining the RNN Mannequin

We outline our RNN (Recurrent Neural Community) mannequin utilizing the Sequential class from the tensorflow.keras.fashions module. The mannequin consists of two LSTM (Lengthy Quick-Time period Reminiscence) layers, adopted by a dropout layer to stop overfitting. The final layer is a Dense layer with a softmax activation operate to output the chances of every word.

# Outline the RNN mannequin
mannequin = Sequential()
mannequin.add(LSTM(256, input_shape=(X.form[1], X.form[2]), 
return_sequences=True))
mannequin.add(Dropout(0.3))
mannequin.add(LSTM(256))
mannequin.add(Dense(y.form[1], activation='softmax'))

Compiling and Coaching the Mannequin

We compile the mannequin by specifying the loss operate and optimizer. We then proceed to coach the mannequin on the enter sequences (X) and output sequences (y) for a selected variety of epochs and with a given batch dimension.

# Compile the mannequin
mannequin.compile(loss="categorical_crossentropy", optimizer="adam")

# Step 4: Prepare the mannequin
mannequin.match(X, y, batch_size=64, epochs=100)

Music Technology

As soon as we prepare the mannequin, we will generate new music sequences. We outline a operate named generate_music that takes three inputs: the educated mannequin, seed_sequence, and size. It makes use of the mannequin to foretell the following word within the sequence primarily based on the earlier notes and repeats this course of to generate the specified size of music.

To start out, we create a replica of the seed_sequence to stop any modifications to the unique sequence. This seed_sequence serves because the preliminary level for producing the music.

We then enter a loop that runs size occasions. Inside every iteration, carry out the next steps:

1. Convert the generated_sequence right into a numpy array.
2. Reshape the input_sequence by including an additional dimension to match the anticipated enter form of the mannequin.
3. Normalize the input_sequence by dividing it by the whole variety of distinctive notes. This ensures that the values fall inside an acceptable vary for the mannequin to work successfully.

After normalizing the input_sequence, use the mannequin to foretell the chances of the following word. The mannequin.predict methodology takes the input_sequence as enter and returns the anticipated chances.

To pick the following word, the np.random.selection operate is used, which randomly picks an index primarily based on the chances obtained. This randomness introduces variety and unpredictability into the generated music.

The chosen index represents the brand new word, which is appended to the generated_sequence. The generated_sequence is then up to date by eradicating the primary factor to keep up the specified size. As soon as the loop completes, the generated_sequence is returned, representing the newly generated music.

The seed_sequence and the specified generated_length have to be set to generate the music. The seed_sequence ought to be a sound enter sequence that the mannequin has been educated on, and the generated_length determines the variety of notes the generated music ought to comprise.

# Generate new music
def generate_music(mannequin, seed_sequence, size):
    generated_sequence = seed_sequence.copy()

    for _ in vary(size):
        input_sequence = np.array(generated_sequence)
        input_sequence = np.reshape(input_sequence, (1, len(input_sequence), 1))
        input_sequence = input_sequence / float(num_unique_notes)  # Normalize enter sequence

        predictions = mannequin.predict(input_sequence)[0]
        new_note = np.random.selection(vary(len(predictions)), p=predictions)
        generated_sequence.append(new_note)
        generated_sequence = generated_sequence[1:]

    return generated_sequence

# Set the seed sequence and size of the generated music
seed_sequence = input_sequences[0]   # Exchange with your personal seed sequence
generated_length = 100  # Exchange with the specified size of the generated music

generated_music = generate_music(mannequin, seed_sequence, generated_length)
generated_music

# Output of the above code
[1928,
 1916,
 1959,
 1964,
 1948,
 1928,
 1190,
 873,
 1965,
 1946,
 1928,
 1970,
 1947,
 1946,
 1964,
 1948,
 1022,
 1945,
 1916,
 1653,
 873,
 873,
 1960,
 1946,
 1959,
 1942,
 1348,
 1960,
 1961,
 1971,
 1966,
 1927,
 705,
 1054,
 150,
 1935,
 864,
 1932,
 1936,
 1763,
 1978,
 1949,
 1946,
 351,
 1926,
 357,
 363,
 864,
 1965,
 357,
 1928,
 1949,
 351,
 1928,
 1949,
 1662,
 1352,
 1034,
 1021,
 977,
 150,
 325,
 1916,
 1960,
 363,
 943,
 1949,
 553,
 1917,
 1962,
 1917,
 1916,
 1947,
 1021,
 1021,
 1051,
 1648,
 873,
 977,
 1959,
 1927,
 1959,
 1947,
 434,
 1949,
 553,
 360,
 1916,
 1190,
 1022,
 1348,
 1051,
 325,
 1965,
 1051,
 1917,
 1917,
 407,
 1948,
 1051]

Publish-Processing

The generated output, as seen, is a sequence of integers representing the notes or chords in our generated music. With the intention to hearken to the generated output, we should convert this again into music by reversing the mapping we created earlier to get the unique notes/chords. To do that, we are going to firstly create a dictionary referred to as int_to_note, the place the integers are the keys and the corresponding notes are the values.

Subsequent, we create a stream referred to as output_stream to retailer the generated notes and chords. This stream acts as a container to carry the musical components that can represent the generated music.

We then iterate by way of every factor within the generated_music sequence. Every factor is a quantity representing a word or a chord. We use the int_to_note dictionary to transform the quantity again to its authentic word or chord string illustration.

If the sample is a chord, which may be recognized by the presence of a dot or being a digit, we break up the sample string into particular person notes. For every word, we create a word.Word object, assign it a piano instrument, and add it to the notes checklist. Lastly, we create a chord.Chord object from the notes checklist, representing the chord, and append it to the output_stream.

If the sample is a single word, we create a word.Word object for that word, assign it a piano instrument, and add it on to the output_stream.

As soon as all of the patterns within the generated_music sequence have been processed, we write the output_stream to a MIDI file named ‘generated_music.mid’. Lastly, we obtain the generated music file from Colab utilizing the recordsdata.obtain operate.

# Reverse the mapping from notes to integers
int_to_note = {i: word for word, i in note_to_int.objects()}

# Create a stream to carry the generated notes/chords
output_stream = stream.Stream()

# Convert the output from the mannequin into notes/chords
for sample in generated_music:
    # sample is a quantity, so we convert it again to a word/chord string
    sample = int_to_note[pattern]

    # If the sample is a chord
    if ('.' in sample) or sample.isdigit():
        notes_in_chord = sample.break up('.')
        notes = []
        for current_note in notes_in_chord:
            new_note = word.Word(int(current_note))
            new_note.storedInstrument = instrument.Piano()
            notes.append(new_note)
        new_chord = chord.Chord(notes)
        output_stream.append(new_chord)
    # If the sample is a word
    else:
        new_note = word.Word(sample)
        new_note.storedInstrument = instrument.Piano()
        output_stream.append(new_note)

# Write the stream to a MIDI file
output_stream.write('midi', fp='generated_music.mid')

# Obtain the generated music file from Colab
recordsdata.obtain('generated_music.mid')

Last output

Now, it’s time to hearken to the result of our AI-generated music. You’ll find the hyperlink to hearken to the music beneath.

To be trustworthy, the preliminary end result might sound like somebody with restricted expertise enjoying musical devices. That is primarily as a result of we educated our mannequin utilizing solely a single MIDI file. Nevertheless, we will improve the standard of the music by repeating the method and coaching our mannequin on a bigger dataset. On this case, we are going to prepare our mannequin utilizing 5 MIDI recordsdata, all of which will likely be instrumental music of the same type.

The distinction within the high quality of the music generated from the expanded dataset is sort of exceptional. It clearly demonstrates that coaching the mannequin on a extra numerous vary of MIDI recordsdata results in important enhancements within the generated music. This emphasizes the significance of accelerating the scale and number of the coaching dataset to realize higher musical outcomes.

Limitations

Although we managed to generate music utilizing a complicated mannequin, however there are particular limitations to scaling such a system.

1. Restricted Dataset: The standard and variety of the generated music depend upon the variability and dimension of the dataset used for coaching. A restricted dataset can limit the vary of musical concepts and types our mannequin can study from.
2. Creativity Hole: Though AI-generated music can produce spectacular outcomes, it lacks the inherent creativity and emotional depth that human composers convey to their compositions. The music generated by AI might sound robotic or miss the delicate nuances that make music actually charming.
3. Knowledge Dependency: Affect the generated music by the enter MIDI recordsdata used for coaching. If the coaching dataset has biases or particular patterns, the generated music might exhibit comparable biases or patterns, limiting its originality.
4. Computational Necessities: Coaching and producing music utilizing AI fashions may be computationally costly and time-consuming. It requires highly effective {hardware} and environment friendly algorithms to coach advanced fashions and generate music in an affordable timeframe.
5. Subjective Analysis: Assessing the standard and creative worth of AI-generated music may be subjective. Totally different individuals might have completely different opinions on the aesthetics and emotional affect of the music, making it difficult to ascertain common analysis requirements.

Conclusion

On this mission, we launched into the fascinating journey of producing music utilizing AI. Our purpose was to discover the capabilities of AI in music composition and unleash its potential in creating distinctive musical items. By means of the implementation of AI fashions and deep studying strategies, we efficiently generated music that carefully resembled the type of the enter MIDI recordsdata. The mission showcased the flexibility of AI to help and encourage within the artistic technique of music composition.

Key Takeaways

Listed below are a few of the key takeaways from this mission:

1. We discovered that AI can function a invaluable assistant within the artistic course of, providing new views and concepts for musicians and composers.
2. The standard and variety of the coaching dataset significantly affect the output of AI-generated music. Curating a well-rounded and various dataset is essential to reaching extra authentic and numerous compositions.
3. Whereas AI-generated music exhibits promise, it can’t exchange the creative and emotional depth introduced by human composers. The optimum method is to leverage AI as a collaborative device that enhances human creativity.
4. Exploring AI-generated music raises essential moral issues, comparable to copyright and mental property rights. It’s important to respect these rights and foster a wholesome and supportive surroundings for each AI and human artists.
5. This mission bolstered the importance of steady studying within the discipline of AI-generated music. Staying up to date with developments and embracing new strategies allows us to push the boundaries of musical expression and innovation.

Regularly Requested Questions

The media proven on this article shouldn’t be owned by Analytics Vidhya and is used on the Writer’s discretion.

Associated