ARCed Audio Player

Started by ForeverZer0, November 19, 2011, 03:50:58 pm

Previous topic - Next topic

ForeverZer0

Alright, so I have been working on the audio player, using pyglet's media module, and have come to the conclusion that it is insufficient for what we need.


  • First and foremost (I should have looked into this sooner), MIDI is not a supported format, neither through pyglet, nor AVbin.

  • I *think* in order for AVbin to play, you need to first start the pyglet loop with "pyglet.app.run()", which we cannot do obviously, we are using wxPython.

  • It basically requires a hack to have events fire properly. Normally, "dispath_events()", which must be called regularly to have events you have binded actually fire, is called via the pyglet app's loop. I already had to make a wxTimer to handle updating for such a minor thing that is handled better in other audio add-ons.
  • Playback is often times very buggy, especially when files are paused, then restarted.

  • I really wanted to have an audio module that better supported reading raw data a little better that I could convert easily into numpy arrays. The reason for this is I planned on implementing a simple spectrogram to provide a visual representation of the stream. Most other audio modules make this conversion relatively simple to do, even pygame's built in audio module, but pyglet lacks any clean way of doing this.



I have been looking into some other modules, but all seem to have a few pros and cons. Here's a a quick summary of a few I found so far.

PyGame.Mixer module
Has the easiest way of converting data to numpy arrays with its sndarray function. Has seek/volume functions, and supports all the file formats that we plan to utilize, and is pretty easy to use. The only real con is missing support of changing pitch. This can still be done by manipulating the arrays, though something a bit cleaner would be nice. If changing pitch gets dropped for the engine, this won't be a ordeal at all. :P

AudioLab
Probably has the most amount of actual function, most of which we won't be using. Biggest drawback is that it is an addon to a SciPy, which is quite q large package that would also need to be included. The current version is only 0.1, so it is also rather immature, though in practice, it seems to work pretty well.

PyAudiere
A rather immature audio module. It really doesn't offer much of anything over the previous two.

PyAudio
A wrapper for PortAudio, a cross-platform audio library. I haven't looked as deep into this one, so I don't have much to say about it yet.


If you can't tell by my descriptions, I am leaning towards PyGame as the player of choice.


Here is a screenshot of what I had for the audio player.
Spoiler: ShowHide

The spectrograph is copy-paste edit, but it will look very similar with a quick plot using matplotlib

I am done scripting for RMXP. I will likely not offer support for even my own scripts anymore, but feel free to ask on the forum, there are plenty of other talented scripters that can help you.

Blizzard

I think PyGame would be the best choice here.
Check out Daygames and our games:

King of Booze 2      King of Booze: Never Ever
Drinking Game for Android      Never have I ever for Android
Drinking Game for iOS      Never have I ever for iOS


Quote from: winkioI do not speak to bricks, either as individuals or in wall form.

Quote from: Barney StinsonWhen I get sad, I stop being sad and be awesome instead. True story.

Ryex

November 19, 2011, 04:21:39 pm #2 Last Edit: November 19, 2011, 04:24:51 pm by Ryex
Wrong. I've already gone and made a working pyglet version, but it was via the command line just as a test to be sure it worked. I'll have to pull the code together again.

and yes Midi isn't supported I was thinking it would be a rather simple thing to wrap the midi support from the engine with cython to access it.
I no longer keep up with posts in the forum very well. If you have a question or comment, about my work, or in general I welcome PM's. if you make a post in one of my threads and I don't reply with in a day or two feel free to PM me and point it out to me.<br /><br />DropBox, the best free file syncing service there is.<br />

ForeverZer0

Either way, I'm going with PyGame. Pyglet wasn't meant to be used as a fully functional mixer, and the cons are far out-weighing the pros of sticking with it out of principle. Its usage is a bit messy, and it doesn't offer the capabilities the others have by default. Of course we could write are own whole mixing library, and do all kinds of stuff, but I think its a far cleaner and better solution to just use a tried and tested library that offers what we need by default.
I am done scripting for RMXP. I will likely not offer support for even my own scripts anymore, but feel free to ask on the forum, there are plenty of other talented scripters that can help you.

Ryex

meh, fine. I really didn't want to include pygame but if we have too. also, I take it you at least have an idea on how to to the pitch change with the numpy arrays?
I no longer keep up with posts in the forum very well. If you have a question or comment, about my work, or in general I welcome PM's. if you make a post in one of my threads and I don't reply with in a day or two feel free to PM me and point it out to me.<br /><br />DropBox, the best free file syncing service there is.<br />

ForeverZer0

Yeah, I stumbled across an example a week or so ago on how to do it. I'll have to find it again, though.

I just noticed that PyGame allows for you to set the frequency (default is 22050) .
Aren't pitch and frequency the same thing? Its seems to have the same effect by as changing the pitch when I raise and lower the values. If so, then this will be very simple. I need only make a simple algorithm to convert a 50-150 range into a range of equivalent frequencies. Of course we don't need to stick with the 50-150 range, and can do whatever, even just use the actual frequency values, which may be easier to understand for those who are music oriented.
I am done scripting for RMXP. I will likely not offer support for even my own scripts anymore, but feel free to ask on the forum, there are plenty of other talented scripters that can help you.

Blizzard

November 19, 2011, 05:18:48 pm #6 Last Edit: November 19, 2011, 05:20:00 pm by Blizzard
Sampling rate and frequency are not the exact same thing, but they are related. If you take something that has been sampled with 22050 Hz, double the speed so you get 44100 Hz (for whatever reason) and then play it like that, the pitch will go up by one octave, because the samples are played twice as fast. But if you resample the 22050 Hz into 44100 Hz and then play it, it will sound the same, though it will take up twice the memory (since there is twice as much data on describing the waveform, even if every other sample is a double).
Check out Daygames and our games:

King of Booze 2      King of Booze: Never Ever
Drinking Game for Android      Never have I ever for Android
Drinking Game for iOS      Never have I ever for iOS


Quote from: winkioI do not speak to bricks, either as individuals or in wall form.

Quote from: Barney StinsonWhen I get sad, I stop being sad and be awesome instead. True story.

ForeverZer0

Ah, I see. I may be a lot of things, but musically inclined is not one of them if you haven't noticed, lol.

I'm pretty sure that this is actually changing the pitch. I am using '11-Heal07' as a test SE, and when I set the gauge low (at like 1024 Hz) , it emits as a low rumble, and when high (44100 Hz), it is a fast higher-pitched sound.

Does this mean that it is doing what we want? :hm:
I am done scripting for RMXP. I will likely not offer support for even my own scripts anymore, but feel free to ask on the forum, there are plenty of other talented scripters that can help you.

Ryex

yes and no, you have to set the frequency relative to what the sample is sampled at. if for example the sound was sampled at 44100 Hz playing at 44100 Hz would make it sound normal. and playing at 22050 Hz should be the same as a 50% drop in pitch
I no longer keep up with posts in the forum very well. If you have a question or comment, about my work, or in general I welcome PM's. if you make a post in one of my threads and I don't reply with in a day or two feel free to PM me and point it out to me.<br /><br />DropBox, the best free file syncing service there is.<br />

Blizzard

November 19, 2011, 05:50:04 pm #9 Last Edit: November 19, 2011, 05:51:19 pm by Blizzard
What Ryex said. But there is one thing that you have to consider. Some audio systems require you to specify the output frequency so you have to do the resampling yourself. The output would be 44.1 kHz and you would have to resample audio first to that frequency including an eventual pitch shift. e.g. if you have a sound of 32 kHz and want to play it with 150% pitch, you would have to resample it by 44.1 / (32 * 1.5) = 0.91875 which would then be the final data that you just fill the buffer with. So make sure you know how it works. If you have to do the resampling and mixing yourself (like in SDL), this is how it's done. If the system does it by itself (like in OpenAL), you just specify the output frequency, the secondary buffer frequency and the system will take care of resampling and mixing.
Check out Daygames and our games:

King of Booze 2      King of Booze: Never Ever
Drinking Game for Android      Never have I ever for Android
Drinking Game for iOS      Never have I ever for iOS


Quote from: winkioI do not speak to bricks, either as individuals or in wall form.

Quote from: Barney StinsonWhen I get sad, I stop being sad and be awesome instead. True story.

ForeverZer0

November 19, 2011, 06:03:29 pm #10 Last Edit: November 19, 2011, 06:14:10 pm by ForeverZer0
I see what you are saying. I played around and compared changing the sample speed and the pitch in Audacity, and I see difference each makes. Unfortunately, I think my current method is simply changing the speed.

Damn. I thought I was gonna get off easy... :P
I am done scripting for RMXP. I will likely not offer support for even my own scripts anymore, but feel free to ask on the forum, there are plenty of other talented scripters that can help you.

Ryex

you are useing

import os, sys
# set SDL to use the dummy NULL video driver,
#   so it doesn't need a windowing system.
os.environ["SDL_VIDEODRIVER"] = "dummy"

before calling pygame.init() and then the mixer's init right?
some platform require the full pygame to be inited before the mixer can be used and you have to use that bit of code to force the windowless system.
I no longer keep up with posts in the forum very well. If you have a question or comment, about my work, or in general I welcome PM's. if you make a post in one of my threads and I don't reply with in a day or two feel free to PM me and point it out to me.<br /><br />DropBox, the best free file syncing service there is.<br />

ForeverZer0

I haven't added that yet, but I did read about it. Will add before its all done.
I am done scripting for RMXP. I will likely not offer support for even my own scripts anymore, but feel free to ask on the forum, there are plenty of other talented scripters that can help you.

Blizzard

November 20, 2011, 03:55:16 am #13 Last Edit: November 20, 2011, 03:57:03 am by Blizzard
Quote from: ForeverZer0 on November 19, 2011, 06:03:29 pm
I see what you are saying. I played around and compared changing the sample speed and the pitch in Audacity, and I see difference each makes. Unfortunately, I think my current method is simply changing the speed.

Damn. I thought I was gonna get off easy... :P



You still might get off easy if the system mixes it automatically. I don't think a system like PyGame doesn't have an automatic mixer. Try playing one sound sped up and another one with normal speed. If they mess with each other, you have a problem. If not, problme solved. xD With playing sounds I obviously mean playing the simultaneously.
Check out Daygames and our games:

King of Booze 2      King of Booze: Never Ever
Drinking Game for Android      Never have I ever for Android
Drinking Game for iOS      Never have I ever for iOS


Quote from: winkioI do not speak to bricks, either as individuals or in wall form.

Quote from: Barney StinsonWhen I get sad, I stop being sad and be awesome instead. True story.

Ryex

I jut want to say I got a working pygame thing that can re sample the file and play it. one problem. oversampling takes a few seconds when the new rate is multiplied by 0.5 or 2.0  (50% or 200%) but takes a good five minutes (possible more I didn't really time it) at anything else (I tried 1.5 or 150% and spent a long time waiting with one core of my cpu at 100% waiting for the resampling call to finish)

but the result was perfect. I used scipy.signal.resample on the sndarray made from the sound.

Spoiler: ShowHide

import os, sys, time
import numpy as np
import scipy.signal as signal
os.environ["SDL_VIDEODRIVER"] = "dummy"

SAMPLE_RATE = 44100

import pygame.mixer as mixer
import pygame.sndarray as sndarray

mixer.init(frequency=SAMPLE_RATE, size=-16, channels=2, buffer=4096)

file = r"C:\Program Files\Common Files\Enterbrain\RGSS\Standard\Audio\BGS\001-Wind01.ogg"

sound = mixer.Sound(file)







def resample(sound, rate):
    array = sndarray.array(sound)
    n_samples = len(array)
    length = sound.get_length()
    new_n_samples = int(n_samples / rate)
    new_length = length / rate
    new_array = signal.resample(array, new_n_samples)
    new_sound = sndarray.make_sound(np.ascontiguousarray(new_array, dtype=array.dtype))
    return new_sound

shifted = resample(sound, 0.5)     
length = int(sound.get_length())
print "ready"
while True:
    input = raw_input()
    if "c|".lower() in input.lower():
        sound.stop()
        shifted.stop()
        rate = float(input.lower()[2:])
        print "resampeling to: %s"  % rate
        shifted = resample(sound, rate)
        print "done"
    elif "q".lower() in input.lower():
        print "quiting"
        break
    elif "n".lower() in input.lower():
        sound.stop()
        shifted.stop()
        print "playing normal sound"
        sound.play()
    elif "s".lower() in input.lower():
        sound.stop()
        shifted.stop()
        print "playing shifted sound"
        shifted.play()
    else:
        sound.stop()
        shifted.stop()
        print "stoping playback: q for quit, n - for normal sound, or s for shifted sound, or c|float to change the shift"


I'm going to try and see if I can use a less expensive form of resampeling to get a effect that is at least close
I no longer keep up with posts in the forum very well. If you have a question or comment, about my work, or in general I welcome PM's. if you make a post in one of my threads and I don't reply with in a day or two feel free to PM me and point it out to me.<br /><br />DropBox, the best free file syncing service there is.<br />

Blizzard

If it took that long, it might be that the sound isn't being streamed. That's why it would take so long to resample. Resampling on the fly should be quick as all it does is just taking a few samples and calculating the average. :facepalm:
Check out Daygames and our games:

King of Booze 2      King of Booze: Never Ever
Drinking Game for Android      Never have I ever for Android
Drinking Game for iOS      Never have I ever for iOS


Quote from: winkioI do not speak to bricks, either as individuals or in wall form.

Quote from: Barney StinsonWhen I get sad, I stop being sad and be awesome instead. True story.

Ryex

well in that case I would need to figure out how to not re sample an entire sound at once and do it in a buffer manner.
I no longer keep up with posts in the forum very well. If you have a question or comment, about my work, or in general I welcome PM's. if you make a post in one of my threads and I don't reply with in a day or two feel free to PM me and point it out to me.<br /><br />DropBox, the best free file syncing service there is.<br />

ForeverZer0

I didn't know we were using SciPy. I have been struggling trying to figure out how to do this without using it. Almost every example out there that shows how to do this uses it.
I am done scripting for RMXP. I will likely not offer support for even my own scripts anymore, but feel free to ask on the forum, there are plenty of other talented scripters that can help you.

Ryex

it's basically the only option. also OMG BLIZZ YOUR RIGHT! the lag was cause by the fact that it was interpolating using every point in the array instead of just a small sample set having it resample small chunks at a time chops it down to less than a second for every % I'm use 200 samples per part right now and it's generating a bit of noise when probing the pitch but I bet if I increase the chunk size a bit it will be better.
I no longer keep up with posts in the forum very well. If you have a question or comment, about my work, or in general I welcome PM's. if you make a post in one of my threads and I don't reply with in a day or two feel free to PM me and point it out to me.<br /><br />DropBox, the best free file syncing service there is.<br />

Ryex

Ok, I have a working resample method that work in less than a second. dropping the pitch takes more time than raising it.

Spoiler: ShowHide

print "++++++++Doing setup, please wait until ready+++++++++"
import os, sys, time, math
import numpy as np
import scipy.signal as signal
os.environ["SDL_VIDEODRIVER"] = "dummy"

SAMPLE_RATE = 44100

import pygame.mixer as mixer
import pygame.sndarray as sndarray

mixer.init(frequency=SAMPLE_RATE, size=-16, channels=2, buffer=4096)

file = r"C:\Program Files\Common Files\Enterbrain\RGSS\Standard\Audio\BGS\001-Wind01.ogg"

sound = mixer.Sound(file)

CHUNKSIZE = 4096

def resample(sound, rate):
    time_now = time.time()
    array = sndarray.array(sound)
    n_samples = len(array)
    new_n_samples = int(n_samples / rate)
    rounds = n_samples / CHUNKSIZE
    samples_per = new_n_samples / rounds
    new_array = np.zeros((new_n_samples, 2))
    for i in xrange(rounds):
        part = signal.resample(array[i*CHUNKSIZE:(i+1)*CHUNKSIZE], samples_per)
        new_array[i*samples_per:(i+1)*samples_per] = part
    #new_array = signal.resample(array, new_n_samples)
    new_sound = sndarray.make_sound(np.ascontiguousarray(new_array, dtype=array.dtype))
    print "resampeling took: %s seconds" % (time.time() - time_now)
    return new_sound

shifted = resample(sound, 0.5)     
length = int(sound.get_length())
print """======Ready========
Commands: q for quit
n - to play normal sound
s - to play shifted sound
c|float - where float is 0.5 or 1.5 ect.to change the shift
Leave blank to stop playback"""
while True:
    input = raw_input("Command: ")
    if "c|".lower() in input.lower():
        sound.stop()
        shifted.stop()
        rate = float(input.lower()[2:])
        print "resampeling to: %s"  % rate
        shifted = resample(sound, rate)
    elif "q".lower() in input.lower():
        print "quiting"
        break
    elif "n".lower() in input.lower():
        sound.stop()
        shifted.stop()
        print "playing normal sound"
        sound.play()
    elif "s".lower() in input.lower():
        sound.stop()
        shifted.stop()
        print "playing shifted sound"
        shifted.play()
    else:
        sound.stop()
        shifted.stop()
        print "stoping playback"


I'm going to test it with a midi file real quick and report back
I no longer keep up with posts in the forum very well. If you have a question or comment, about my work, or in general I welcome PM's. if you make a post in one of my threads and I don't reply with in a day or two feel free to PM me and point it out to me.<br /><br />DropBox, the best free file syncing service there is.<br />