ARCed Audio Player

[ForeverZer0]

Alright, so I have been working on the audio player, using pyglet's media module, and have come to the conclusion that it is insufficient for what we need.

• First and foremost (I should have looked into this sooner), MIDI is not a supported format, neither through pyglet, nor AVbin.


• I *think* in order for AVbin to play, you need to first start the pyglet loop with "pyglet.app.run()", which we cannot do obviously, we are using wxPython.


• It basically requires a hack to have events fire properly. Normally, "dispath_events()", which must be called regularly to have events you have binded actually fire, is called via the pyglet app's loop. I already had to make a wxTimer to handle updating for such a minor thing that is handled better in other audio add-ons.
  
• Playback is often times very buggy, especially when files are paused, then restarted.


• I really wanted to have an audio module that better supported reading raw data a little better that I could convert easily into numpy arrays. The reason for this is I planned on implementing a simple spectrogram to provide a visual representation of the stream. Most other audio modules make this conversion relatively simple to do, even pygame's built in audio module, but pyglet lacks any clean way of doing this.

I have been looking into some other modules, but all seem to have a few pros and cons. Here's a a quick summary of a few I found so far.

PyGame.Mixer module
Has the easiest way of converting data to numpy arrays with its sndarray function. Has seek/volume functions, and supports all the file formats that we plan to utilize, and is pretty easy to use. The only real con is missing support of changing pitch. This can still be done by manipulating the arrays, though something a bit cleaner would be nice. If changing pitch gets dropped for the engine, this won't be a ordeal at all.

AudioLab
Probably has the most amount of actual function, most of which we won't be using. Biggest drawback is that it is an addon to a SciPy, which is quite q large package that would also need to be included. The current version is only 0.1, so it is also rather immature, though in practice, it seems to work pretty well.

PyAudiere
A rather immature audio module. It really doesn't offer much of anything over the previous two.

PyAudio
A wrapper for PortAudio, a cross-platform audio library. I haven't looked as deep into this one, so I don't have much to say about it yet.


If you can't tell by my descriptions, I am leaning towards PyGame as the player of choice.


Here is a screenshot of what I had for the audio player.

Spoiler: ShowHide

The spectrograph is copy-paste edit, but it will look very similar with a quick plot using matplotlib

[Blizzard]

I think PyGame would be the best choice here.

[Ryex]

Wrong. I've already gone and made a working pyglet version, but it was via the command line just as a test to be sure it worked. I'll have to pull the code together again.

and yes Midi isn't supported I was thinking it would be a rather simple thing to wrap the midi support from the engine with cython to access it.

[ForeverZer0]

Either way, I'm going with PyGame. Pyglet wasn't meant to be used as a fully functional mixer, and the cons are far out-weighing the pros of sticking with it out of principle. Its usage is a bit messy, and it doesn't offer the capabilities the others have by default. Of course we could write are own whole mixing library, and do all kinds of stuff, but I think its a far cleaner and better solution to just use a tried and tested library that offers what we need by default.

[Ryex]

meh, fine. I really didn't want to include pygame but if we have too. also, I take it you at least have an idea on how to to the pitch change with the numpy arrays?

[ForeverZer0]

Yeah, I stumbled across an example a week or so ago on how to do it. I'll have to find it again, though.

I just noticed that PyGame allows for you to set the frequency (default is 22050) .
Aren't pitch and frequency the same thing? Its seems to have the same effect by as changing the pitch when I raise and lower the values. If so, then this will be very simple. I need only make a simple algorithm to convert a 50-150 range into a range of equivalent frequencies. Of course we don't need to stick with the 50-150 range, and can do whatever, even just use the actual frequency values, which may be easier to understand for those who are music oriented.

[Blizzard]

Sampling rate and frequency are not the exact same thing, but they are related. If you take something that has been sampled with 22050 Hz, double the speed so you get 44100 Hz (for whatever reason) and then play it like that, the pitch will go up by one octave, because the samples are played twice as fast. But if you resample the 22050 Hz into 44100 Hz and then play it, it will sound the same, though it will take up twice the memory (since there is twice as much data on describing the waveform, even if every other sample is a double).

[ForeverZer0]

Ah, I see. I may be a lot of things, but musically inclined is not one of them if you haven't noticed, lol.

I'm pretty sure that this is actually changing the pitch. I am using '11-Heal07' as a test SE, and when I set the gauge low (at like 1024 Hz) , it emits as a low rumble, and when high (44100 Hz), it is a fast higher-pitched sound.

Does this mean that it is doing what we want?

[Ryex]

yes and no, you have to set the frequency relative to what the sample is sampled at. if for example the sound was sampled at 44100 Hz playing at 44100 Hz would make it sound normal. and playing at 22050 Hz should be the same as a 50% drop in pitch

[Blizzard]

What Ryex said. But there is one thing that you have to consider. Some audio systems require you to specify the output frequency so you have to do the resampling yourself. The output would be 44.1 kHz and you would have to resample audio first to that frequency including an eventual pitch shift. e.g. if you have a sound of 32 kHz and want to play it with 150% pitch, you would have to resample it by 44.1 / (32 * 1.5) = 0.91875 which would then be the final data that you just fill the buffer with. So make sure you know how it works. If you have to do the resampling and mixing yourself (like in SDL), this is how it's done. If the system does it by itself (like in OpenAL), you just specify the output frequency, the secondary buffer frequency and the system will take care of resampling and mixing.

[ForeverZer0]

I see what you are saying. I played around and compared changing the sample speed and the pitch in Audacity, and I see difference each makes. Unfortunately, I think my current method is simply changing the speed.

Damn. I thought I was gonna get off easy...

[Ryex]

you are useing

import os, sys
# set SDL to use the dummy NULL video driver, 
#   so it doesn't need a windowing system.
os.environ["SDL_VIDEODRIVER"] = "dummy"

before calling pygame.init() and then the mixer's init right?
some platform require the full pygame to be inited before the mixer can be used and you have to use that bit of code to force the windowless system.

[ForeverZer0]

I haven't added that yet, but I did read about it. Will add before its all done.

[Blizzard]

I see what you are saying. I played around and compared changing the sample speed and the pitch in Audacity, and I see difference each makes. Unfortunately, I think my current method is simply changing the speed.

Damn. I thought I was gonna get off easy...

You still might get off easy if the system mixes it automatically. I don't think a system like PyGame doesn't have an automatic mixer. Try playing one sound sped up and another one with normal speed. If they mess with each other, you have a problem. If not, problme solved. xD With playing sounds I obviously mean playing the simultaneously.

[Ryex]

I jut want to say I got a working pygame thing that can re sample the file and play it. one problem. oversampling takes a few seconds when the new rate is multiplied by 0.5 or 2.0  (50% or 200%) but takes a good five minutes (possible more I didn't really time it) at anything else (I tried 1.5 or 150% and spent a long time waiting with one core of my cpu at 100% waiting for the resampling call to finish)

but the result was perfect. I used scipy.signal.resample on the sndarray made from the sound.

Spoiler: ShowHide

import os, sys, time
import numpy as np
import scipy.signal as signal
os.environ["SDL_VIDEODRIVER"] = "dummy"

SAMPLE_RATE = 44100

import pygame.mixer as mixer
import pygame.sndarray as sndarray

mixer.init(frequency=SAMPLE_RATE, size=-16, channels=2, buffer=4096)

file = r"C:\Program Files\Common Files\Enterbrain\RGSS\Standard\Audio\BGS\001-Wind01.ogg"

sound = mixer.Sound(file)







def resample(sound, rate):
    array = sndarray.array(sound)
    n_samples = len(array)
    length = sound.get_length()
    new_n_samples = int(n_samples / rate)
    new_length = length / rate
    new_array = signal.resample(array, new_n_samples)
    new_sound = sndarray.make_sound(np.ascontiguousarray(new_array, dtype=array.dtype))
    return new_sound

shifted = resample(sound, 0.5)      
length = int(sound.get_length())
print "ready"
while True:
    input = raw_input()
    if "c|".lower() in input.lower():
        sound.stop()
        shifted.stop()
        rate = float(input.lower()[2:])
        print "resampeling to: %s"  % rate
        shifted = resample(sound, rate)
        print "done"
    elif "q".lower() in input.lower():
        print "quiting"
        break
    elif "n".lower() in input.lower():
        sound.stop()
        shifted.stop()
        print "playing normal sound"
        sound.play()
    elif "s".lower() in input.lower():
        sound.stop()
        shifted.stop()
        print "playing shifted sound"
        shifted.play()
    else:
        sound.stop()
        shifted.stop()
        print "stoping playback: q for quit, n - for normal sound, or s for shifted sound, or c|float to change the shift"

I'm going to try and see if I can use a less expensive form of resampeling to get a effect that is at least close

[Blizzard]

If it took that long, it might be that the sound isn't being streamed. That's why it would take so long to resample. Resampling on the fly should be quick as all it does is just taking a few samples and calculating the average.

[Ryex]

well in that case I would need to figure out how to not re sample an entire sound at once and do it in a buffer manner.

[ForeverZer0]

I didn't know we were using SciPy. I have been struggling trying to figure out how to do this without using it. Almost every example out there that shows how to do this uses it.

[Ryex]

it's basically the only option. also OMG BLIZZ YOUR RIGHT! the lag was cause by the fact that it was interpolating using every point in the array instead of just a small sample set having it resample small chunks at a time chops it down to less than a second for every % I'm use 200 samples per part right now and it's generating a bit of noise when probing the pitch but I bet if I increase the chunk size a bit it will be better.

[Ryex]

Ok, I have a working resample method that work in less than a second. dropping the pitch takes more time than raising it.

Spoiler: ShowHide

print "++++++++Doing setup, please wait until ready+++++++++"
import os, sys, time, math
import numpy as np
import scipy.signal as signal
os.environ["SDL_VIDEODRIVER"] = "dummy"

SAMPLE_RATE = 44100

import pygame.mixer as mixer
import pygame.sndarray as sndarray

mixer.init(frequency=SAMPLE_RATE, size=-16, channels=2, buffer=4096)

file = r"C:\Program Files\Common Files\Enterbrain\RGSS\Standard\Audio\BGS\001-Wind01.ogg"

sound = mixer.Sound(file)

CHUNKSIZE = 4096

def resample(sound, rate):
    time_now = time.time()
    array = sndarray.array(sound)
    n_samples = len(array)
    new_n_samples = int(n_samples / rate)
    rounds = n_samples / CHUNKSIZE
    samples_per = new_n_samples / rounds
    new_array = np.zeros((new_n_samples, 2))
    for i in xrange(rounds):
        part = signal.resample(array[i*CHUNKSIZE:(i+1)*CHUNKSIZE], samples_per)
        new_array[i*samples_per:(i+1)*samples_per] = part
    #new_array = signal.resample(array, new_n_samples)
    new_sound = sndarray.make_sound(np.ascontiguousarray(new_array, dtype=array.dtype))
    print "resampeling took: %s seconds" % (time.time() - time_now)
    return new_sound

shifted = resample(sound, 0.5)      
length = int(sound.get_length())
print """======Ready======== 
Commands: q for quit 
n - to play normal sound 
s - to play shifted sound 
c|float - where float is 0.5 or 1.5 ect.to change the shift 
Leave blank to stop playback"""
while True:
    input = raw_input("Command: ")
    if "c|".lower() in input.lower():
        sound.stop()
        shifted.stop()
        rate = float(input.lower()[2:])
        print "resampeling to: %s"  % rate
        shifted = resample(sound, rate)
    elif "q".lower() in input.lower():
        print "quiting"
        break
    elif "n".lower() in input.lower():
        sound.stop()
        shifted.stop()
        print "playing normal sound"
        sound.play()
    elif "s".lower() in input.lower():
        sound.stop()
        shifted.stop()
        print "playing shifted sound"
        shifted.play()
    else:
        sound.stop()
        shifted.stop()
        print "stoping playback"

I'm going to test it with a midi file real quick and report back

[ForeverZer0]

Have you looked into using numpy's FFT functions?

[Ryex]

No I haven't, but form the looks of things it might work as we want. why don't you give it a try?

also, this won't work with midi, in fact I'm not sure how pygame will work with midi period. attempting to play midi files with the sound class fails utterly.

[ForeverZer0]

New screenshot. The waveform is now working correctly, and is not a copy-paste edit anymore. The button bitmaps are subject to change, but they work for now.

Spoiler: ShowHide

EDIT:
Don't mind the "Pitch (Hz)" for the one control. The Hz was accidentally left from when it was a frequency control....

[Reives]

Looking fantastic. o: I've always wished that RMXP had waveform display and a scrollable progress bar, and then BAM, here they are.

[Ryex]

I think I've figure out a way to load media via pyglet's AVbin into pygame's sound class. It hackishly taps into pyglets internal 'private' methods (but everything is public in python) to read data. and will use a trick to create an empty pygame sound so I can obtain a buffer and write the raw bytes to it.

in principle it should work but I'll have to test it to be sure.

the reason I'm looking into it in the first place is because I'm fairly sure pygame can only read ogg, and wav. not flac or any other format we may decide to support.

[Blizzard]

BTW, I forgot what we decided on FLAC suppport. o.o; I think we left it for the actual engine later.

[Ryex]

ok I succeeded in loading an a file from pyglet into pygame.

there is a weird behavior where pyglet.media must be imported before SciPy or it woun't be able to load avbin properly

print "++++++++Doing setup, please wait until ready+++++++++"
import os, sys, time, math
import numpy as np
import pyglet
import pyglet.media as media
import scipy.signal as signal

#os.environ["SDL_VIDEODRIVER"] = "dummy"

SAMPLE_RATE = 44100

import pygame.mixer as mixer
import pygame.sndarray as sndarray

mixer.init(frequency=SAMPLE_RATE, size=-16, channels=2, buffer=4096)

file = r"C:\Program Files\Common Files\Enterbrain\RGSS\Standard\Audio\BGS\001-Wind01.ogg"

CHUNKSIZE = 4096

def resample(sound, rate):
    time_now = time.time()
    array = sndarray.array(sound)
    n_samples = len(array)
    new_n_samples = int(n_samples / rate)
    rounds = n_samples / CHUNKSIZE
    samples_per = new_n_samples / rounds
    new_array = np.zeros((new_n_samples, 2))
    for i in xrange(rounds):
        part = signal.resample(array[i*CHUNKSIZE:(i+1)*CHUNKSIZE], samples_per)
        new_array[i*samples_per:(i+1)*samples_per] = part
    #new_array = signal.resample(array, new_n_samples)
    new_sound = sndarray.make_sound(np.ascontiguousarray(new_array, dtype=array.dtype))
    print "resampeling took: %s seconds" % (time.time() - time_now)
    return new_sound
    
def load_pygame_sound_with_pyglet(filename):
    source = media.load(filename)
    static = media.StaticSource(source)
    data = static._data
    info = mixer.get_init()
    typecodes = { 8 : np.uint8,   # AUDIO_U8
                 16 : np.uint16, # AUDIO_U16 / AUDIO_U16SYS
                 -8 : np.int8,   # AUDIO_S8
                 -16 : np.int16  # AUDUI_S16 / AUDIO_S16SYS
                 }
    channels = info[2]
    pyglettypecode = typecodes[source.audio_format.sample_size]
    pygametypecode = typecodes[info[1]]
    array = np.fromstring(data, pyglettypecode)
    if channels == 2:
        array = np.column_stack((array, array))
    sound = sndarray.make_sound(np.ascontiguousarray(array, dtype=typecode))
    rate = float(source.audio_format.sample_rate) / float(SAMPLE_RATE)
    shifted_sound = resample(sound, rate)
    return shifted_sound

pyglet_sound = load_pygame_sound_with_pyglet(file)
sound = mixer.Sound(file)
shifted = resample(sound, 1.5)     

print """======Ready======== 
Commands: q for quit 
n - to play normal sound 
s - to play shifted sound 
c|float - where float is 0.5 or 1.5 ect. to change the shift 
Leave blank to stop playback"""
while True:
    input = raw_input("Command: ")
    if "c|".lower() in input.lower():
        sound.stop()
        shifted.stop()
        rate = float(input.lower()[2:])
        print "resampeling to: %s"  % rate
        shifted = resample(sound, rate)
    elif "q".lower() in input.lower():
        print "quiting"
        break
    elif "n".lower() in input.lower():
        sound.stop()
        shifted.stop()
        pyglet_sound.stop()
        print "playing normal sound"
        sound.play()
    elif "s".lower() in input.lower():
        sound.stop()
        shifted.stop()
        pyglet_sound.stop()
        print "playing shifted sound"
        shifted.play()
    elif "p" in input:
        sound.stop()
        shifted.stop()
        pyglet_sound.stop()
        print "playing sound loaded with pyglet and resampled"
        pyglet_sound.play()
    else:
        sound.stop()
        shifted.stop()
        pyglet_sound.stop()
        print "stoping playback"
        

[Blizzard]

Feel free to use a bigger bufffer and chunk size. 16384 is just fine. Just be careful as shorter sounds may have problems in being played with a bigger buffer. In that case, you can revert it back to 4096.

BTW, do you know why the often used default is 4096?

[Ryex]

Ok Zero here is how we are are going to do this. I'm going to use cython to wrap XAL (our audio library). Blizz is going to add seeking and a way to get raw audio data and I'll wrap thoes too. after that the editor will be able to play anything the engine can play. no more worries on our side.

[Blizzard]

Oh, BTW, Ryex. You will also have to compile hltypes for XAL to work. This should work without problems, though. I mean, I can compile you the DLLs and LIBs.

[ForeverZer0]

That sounds good. My implementation of a seek function and keeping the slider updated with timers was a bit sketchy.

[Ryex]

ok, I've successfully wrapped XAL and it can load and play ogg files. I had to slightly change the interface to get it to wrap properly.

you create a PyXAL.XALManger object to init XAL
before you destroy the XALManager object you must call it's DestroyXAL method as it doesn't seem to happen automatically
to load a sound call 'XALManager.createSound(filename)' this will return a PySound object which is an interface for the xal::sound object
then create a player for that sound with 'XALManager.createPlayer(PySound)' passing the sound object returned from createSound. this will return a PyPlayer object whihc is an interface for a xal::player object
then just call 'player.play()' ext as normal

not that it currently crashes python if you try to load a wav file. this needs to be fixed. loading a file that xal can't decode needs to throw a exception so that python can recover.

[Blizzard]

Hm... XAL only logs the error if it can't load a file. This is so loading is sped up (because exceptions are slow) and unsupported files can be simply just ignored. I can add an additional parameter during initialization called "rigorousHandling" which will be false by default and that will only log when false and throw exceptions when true. Does that work fine with you? Alternatively createSound will actually return NULL in C++ (so it should be "None" in Python) if a sound could not be loaded so you can actually detect when a file was trying to be loaded that isn't supported. I suggest that you use the return value of createSound to detect that.

[Ryex]

ah, if it returns NULL then there is another option. I just have to make sure i'm checking for that return value and act on it. I thought for a moment that it was just failing internally and there was nothing I could do.

also, why is WAV support currently not included?

[Blizzard]

Actually it should be working. O_o

Try following: Compile the XAL demo_simple and try the wav there. If it doesn't work there, it's not a normal PCM wav file.

Alternatively try to check the log that XAL prints out. Make sure you are accessing the right file. XAL tries to load files relative from it's cwd. The loading might have failed, because the path is wrong and the file actually doesn't exist there. Try loading an absolute path just to see if the problem is with the path spec or with the wav loading.

[Ryex]

OK!

I've wrapped all XAL functionality to date, or all useful functionality at least.
a few notes:
there is a test file PyXAL_Test.py in editor/ARCed/src
pitch shifting is still not there
WAV files do not set their duration when they are loaded. I implemented a calculation in the test file so we could do it our selves if we have too but that should really be fixed.
seek functionality is not there Blizz said that XAL is not in a position to do it properly for wav files so he isn't doing it yet

sound.readRawData return a 2 tuple of (size_in_bits_not_bytes, data_string)
I implemented a function to properly create a numpy array from the data in the test file, the 24 bit integers that were in the wav files I tested were a bitch to convert...
the function returns a 2d array of the left and right channel data I barrows FZero's waveform panel and implemented it in the test to demonstrate how to draw the left and right audio channels

an Image for you:

Spoiler: ShowHide

@FZer0 you should be able to implement most of the audio panel now just disable the position scroll bar so the user can't interact with it. you can still update it's position to match the offset of the music but disable user input.
as for setting pitch just comment out the liens where the pitch of the playing sound would be set
the play method accepts two arguments,
sound.play(float fade_time, bool looping)
so implanting looping is easy
and as I said I implemented a function to create an array form the raw data and demonstrated drawing the left and right audio channel in a graph.

[Blizzard]

WAV files do not set their duration when they are loaded. I implemented a calculation in the test file so we could do it our selves if we have too but that should really be fixed.

OMG, I'm so sorry. ._.;;; I never implemented the calculation for it after loading the WAV data. I implemented it now and commited the new binaries and libs. Basically duration = byteSize / (samplingRate * channels * bytesPerSample), but you probably know that already.

EDIT:

pitch shifting is still not there

I just implemented xal::Player::setPitch and it works. The value is a float where 1.0 means 100%, 0.5 means 50% and 2.0 means 200%. Implement everything accordingly in the editor. I updated the binaries and libs again, but I didn't interface them to Python.

This feature is currently only supported by the DirectSound audio system.

[ForeverZer0]

There is a memory leak in the current implementation. I cannot for the life of me find it, everything appears to be being disposed properly, but nonetheless. it is there.  I tracked it down to the "on_change_file" method of the PyXAL_Test file. Simply hardcode a path to the "file" attribute, and have the play button call this event to see for yourself. The more you press it, the more memory usage just keeps going up. The player and the sound are both being destroyed, which I confirmed with print statements. I'm at a bit of a loss at the moment...

[Ryex]

if there is a memory leak it is in XAL because I'm calling the destroy methods... wait no... oh crap! one sec I think I forgot to write the __dealloc__ methods for the sound and player!


EDIT:

actually no there was a reason I didn't write the dealloc methods for the sound and player even if the interface is destroyed XAL still keeps the objects internally and has the ability to retrieve them at which point they are wrapped with another python interface. the python interface is being destroyed properly so the memory leak is indeed in XAL.

I even tried seeing if the memory dropped after xal::destroy was called. It doesn't. the memory goes down slightly if you destroy the sound with out playing it but if you play it first the memory usage goes up and then stays up.

in any case I wrapped the set and get pitch methods. I also put doc string on everything and called the help() function to generate a help file for you.
editor/ARCed/src/Core/PyXAL/PyXAL_Help.txt

[ForeverZer0]

I have just been using the XAL C++ source as a reference. Its not really difficult.

EDIT:

The audio player is almost done. I already implemented pitch for when it was done, so now I can actually test it. For now I just have a flag set to determine if pitch changing is permitted or not:

PITCH_ENABLED = sys.platform == 'win32'

EDIT2:

Setting the pitch officially does nothing. I can create an instance of a player, then:

self.player.setPitch(2.0)
self.player.getPitch()
--> 0.0

Also, it would be slightly easier to use if the values could be passed with the initializer as they are in the C++ version. That's not a big deal at all, though.

EDIT3:

As it turns out, it works well when there is a file playing, but not if it is set beforehand.

[Ryex]

well I was a bit stupid when wrapping the getPitch method and returns the offset instead (copypaste you kill me) but it should other wise work. even if you set it before playing. but if it doesn't work until you start the file playing just set the pitch right after calling the play method and it should work for now but that is probably a bug and needs to be fixed.

[ForeverZer0]

Yeah, I already made it like that and committed. Its not really a critical bug, simply calling it right after works well enough.

[Ryex]

also it seems you forgot to commit a file

ExpGrip_Dialog.py ?

[ForeverZer0]

Oops. You are correct, I'm committing them now.

[Blizzard]

So basically you are experiencing a memory leak when you play files as if the old ones haven't been destroyed? I'll take a look at it in XAL and if I can't find anything, I'll see if I can give some suggestions what should be checked in PyXAL.

[Ryex]

also it seems that the value returned by getOffset is only updated when the sound is paused or otherwise stopped AND it returns the bytes into the buffer not seconds as expected but the float type of the value

I like how we are acting as a debug team for your company's systems ARC's as well I suppose but the irony is delicious. you've roped in freelacers that arn't getting paied a cent to test and help you refine a system created by your company! you sneaky bastard you. your awesome.

and yes somewhere in the process of creating and destroying sounds there is a significant memory leak roughly equivalent to the size of loading a file. the wrapper can't be involved because it is treated as a python object and garbage collected. the wrapper only keeps a pointer to the XAL object. a pointer that becomes invalid when the object is destroyed via the xal::mgr.destroy[Sound/Player] call.

[ForeverZer0]

Here's some other things to check, some bugs, some just wish lists. (Not all of these are in the C++ implementation, but just making a small compiled list)

• getOffset's return value is in bytes


• Changing the pitch without the audio actually playing does nothing (not tested with volume)


• Return value for getPitch() is always 0.0


• The memory leak


• The offset value is not set until playback is paused. I would really love for this to be kept updated for providing the scrollbar functionality

That's all I can think of for now.

EDIT: Ryex posted while I wa typing, but I am posting anyways

[Blizzard]

Thanks for the list. It will make it easier to check everything. I can explain a few things right away.

• This actually depends on the audio system. On Mac OS in OpenAL somebody messed up badly and when using OpenAL's internal function to get the byte offset and it actually returns the time offset. That's why it's a float value. ._. Generally, you are not supposed to use Offset. I can't remember why I made that public getter in the first place. I'll check, maybe I should remove it as it's mostly something used internally to determine where a "paused" sound has stopped.


• That's because of how the audio system works. You first have to play the sound, then you have to set the volume and the pitch. While it should be possible to change the pitch of a sound while it's playing, RMXP handles pitch change by restarting the played sound (if you want to play the same sound with a different pitch). I think this is intentional and while it works that way in the engine, I'm not sure if you need that in the editor.


• Must be a bug. I'll look into it.


• As I said, I'll check that.


• Again the problem with offset. I can see if I am able to implement a getPosition function for that.

@Ryex: A little bit.

[Ryex]

I just fixed the getPitch function earlier. as I said it we returning the offset not the pitch. update and you'll see.

[Blizzard]

Ah, right. xD Then I'll take care of the rest.

[ForeverZer0]

Here's a little test I made to demonstrate the memory leak. Apparently the leak is in sound, but only when a different file is loaded. Loading the same file does not cause an increase in memory.

http://pastebin.com/2nXMjjeK

[Ryex]

quick question do I need to call free on the data pointer passed to the readRawData function? I wouldn't think it was dynamically allocated memory but if it is that would account for part of the memory leak but not all of it it's still there even when you go through a loop of only creating and destroying sounds.

[Blizzard]

I implemented Sound::readRawData. It takes an unintialized "unsigned char**" as argument and returns the size of the array it created. If the return value is greater than 0, make sure you use "delete []" later to avoid memory leaks. If the return value is 0, then no data will have been created and the pointer will be set to NULL.

The reason why it works like this is because some sounds (such as streamed sounds) won't have the whole audio data loaded. That's why I have to load the whole data manually and assign the double pointer argument to that data. Rather than dealing with inconsistencies when the data should be deleted and when not, I simply made it so you always have to use "delete []" on the data you get. What you do with the data before you destroy it is up to you, but you have to delete it yourself.

[Ryex]

ok so that was DEFIANTLY a memory leak. I called free on the array after I have gotten the python string and the memory usage dropped dramatically in my test.
if you have time blizz you might want to do a test of your own in strait C++ and see if you can see a memory leak because I'm no longer sure if it exists. for me the memory usage isn't rising decernably when changing sounds.

[Blizzard]

We haven't had a memory leak in XAL for ages. I don't think there is one, because I checked and tested the code many, many times.

[Ryex]

just in case could you take a look at the pyx file and see if you can see any other places where memory might be leaking or where I'm doing something wrong (like deleting the 'hstr's that I'm passing that I create from python strings)? Cython takes a knowledge of C++ and python and I only have a passable knowledge of the former

[Blizzard]

hstrs don't need any cleaning.
I suggest you manually use "del" with PyXAL objects. This skips the garbage collector and might remove the problem. Or maybe you should implement the __del__ function in the PyXAL objects (instead of dealloc or whatever it is called).

[Ryex]

I see
what should I put in the __del__ method then? it seems counter intuitive to destroy the sound or player there as the manger interface should still theoretically be able to find and wrap those objects again.
and id I do make it so that the __del__ method destroys the sound or player I need to make sure that I either remove access to the MGR.findPlayer method or make it keep track of wrapper instances so that two wrappers both of which would call destroy when garbage collected don;t exist at the same time. as the second call to destroy would fail.

[Blizzard]

Actually this all depends on how you want to implement things. If you are making a 1:1 implementation of the C++ implementation, then no, you don't need __del__. But if you are wrapping every class separately, you do need it.
Keep in mind that Python 2.5 has problems with the garbage collector if you have a recursive object reference (IDK if 2.6 has it as well, I think it wasn't fixed until a much later version). That's why you should use "del sound" either way.

[Ryex]

how about this. in the player object I'll keep a reference to the sound object that created it. the del statement works by removing the reference to the object in the local scope so that garbage collection would pick up and dispose the object so the deletion of the sound object would only dispose the sound if no player that used the sound were still around in the players delete method I'll destroy the player and delete the reference to the sound object. and in the sounds delete method I'll destroy the sound.

Also I think I'm going to look into implementing this binding with the python ctypes library. that way we don't have to worry about rebuilding pyxal for other operating systems we would just need to build XAL for them and the ctypes would take care of the rest.

EDIT: Nevermind, ctypes can only talk to c function so if we wanted to use it we would need to provide a flattened extern c interface dll to XAL Cython is much better for our uses.

also a hstr created with 'hstr* string = new hstr(char*)' does not need to be deleted later?

[Ryex]

ok, so I re-factored PyXAL a bit. same interface on the surface but it should now force the garbage collection in python to destroy the C++ stuff too. the GC in 2.7 seems great it destorys the sounds as soon as all the references to them is lost (that includes players playing them)

[Blizzard]

You don't ever need to use a hstr pointer. Consider hstr a base type like int or float. If you create a hstr from a char pointer, the data will be copied and you still have to delete the original char pointer if you allocated it with the new operator.

[Ryex]

ok so. whats new in XAL 2.2?

[Blizzard]

Damn, you're fast to notice. >_>
It has some threading issues fixed along and should be much more stable now. It also has updated code for XLN files to be extension insensitive.

And I just realized that I didn't compile it with WAV support yet again. >.<

EDIT: Fixed. I also tagged these versions in the SVN repository as xal/tags/2.2 and hltypes/tags/1.4.1 so they can be recompiled any time if needed.

[Ryex]

Ok, so I updated the interfaced to Pyxal the best I could but I can't be sure I got everything. can you list the recent changed tot he interface so I can be sure?

[Blizzard]

There shouldn't be too many changes. I don't remember which SVN revision of XAL/HLTypes I built last time so I can't really check on the interface. :/ Your best bet is to go through the headers and update everything accordingly. Though, I think the only significant changes should be in AudioManager.h as a couple of methods were renamed.