ARCed Audio Player

Started by ForeverZer0, November 19, 2011, 03:50:58 pm

Previous topic - Next topic

ForeverZer0

Have you looked into using numpy's FFT functions?
I am done scripting for RMXP. I will likely not offer support for even my own scripts anymore, but feel free to ask on the forum, there are plenty of other talented scripters that can help you.

Ryex

No I haven't, but form the looks of things it might work as we want. why don't you give it a try?

also, this won't work with midi, in fact I'm not sure how pygame will work with midi period. attempting to play midi files with the sound class fails utterly.
I no longer keep up with posts in the forum very well. If you have a question or comment, about my work, or in general I welcome PM's. if you make a post in one of my threads and I don't reply with in a day or two feel free to PM me and point it out to me.<br /><br />DropBox, the best free file syncing service there is.<br />

ForeverZer0

November 20, 2011, 08:21:52 pm #22 Last Edit: November 20, 2011, 08:23:06 pm by ForeverZer0
New screenshot. The waveform is now working correctly, and is not a copy-paste edit anymore. The button bitmaps are subject to change, but they work for now.

Spoiler: ShowHide



EDIT:
Don't mind the "Pitch (Hz)" for the one control. The Hz was accidentally left from when it was a frequency control.... :P
I am done scripting for RMXP. I will likely not offer support for even my own scripts anymore, but feel free to ask on the forum, there are plenty of other talented scripters that can help you.

Reives

Looking fantastic. o: I've always wished that RMXP had waveform display and a scrollable progress bar, and then BAM, here they are.
:~

Ryex

I think I've figure out a way to load media via pyglet's AVbin into pygame's sound class. It hackishly taps into pyglets internal 'private' methods (but everything is public in python) to read data. and will use a trick to create an empty pygame sound so I can obtain a buffer and write the raw bytes to it.

in principle it should work but I'll have to test it to be sure.

the reason I'm looking into it in the first place is because I'm fairly sure pygame can only read ogg, and wav. not flac or any other format we may decide to support.
I no longer keep up with posts in the forum very well. If you have a question or comment, about my work, or in general I welcome PM's. if you make a post in one of my threads and I don't reply with in a day or two feel free to PM me and point it out to me.<br /><br />DropBox, the best free file syncing service there is.<br />

Blizzard

BTW, I forgot what we decided on FLAC suppport. o.o; I think we left it for the actual engine later.
Check out Daygames and our games:

King of Booze 2      King of Booze: Never Ever
Drinking Game for Android      Never have I ever for Android
Drinking Game for iOS      Never have I ever for iOS


Quote from: winkioI do not speak to bricks, either as individuals or in wall form.

Quote from: Barney StinsonWhen I get sad, I stop being sad and be awesome instead. True story.

Ryex

ok I succeeded in loading an a file from pyglet into pygame.

there is a weird behavior where pyglet.media must be imported before SciPy or it woun't be able to load avbin properly


print "++++++++Doing setup, please wait until ready+++++++++"
import os, sys, time, math
import numpy as np
import pyglet
import pyglet.media as media
import scipy.signal as signal

#os.environ["SDL_VIDEODRIVER"] = "dummy"

SAMPLE_RATE = 44100

import pygame.mixer as mixer
import pygame.sndarray as sndarray

mixer.init(frequency=SAMPLE_RATE, size=-16, channels=2, buffer=4096)

file = r"C:\Program Files\Common Files\Enterbrain\RGSS\Standard\Audio\BGS\001-Wind01.ogg"

CHUNKSIZE = 4096

def resample(sound, rate):
    time_now = time.time()
    array = sndarray.array(sound)
    n_samples = len(array)
    new_n_samples = int(n_samples / rate)
    rounds = n_samples / CHUNKSIZE
    samples_per = new_n_samples / rounds
    new_array = np.zeros((new_n_samples, 2))
    for i in xrange(rounds):
        part = signal.resample(array[i*CHUNKSIZE:(i+1)*CHUNKSIZE], samples_per)
        new_array[i*samples_per:(i+1)*samples_per] = part
    #new_array = signal.resample(array, new_n_samples)
    new_sound = sndarray.make_sound(np.ascontiguousarray(new_array, dtype=array.dtype))
    print "resampeling took: %s seconds" % (time.time() - time_now)
    return new_sound
   
def load_pygame_sound_with_pyglet(filename):
    source = media.load(filename)
    static = media.StaticSource(source)
    data = static._data
    info = mixer.get_init()
    typecodes = { 8 : np.uint8,   # AUDIO_U8
                 16 : np.uint16, # AUDIO_U16 / AUDIO_U16SYS
                 -8 : np.int8,   # AUDIO_S8
                 -16 : np.int16  # AUDUI_S16 / AUDIO_S16SYS
                 }
    channels = info[2]
    pyglettypecode = typecodes[source.audio_format.sample_size]
    pygametypecode = typecodes[info[1]]
    array = np.fromstring(data, pyglettypecode)
    if channels == 2:
        array = np.column_stack((array, array))
    sound = sndarray.make_sound(np.ascontiguousarray(array, dtype=typecode))
    rate = float(source.audio_format.sample_rate) / float(SAMPLE_RATE)
    shifted_sound = resample(sound, rate)
    return shifted_sound

pyglet_sound = load_pygame_sound_with_pyglet(file)
sound = mixer.Sound(file)
shifted = resample(sound, 1.5)     

print """======Ready========
Commands: q for quit
n - to play normal sound
s - to play shifted sound
c|float - where float is 0.5 or 1.5 ect. to change the shift
Leave blank to stop playback"""
while True:
    input = raw_input("Command: ")
    if "c|".lower() in input.lower():
        sound.stop()
        shifted.stop()
        rate = float(input.lower()[2:])
        print "resampeling to: %s"  % rate
        shifted = resample(sound, rate)
    elif "q".lower() in input.lower():
        print "quiting"
        break
    elif "n".lower() in input.lower():
        sound.stop()
        shifted.stop()
        pyglet_sound.stop()
        print "playing normal sound"
        sound.play()
    elif "s".lower() in input.lower():
        sound.stop()
        shifted.stop()
        pyglet_sound.stop()
        print "playing shifted sound"
        shifted.play()
    elif "p" in input:
        sound.stop()
        shifted.stop()
        pyglet_sound.stop()
        print "playing sound loaded with pyglet and resampled"
        pyglet_sound.play()
    else:
        sound.stop()
        shifted.stop()
        pyglet_sound.stop()
        print "stoping playback"
       
I no longer keep up with posts in the forum very well. If you have a question or comment, about my work, or in general I welcome PM's. if you make a post in one of my threads and I don't reply with in a day or two feel free to PM me and point it out to me.<br /><br />DropBox, the best free file syncing service there is.<br />

Blizzard

Feel free to use a bigger bufffer and chunk size. 16384 is just fine. Just be careful as shorter sounds may have problems in being played with a bigger buffer. In that case, you can revert it back to 4096.

BTW, do you know why the often used default is 4096?
Check out Daygames and our games:

King of Booze 2      King of Booze: Never Ever
Drinking Game for Android      Never have I ever for Android
Drinking Game for iOS      Never have I ever for iOS


Quote from: winkioI do not speak to bricks, either as individuals or in wall form.

Quote from: Barney StinsonWhen I get sad, I stop being sad and be awesome instead. True story.

Ryex

Ok Zero here is how we are are going to do this. I'm going to use cython to wrap XAL (our audio library). Blizz is going to add seeking and a way to get raw audio data and I'll wrap thoes too. after that the editor will be able to play anything the engine can play. no more worries on our side.
I no longer keep up with posts in the forum very well. If you have a question or comment, about my work, or in general I welcome PM's. if you make a post in one of my threads and I don't reply with in a day or two feel free to PM me and point it out to me.<br /><br />DropBox, the best free file syncing service there is.<br />

Blizzard

Oh, BTW, Ryex. You will also have to compile hltypes for XAL to work. This should work without problems, though. I mean, I can compile you the DLLs and LIBs.
Check out Daygames and our games:

King of Booze 2      King of Booze: Never Ever
Drinking Game for Android      Never have I ever for Android
Drinking Game for iOS      Never have I ever for iOS


Quote from: winkioI do not speak to bricks, either as individuals or in wall form.

Quote from: Barney StinsonWhen I get sad, I stop being sad and be awesome instead. True story.

ForeverZer0

That sounds good. My implementation of a seek function and keeping the slider updated with timers was a bit sketchy.
I am done scripting for RMXP. I will likely not offer support for even my own scripts anymore, but feel free to ask on the forum, there are plenty of other talented scripters that can help you.

Ryex

ok, I've successfully wrapped XAL and it can load and play ogg files. I had to slightly change the interface to get it to wrap properly.

you create a PyXAL.XALManger object to init XAL
before you destroy the XALManager object you must call it's DestroyXAL method as it doesn't seem to happen automatically
to load a sound call 'XALManager.createSound(filename)' this will return a PySound object which is an interface for the xal::sound object
then create a player for that sound with 'XALManager.createPlayer(PySound)' passing the sound object returned from createSound. this will return a PyPlayer object whihc is an interface for a xal::player object
then just call 'player.play()' ext as normal

not that it currently crashes python if you try to load a wav file. this needs to be fixed. loading a file that xal can't decode needs to throw a exception so that python can recover.
I no longer keep up with posts in the forum very well. If you have a question or comment, about my work, or in general I welcome PM's. if you make a post in one of my threads and I don't reply with in a day or two feel free to PM me and point it out to me.<br /><br />DropBox, the best free file syncing service there is.<br />

Blizzard

Hm... XAL only logs the error if it can't load a file. This is so loading is sped up (because exceptions are slow) and unsupported files can be simply just ignored. I can add an additional parameter during initialization called "rigorousHandling" which will be false by default and that will only log when false and throw exceptions when true. Does that work fine with you? Alternatively createSound will actually return NULL in C++ (so it should be "None" in Python) if a sound could not be loaded so you can actually detect when a file was trying to be loaded that isn't supported. I suggest that you use the return value of createSound to detect that.
Check out Daygames and our games:

King of Booze 2      King of Booze: Never Ever
Drinking Game for Android      Never have I ever for Android
Drinking Game for iOS      Never have I ever for iOS


Quote from: winkioI do not speak to bricks, either as individuals or in wall form.

Quote from: Barney StinsonWhen I get sad, I stop being sad and be awesome instead. True story.

Ryex

ah, if it returns NULL then there is another option. I just have to make sure i'm checking for that return value and act on it. I thought for a moment that it was just failing internally and there was nothing I could do.

also, why is WAV support currently not included?
I no longer keep up with posts in the forum very well. If you have a question or comment, about my work, or in general I welcome PM's. if you make a post in one of my threads and I don't reply with in a day or two feel free to PM me and point it out to me.<br /><br />DropBox, the best free file syncing service there is.<br />

Blizzard

Actually it should be working. O_o

Try following: Compile the XAL demo_simple and try the wav there. If it doesn't work there, it's not a normal PCM wav file.

Alternatively try to check the log that XAL prints out. Make sure you are accessing the right file. XAL tries to load files relative from it's cwd. The loading might have failed, because the path is wrong and the file actually doesn't exist there. Try loading an absolute path just to see if the problem is with the path spec or with the wav loading.
Check out Daygames and our games:

King of Booze 2      King of Booze: Never Ever
Drinking Game for Android      Never have I ever for Android
Drinking Game for iOS      Never have I ever for iOS


Quote from: winkioI do not speak to bricks, either as individuals or in wall form.

Quote from: Barney StinsonWhen I get sad, I stop being sad and be awesome instead. True story.

Ryex

OK!

I've wrapped all XAL functionality to date, or all useful functionality at least.
a few notes:
there is a test file PyXAL_Test.py in editor/ARCed/src
pitch shifting is still not there :(
WAV files do not set their duration when they are loaded. I implemented a calculation in the test file so we could do it our selves if we have too but that should really be fixed.
seek functionality is not there Blizz said that XAL is not in a position to do it properly for wav files so he isn't doing it yet

sound.readRawData return a 2 tuple of (size_in_bits_not_bytes, data_string)
I implemented a function to properly create a numpy array from the data in the test file, the 24 bit integers that were in the wav files I tested were a bitch to convert...
the function returns a 2d array of the left and right channel data I barrows FZero's waveform panel and implemented it in the test to demonstrate how to draw the left and right audio channels

an Image for you:
Spoiler: ShowHide


@FZer0 you should be able to implement most of the audio panel now just disable the position scroll bar so the user can't interact with it. you can still update it's position to match the offset of the music but disable user input.
as for setting pitch just comment out the liens where the pitch of the playing sound would be set
the play method accepts two arguments,
sound.play(float fade_time, bool looping)
so implanting looping is easy
and as I said I implemented a function to create an array form the raw data and demonstrated drawing the left and right audio channel in a graph.
I no longer keep up with posts in the forum very well. If you have a question or comment, about my work, or in general I welcome PM's. if you make a post in one of my threads and I don't reply with in a day or two feel free to PM me and point it out to me.<br /><br />DropBox, the best free file syncing service there is.<br />

Blizzard

November 26, 2011, 06:38:15 am #36 Last Edit: November 26, 2011, 08:04:27 am by Blizzard
Quote from: Ryex on November 26, 2011, 02:10:23 am
WAV files do not set their duration when they are loaded. I implemented a calculation in the test file so we could do it our selves if we have too but that should really be fixed.


OMG, I'm so sorry. ._.;;; I never implemented the calculation for it after loading the WAV data. I implemented it now and commited the new binaries and libs. Basically duration = byteSize / (samplingRate * channels * bytesPerSample), but you probably know that already.

EDIT:

Quote from: Ryex on November 26, 2011, 02:10:23 am
pitch shifting is still not there :(


I just implemented xal::Player::setPitch and it works. The value is a float where 1.0 means 100%, 0.5 means 50% and 2.0 means 200%. Implement everything accordingly in the editor. I updated the binaries and libs again, but I didn't interface them to Python.

This feature is currently only supported by the DirectSound audio system.
Check out Daygames and our games:

King of Booze 2      King of Booze: Never Ever
Drinking Game for Android      Never have I ever for Android
Drinking Game for iOS      Never have I ever for iOS


Quote from: winkioI do not speak to bricks, either as individuals or in wall form.

Quote from: Barney StinsonWhen I get sad, I stop being sad and be awesome instead. True story.

ForeverZer0

There is a memory leak in the current implementation. I cannot for the life of me find it, everything appears to be being disposed properly, but nonetheless. it is there.  I tracked it down to the "on_change_file" method of the PyXAL_Test file. Simply hardcode a path to the "file" attribute, and have the play button call this event to see for yourself. The more you press it, the more memory usage just keeps going up. The player and the sound are both being destroyed, which I confirmed with print statements. I'm at a bit of a loss at the moment...
I am done scripting for RMXP. I will likely not offer support for even my own scripts anymore, but feel free to ask on the forum, there are plenty of other talented scripters that can help you.

Ryex

November 26, 2011, 03:22:21 pm #38 Last Edit: November 26, 2011, 05:30:45 pm by Ryex
if there is a memory leak it is in XAL because I'm calling the destroy methods... wait no... oh crap! one sec I think I forgot to write the __dealloc__ methods for the sound and player!


EDIT:

actually no there was a reason I didn't write the dealloc methods for the sound and player even if the interface is destroyed XAL still keeps the objects internally and has the ability to retrieve them at which point they are wrapped with another python interface. the python interface is being destroyed properly so the memory leak is indeed in XAL.

I even tried seeing if the memory dropped after xal::destroy was called. It doesn't. the memory goes down slightly if you destroy the sound with out playing it but if you play it first the memory usage goes up and then stays up.

in any case I wrapped the set and get pitch methods. I also put doc string on everything and called the help() function to generate a help file for you.
editor/ARCed/src/Core/PyXAL/PyXAL_Help.txt
I no longer keep up with posts in the forum very well. If you have a question or comment, about my work, or in general I welcome PM's. if you make a post in one of my threads and I don't reply with in a day or two feel free to PM me and point it out to me.<br /><br />DropBox, the best free file syncing service there is.<br />

ForeverZer0

November 26, 2011, 05:21:49 pm #39 Last Edit: November 26, 2011, 06:07:28 pm by ForeverZer0
I have just been using the XAL C++ source as a reference. Its not really difficult.

EDIT:

The audio player is almost done. I already implemented pitch for when it was done, so now I can actually test it. For now I just have a flag set to determine if pitch changing is permitted or not:

PITCH_ENABLED = sys.platform == 'win32'



EDIT2:

Setting the pitch officially does nothing. I can create an instance of a player, then:

self.player.setPitch(2.0)
self.player.getPitch()
--> 0.0


Also, it would be slightly easier to use if the values could be passed with the initializer as they are in the C++ version. That's not a big deal at all, though.

EDIT3:

As it turns out, it works well when there is a file playing, but not if it is set beforehand.
I am done scripting for RMXP. I will likely not offer support for even my own scripts anymore, but feel free to ask on the forum, there are plenty of other talented scripters that can help you.