ok, so i'm finished with bother versions of the module
I've had to edit the ruby version as there were a few things I missed that needed to be done to make it work across languages. also I had to facilitate a few fringe cases like nested classes and things like the table class.
the python version is a bit slower than the ruby version about 9 times slower on load but that was expected due to python naturally slower looping and the fact the fact that the python version of msgpack unpacks arrays as tuples which are imitable. so I have to write a deep conversion method to loop though nested lists of tuples and convert them to python lists that ate about a third of the load time. dump time on the python version is roughly twice the dump time on the ruby version
also I though I should point this out, despite being written in pure ruby arc files load THREE TIMES FASTER than marshal. you heard me right ARC games load and reconstitute objects three times faster than RMXP. that means that black time when you first open the game will be cut down by 2/3. impressive yes?
the ruby version dose however dump slightly slower so saving will take a bit longer. but hey it only slower by about a 1/3 so it's not that bad.
python's dumping of arc is about 1/5 slower than ruby's so not bad there.
I've tested both and they both work perfectly. I tested by using ruby to load a existing RMXP project and dump it into arc, from there I loaded the arc files back up and dumped back to RMXP and tested the files by loading them with RMXP and test playing. for python I took the arc files dumped from ruby and loaded them, I then dumped back to arc in a different location I then used ruby to load the python dumped files and dump to RMXP and used RMXP to open these new files.
if not for a bug in how my recreation of RMXP's Table class in ruby and how it dumps to marshal the games would be playable. tables when dumped from my ruby class are for some reason consided to be 3d even if both ysize and zsize are 1 thus the default scripts raise "wrong number of arguments" errors trying to use 2d indexes to fetch data.
but this isn't a problem because why on earth would you need to convert ARC files to RMXP anyway
test logs for you viewing pleasure
(
Load: 0.321018 Seconds,
Dump: 0.508029 Seconds)
C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump>arc_dump_io.rb
================================
Importing RMXP data to ARC
================================
- Loading ./Data/Actors.rxdata...
- Loading ./Data/Classes.rxdata...
- Loading ./Data/Skills.rxdata...
- Loading ./Data/Items.rxdata...
- Loading ./Data/Weapons.rxdata...
- Loading ./Data/Armors.rxdata...
- Loading ./Data/Enemies.rxdata...
- Loading ./Data/Troops.rxdata...
- Loading ./Data/States.rxdata...
- Loading ./Data/Animations.rxdata...
- Loading ./Data/Tilesets.rxdata...
- Loading ./Data/CommonEvents.rxdata...
- Loading ./Data/System.rxdata...
- Loading ./Data/MapInfos.rxdata...
- Loading ./Data/Scripts.rxdata...
- Loading ./Data/Map001.rxdata...
Compleated Loading in: 0.321018 Seconds
- Dumping ARC ./import/Data/Actors.arc...
- Dumping ARC ./import/Data/Classes.arc...
- Dumping ARC ./import/Data/Skills.arc...
- Dumping ARC ./import/Data/Items.arc...
- Dumping ARC ./import/Data/Weapons.arc...
- Dumping ARC ./import/Data/Armors.arc...
- Dumping ARC ./import/Data/Enemies.arc...
- Dumping ARC ./import/Data/Troops.arc...
- Dumping ARC ./import/Data/States.arc...
- Dumping ARC ./import/Data/Animations.arc...
- Dumping ARC ./import/Data/Tilesets.arc...
- Dumping ARC ./import/Data/CommonEvents.arc...
- Dumping ARC ./import/Data/System.arc...
- Dumping ARC ./import/Data/MapInfos.arc...
- Dumping ARC ./import/Data/Scripts.arc...
- Dumping ARC ./import/Data/Map001.arc...
Compleated Dumping in: 0.508029 Seconds
Done
writeing log
(
Load: 1.0759999752 Seconds,
Dump: 0.654000043869 Seconds)
C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump>arc_dump_io.py
================================
Python ARC load and dump test
================================
- Loading ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_i
port/Data/Actors.arc...
- Loading ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_i
port/Data/Classes.arc...
- Loading ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_i
port/Data/Skills.arc...
- Loading ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_i
port/Data/Items.arc...
- Loading ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_i
port/Data/Weapons.arc...
- Loading ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_i
port/Data/Armors.arc...
- Loading ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_i
port/Data/Enemies.arc...
- Loading ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_i
port/Data/Troops.arc...
- Loading ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_i
port/Data/States.arc...
- Loading ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_i
port/Data/Animations.arc...
- Loading ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_i
port/Data/Tilesets.arc...
- Loading ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_i
port/Data/CommonEvents.arc...
- Loading ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_i
port/Data/System.arc...
- Loading ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_i
port/Data/MapInfos.arc...
- Loading ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_i
port/Data/Scripts.arc...
- Loading ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_i
port/Data/Map001.arc...
Loading took 1.0759999752 s
- Dumping ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_e
port/Data/Actors.arc...
- Dumping ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_e
port/Data/Classes.arc...
- Dumping ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_e
port/Data/Skills.arc...
- Dumping ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_e
port/Data/Items.arc...
- Dumping ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_e
port/Data/Weapons.arc...
- Dumping ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_e
port/Data/Armors.arc...
- Dumping ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_e
port/Data/Enemies.arc...
- Dumping ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_e
port/Data/Troops.arc...
- Dumping ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_e
port/Data/States.arc...
- Dumping ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_e
port/Data/Animations.arc...
- Dumping ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_e
port/Data/Tilesets.arc...
- Dumping ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_e
port/Data/CommonEvents.arc...
- Dumping ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_e
port/Data/System.arc...
- Dumping ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_e
port/Data/MapInfos.arc...
- Dumping ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_e
port/Data/Scripts.arc...
- Dumping ARC C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump/python_e
port/Data/Map001.arc...
Dumping took 0.654000043869 s
Done
(
Load: 0.18401 Seconds,
Dump: 0.736042 Seconds)
C:\Users\Ben\Desktop\ARC\SVN\editor\ARCed\tests\ARC_dump>arc_dump_io.rb
================================
Exporting ARC data to RMXP
================================
- Loading ARC ./python_export/Data/Actors.arc...
- Loading ARC ./python_export/Data/Classes.arc...
- Loading ARC ./python_export/Data/Skills.arc...
- Loading ARC ./python_export/Data/Items.arc...
- Loading ARC ./python_export/Data/Weapons.arc...
- Loading ARC ./python_export/Data/Armors.arc...
- Loading ARC ./python_export/Data/Enemies.arc...
- Loading ARC ./python_export/Data/Troops.arc...
- Loading ARC ./python_export/Data/States.arc...
- Loading ARC ./python_export/Data/Animations.arc...
- Loading ARC ./python_export/Data/Tilesets.arc...
- Loading ARC ./python_export/Data/CommonEvents.arc...
- Loading ARC ./python_export/Data/System.arc...
- Loading ARC ./python_export/Data/MapInfos.arc...
- Loading ARC ./python_export/Data/Scripts.arc...
- Loading ARC ./python_export/Data/Map001.arc...
Compleated Loading in: 0.18401 Seconds
- Dumping RMXP ./ruby_python_export/Data/Actors.rxdata...
- Dumping RMXP ./ruby_python_export/Data/Classes.rxdata...
- Dumping RMXP ./ruby_python_export/Data/Skills.rxdata...
- Dumping RMXP ./ruby_python_export/Data/Items.rxdata...
- Dumping RMXP ./ruby_python_export/Data/Weapons.rxdata...
- Dumping RMXP ./ruby_python_export/Data/Armors.rxdata...
- Dumping RMXP ./ruby_python_export/Data/Enemies.rxdata...
- Dumping RMXP ./ruby_python_export/Data/Troops.rxdata...
- Dumping RMXP ./ruby_python_export/Data/States.rxdata...
- Dumping RMXP ./ruby_python_export/Data/Animations.rxdata...
- Dumping RMXP ./ruby_python_export/Data/Tilesets.rxdata...
- Dumping RMXP ./ruby_python_export/Data/CommonEvents.rxdata...
- Dumping RMXP ./ruby_python_export/Data/System.rxdata...
- Dumping RMXP ./ruby_python_export/Data/MapInfos.rxdata...
- Dumping RMXP ./ruby_python_export/Data/Scripts.rxdata...
- Dumping RMXP ./ruby_python_export/Data/Map001.rxdata...
Compleated Dumping in: 0.736042 Seconds
Done
writeing log
that last one is converting python arc dumped files not ruby arc dumped files but the times are the same as it is the exact same data, the files are as far as I can tell identical
Ok now onto how to use the features
the methods of the ARC_Dump module are as such
dump(obj, io, redirects = {})obj = the object to be dumped
io = an io object like an open file object
redirects = this is a hash mapping class names to alt names this is used to help with namespace issues with nested classes ect on the python side of ARC_Dump to can map a class name like RPG::System::Words to RGSS_RPG_System_Words and then on the python side of loading it map RGSS_RPG_System_Words to a name space path that will find the proper class object. you have to do the same mapping when you load it in ruby too (python deals with name spaces and nested classes very differently so this is VERY necessarily)
load(io, redirects = {} [,extended_namespace={}]) io = an io object like an open file object
redirects = this is a hash mapping class names to alt names this is used to help with namespace issues with nested classes ect on the python side of ARC_Dump to can map a class name like RPG::System::Words to RGSS_RPG_System_Words and then on the python side of loading it map RGSS_RPG_System_Words to a name space path that will find the proper class object. you have to do the same mapping when you load it in ruby too (python deals with name spaces and nested classes very differently so this is VERY necessarily)
extended_namespace = only available on the python side due to how python deals with namespace this option is here to allow the caller to pass the globals() dictionary extending the namespace within the module temporarily so that the class objects in the loaded object can be found.
all classes have the option of defining the following class methods in ruby or python.
_arc_dumpmust return a string or other basic data type that can be packed via msgpack
it could also raise an error to stop the dump process
_arc_loadthe methods don't
have to do anything.
it accepts the object returned by _arc_dump and should set up the object with the contained data. keep in mind that at this point the only thing that has happened is that the object has been allocated the init method has not been called so any and all instance variables are currently set to nil.
dose not need to return the completed object.
_arc_dump and _arc_load work as a pair, if you define one you MUST define the other or you'll get errors.
_arc_excludeshould return a array or list of strings of the names of instance variables/attributes that should not be dumped with the object (more important for python where everything is public, but default python exclude any attribute prefaced by a _)
_prep_arc_dumpthis method is called just before the object is dumped, it can be used to set up data (this is important for the python side of the table object I use it to convert a private attribute containing the numpy version of the data to a normal python list so it can be dumped normally). this method can be in place regardless whether or not _arc_dump or any of the other methods are defined
_post_arc_loadcalled after the object has been loaded (this is important for the python side of the table object I use it to convert the normal python list of data to a numpy version). this method can be in place regardless of any other methods.
Python is significantly different in how it handles nested classes and instance variables as well as name space thus python classes can define the following global attributes
_arc_class_pathshould contain a string with the namespace path to take to find the class object the module and class names should be separated by "::" not "." as is the python default
_arc_instance_variablesshould contain a list of strings of the names of attributes to dump with the object if it is provided no other attributes are dumped other wise all attributes in the objects __dict__ that are not linked to function objects or prefaced with a _ (names prefaced with a _ are by convention private in python) are dumped with the object.
I'm committing the files to SVN now