Chaos Project

Game Development => Sea of Code => Topic started by: G_G on December 07, 2013, 11:17:13 pm

Title: Reading Ruby's Marshal Format
Post by: G_G on December 07, 2013, 11:17:13 pm
So I've been trying to study Ruby's Marshal binary format. And I've figured out a few things and I'm still trying to figure a couple of things out, maybe some of you guys can help. I've only serialized a few data types and identified a few key bytes.

Here's some code.
class ZObject
 attr_accessor :data
 def initialize(obj = nil)
   @data = obj
 end
end

 obj = [ZObject.new(true), ZObject.new(false)]
 file = File.open("format.bin", "wb")
 Marshal.dump([5, 6, 7, 8], file)
 Marshal.dump(obj, file)
 file.close


Here's the output in bytes and the respective characters

04 08 5B 09 69 0A 69 0B 69 0C 69 0D 04 08 5B 07 - ..[.i.i.i.i...[.
6F 3A 0C 5A 4F 62 6A 65 63 74 06 3A 0A 40 64 61 - o:.ZObject.:.@da
74 61 54 6F 3B 00 06 3B 06 46                   - taTo;..;.F


I read somewhere that the first two bytes are the Marshal's Major and Minor version. So in this case, It would be 4.8.
Arrays are identified with the byte 0x5B, the byte afterwards is the number of entries in the array. However, what's odd, integers are serialized as 5 higher than they should. 0x09 is the size of the array, subtract 5 and you get 4. Integers are identified with the byte 69 (giggity) and the byte(s) following are the actualy integer. Again, following the 5 higher than the real value. Not sure why it is yet, and I've only done small so far, I haven't done 255 or higher. There are no bytes identified as separators. One thing I found out, everytime Marshal.dump is called, it dumps the version number again, as you'll see, after the first array was dumped, you'll find the bytes 04 and 08 again.

True is identified as a capitol T (0x54) while False, capitol F (0x46).

Now let's look at bytes 6F, 3A, 0C, at the beginning of the 2nd line. I'm not entirely sure, but I've narrowed 6F to tell Ruby it's a serialized class. The byte afterwards 3A, I'm not too sure about, I think it specifies a symbol, class, variable, because right after is the size of the class name/variable name. 0C is the value size of the class name, 0C is 12, minus 5, is 7, which is the number of characters in ZObject. The byte right after ZObject, 06, I'm not quite sure what this one means. When a variable is serialized, the data belonging to the variable is immediately serialized. If an integer, it'll be writtien as an i then the number.

Now onto something that I noticed, if an object has already been serialized in the same Marshal.dump call, it gets more compressed.

6F 3B 00 06 3B 06 46 - o;..;.F

These last bytes in the file are the 2nd ZObject, so it gets more compressed. What I'm trying to understand here, is how these list of bytes is equivalent to the first serialized ZObject. Other than @data being set to true rather than false. Is it generating a checksum?

So I added a new class with the same variable name and discovered something else. When serializing data, to reduce space, it seems to generate a byte declaring the name of a class, symbol, variable.
New Code (Only serializes two objects, one ZObject and one SObject. Each has a variable named @data)
Spoiler: ShowHide
class ZObject
 def initialize(obj = nil)
   @data = obj
 end
end
class SObject
 def initialize(obj = nil)
   @data = obj
 end
end
begin
 obj = [ZObject.new(true), SObject.new(false)]
 file = File.open("format.bin", "wb")
 Marshal.dump(obj, file)
 file.close
 exit
rescue
 puts $!
 gets
 exit
end


It generates this. (Pay attention to the highlighted areas)

04 08 5B 07 6F 3A 0C 5A 4F 62 6A 65 63 74 06 3A - ..[.o:.ZObject.:
0A 40 64 61 74 61 54 6F 3A 0C 53 4F 62 6A 65 63 - .@dataTo:.SObjec
74 06 3B 06 46                                  - t.;.F


As you can see, the bytes highlighted in green are @data serialized in the ZObject, however, when declared and serialized in the SObject, they don't come out the same. You can tell they're getting more compressed. I'm still unsure what the byte 06 is identified as (the one that comes right after the last character in a class name).

I'm wondering if anyone can help me figure out or knows anything about how they decide how to compress it.
Title: Re: Reading Ruby's Marshal Format
Post by: orochii on December 08, 2013, 02:33:15 am
I'm going to guess about the red part. "3B" means repeating something. For example, when it says 3B 00 it is stating a repetition of a class (probably the previous one). When it says 3B 06 it is meaning to a repetition for a variable (or maybe a boolean?).
It could help dumping something like this:
[ZObject.new, SObject.new, ZObject.new, SObject.new]

Just to see what happens with the first ZObject name/class/type/something? reference, if it is actually used when writing the second ZObject or recreated.
If it is reused, then we have the second part of our plan: see what reference does the SObject class gets when adding a second one after adding a ZObject.

So, I did that test, defining four objects from 2 different classes in that way. I used your code just to speed up things. Here is the result:
04 08 5B 09 6F 3A 0C 5A 4F 62 6A 65 63 74 06 3A - ..[.o:.ZObject.:
0A 40 64 61 74 61 54 6F 3A 0C 53 4F 62 6A 65 63 - .@dataTo:.SObjec
74 06 3B 06 46 6F 3B 00 06 3B 06 54 6F 3B 07 06 - t.;.Fo;..;.To;..
3B 06 46                                        - ;.F


My assumption is that when reading and writing, it creates a reference list. So for example, it uses 6F for serialized class, 3A for "new reference", and then the reference bytesize. 3B means existing reference, and then throws the reference ID (00 and 07, I wonder what makes it select those numbers).
Same for variables, where it uses 06 for variable (or something?), then 3A or 3B for references.

I wonder what happens when the array has more than 255 different-class objects,
Orochii Zouveleki
Title: Re: Reading Ruby's Marshal Format
Post by: Blizzard on December 08, 2013, 04:35:56 am
I can actually help here, because I've worked a lot with this format. After all, Ryex and me made a Python implementation.

1. Yes, 4.8 is the format version and it's dumped every time you call Marshal.dump.

2. Integers are compressed. 0x00 is simply 0, 0x05 is 1, 0x7F is 122, 0xFF is -122 and 0xFA is -1. This offset by 5 is there because when an integer is 1, 2, 3 or 4, it means that the following X bytes represent an integer. This is done to save space. e.g. if it's 3, then the following 3 bytes are a little endian encoded integer (e.g FF FF 01 woud be 131071).

3. Yes, 6F (the character 'o') means an object follows.

4. The ':' character after 'o' means String since the next thing is the class name.

5. If there is a ';', it means "This is a reference to an object that was already serialized previously", followed by the ID of the object. e.g. If you serialized 3 objects, the value after ';' would be 0x05, 0x06 or 0x07 (1, 2 or 3). So it's not another object that really gets compressed, it's literally a reference to a previously serialized object.
Just keep in mind that Strings may be put there as reference, but they should always be new objects. e.g. If you write a reader, make sure to save the first occurrence of a String, but always use Object#clone when you need to access it.

6. You already did the test with ZObject and SObject both having a variable called @data. @data is a string in the file and it gets treated like other objects, so it uses the ';' system for references.

7. The value after ZObject is yet another integer, indicating the number of instance variables that follow.




I suggest that you check out this topic: http://forum.chaos-project.com/index.php/topic,11920.0.html
The ARC Data format was inspired by Ruby's Marshal, except that it was simplified. The files end up a bit bigger usually, because there is no hardcore integer compression, but the format is easier and faster to read. It might give you some more insights into serialization in general. I even use a similar format for my Lite Serializer library.
Title: Re: Reading Ruby's Marshal Format
Post by: G_G on December 08, 2013, 10:15:52 am
Oh man, that really clears things up Blizzard. Just one more question, how is the id generated for classes/strings to be referenced for later?
Title: Re: Reading Ruby's Marshal Format
Post by: Blizzard on December 08, 2013, 12:24:55 pm
They start at 1 (or 0, I'm not 100% sure anymore) and are incremented for each object and string. So if a reference ID is 2, then it points to the 2nd object that was read from the file.
Title: Re: Reading Ruby's Marshal Format
Post by: G_G on December 08, 2013, 12:27:17 pm
Ah okay. And I'm assuming they're only given reference IDs if they're found in the file again? In my last test @data is given ID 1 rather than 2. Even though it read ZObject first. But if ZObject had been dumped again, it would have been ID 1?
Title: Re: Reading Ruby's Marshal Format
Post by: Blizzard on December 08, 2013, 01:50:34 pm
Yes. The ID is never written down anywhere, it's implicitly defined by the order of the objects. But it probably is saved somewhere internally during reading/writing. e.g. in ARC Data we simply use an array and use "index + 1" as ID.
Title: Re: Reading Ruby's Marshal Format
Post by: G_G on December 08, 2013, 02:20:56 pm
Yup, they start at 0. I did another test. So for every string object that gets read, just store an ID and increment it by 1. That way when I run into it, I can just reference it. Thanks for the help Blizzard. I'm gonna be looking into hashes now.
Title: Re: Reading Ruby's Marshal Format
Post by: Blizzard on December 08, 2013, 02:57:33 pm
The same goes actually for strings, arrays, hashes and objects. Just keep in mind that they all increment the counter. If you put a string, then an object, the object will have ID 1, not 0.
Title: Re: Reading Ruby's Marshal Format
Post by: G_G on February 16, 2014, 03:15:53 am
So I was poking around with RMXP again and I was looking at it's clipboard data. All it's clipboard data consists of is serialized ruby data. I haven't been able to figure out the first few bytes of information is at it seems to be different from everything I copy. Every piece of data is labeled differently. I was able to grab the data in C# and read some of the basic data just because I had an idea in mind for something. Anyways, I just thought it was kinda cool.

Spoiler: ShowHide
(http://puu.sh/6Y6rl.png)
Title: Re: Reading Ruby's Marshal Format
Post by: ForeverZer0 on February 16, 2014, 02:24:38 pm
I actually have all the clipboard data figured out.  During mu absence from the forum, I worked on an editor RMXP in .NET and XNA. Its actually over 90% done, even have a fully working map editor with autotiles, etc. Anyways, I made sure to allow cutting and pasting objects between the Enterbrain editor and my own, and its actually not a very hard thing they did, and I would be happy to show you the source for the clipboard data.

My implementation uses IronRuby for the Marshall, but all the data is simply a Ruby Marshall object. The first few bytes are the number of bytes of the object. If I remember correctly, the only exception is copying map data (using the selector tool on the map to copy/cut a section).  It is a multi-dimensional array, using width, height, layer, and tile IDs.

Here's an example of setting a map event to the clipboard, the "Ruby.MarshalDump" method returns an array of bytes (byte[]):
Spoiler: ShowHide
		public void CopyMapEvent()
{
if (!CanCopy)
return;
var mapEvent = GetEventAt(pointEvent);
if (mapEvent == null)
return;
var data = Ruby.MarshalDump(mapEvent);
var stream = new MemoryStream(data.Length + 4);
stream.Write(BitConverter.GetBytes(data.Length), 0, 4);
stream.Write(data, 0, data.Length);
Clipboard.SetData("RPGXP EVENT", stream);
CanPaste = true;
}


And here's pasting...
Spoiler: ShowHide
		public void PasteMapEvent()
{
if (!CanPaste || !Clipboard.ContainsData("RPGXP EVENT"))
return;
if (GetEventAt(pointEvent) != null)
return;
CreateUndoEventEntry("Paste Event");
var stream = (MemoryStream)Clipboard.GetData("RPGXP EVENT");
var size = BitConverter.ToInt32(stream.ReadBytes(4), 0);
var data = new byte[size];
stream.Read(data, 0, data.Length);
var mapEvent = Ruby.MarshalLoad(data);
var id = 0;
do
{
id++;
} while (Map.events.ContainsKey(id));
mapEvent.id = id;
mapEvent.x = pointEvent.X;
mapEvent.y = pointEvent.Y;
Map.events[id] = mapEvent;
Invalidate();
}


Obviously there are a few methods in there that aren't shown, but you should get the idea. Almost all objects in RMXP use the same way of setting and getting from the clipboard.
Title: Re: Reading Ruby's Marshal Format
Post by: G_G on February 16, 2014, 03:03:11 pm
Thanks F0! I'd love to check out the source code, but my overall goal of this thread was to create my own Ruby (De)Serializer in C#. My side project I had in mind would be a lot easier with IronRuby but my overall goal is to create the library on my own so developers can easily access RMXP's data in their own code.

Regardless, it'd be cool to check out the code. Thanks bud. :3
Title: Re: Reading Ruby's Marshal Format
Post by: ForeverZer0 on February 16, 2014, 03:33:01 pm
From what you say, you could still easily access the data using C# without the need for a custom serializer, just using IronRuby.
In my project, I did use this for a few special instances, specifically the map data. It was a bit cumbersome using a dynamic object, so I created a C# map class that took a dynamic Ruby object in the initializer to create the object.

In each class there is a private field, simply "data" that contains the actual Ruby data, but all the public getters and setters read that data and convert it back and forth between CLR types. For example (not actual code):

Spoiler: ShowHide
namespace RPG
{

   public class Actor
   {
       /// <summary>
       /// The dynamic Ruby instance of the object
       /// </summary>
       private dynamic _data;

       /// <summary>
       /// Gets or sets the actor's name
       /// </summary>
       public string Name
       {
           get { return _data.name.ToString(Encoding.UTF8); }
           set { _data.name = MutableString.Create(value, RubyEncoding.UTF8); }
       }

       // Do something similar with all properties


       /// <summary>
       /// Create a new actor object
       /// </summary>
       /// <param name="actor">A Ruby instance of an RPG::Actor object</param>
       public Actor(dynamic actor)
       {
           _data = actor;
       }
   }
}



Title: Re: Reading Ruby's Marshal Format
Post by: Blizzard on February 16, 2014, 04:05:06 pm
I'm trying to remember whether we ever implemented a Ruby Marshall reader/writer in Python for ARC. If yes, you can use the source code as a guide to make a C# implementation. Technically you could also download Ruby's source code and take a look at the C code, but it's more complicated due to C's low level.
Title: Re: Reading Ruby's Marshal Format
Post by: Ryex on February 16, 2014, 04:15:33 pm
we did, but Our understanding of how tables were serialized was flawed at the time and it didn't work so we scraped it. I've been looking for the code for a long time as we used to have it but it's not in the ARC source control. I would of been in the rmpy source control but I went and made sure I purged all of it when I transferred the code to ARC.

Unless you have copy of it somewhere It's lost.
Title: Re: Reading Ruby's Marshal Format
Post by: Blizzard on February 16, 2014, 04:41:13 pm
Nope, it's lost then.
Title: Re: Reading Ruby's Marshal Format
Post by: Ryex on February 16, 2014, 06:33:11 pm
NEVER, say lost. because Hard Drives never forget. even if you delete.

Code: python

from RPG import *
from struct import pack, unpack


#============================================================================================
# RubyMarshal
#--------------------------------------------------------------------------------------------
# This class is able to read and write Ruby Marshal format.
#============================================================================================

class RubyMarshal:
   
    MARSHAL_MAJOR   = 4
    MARSHAL_MINOR   = 8
   
    TYPE_NIL        = '0'
    TYPE_TRUE       = 'T'
    TYPE_FALSE      = 'F'
    TYPE_FIXNUM     = 'i'
   
    TYPE_EXTENDED   = 'e' # not implemented
    TYPE_UCLASS     = 'C' # not implemented
    TYPE_OBJECT     = 'o'
    TYPE_DATA       = 'd' # not implemented
    TYPE_USERDEF    = 'u'
    TYPE_USRMARSHAL = 'U' # not implemented
    TYPE_FLOAT      = 'f' # not implemented
    TYPE_BIGNUM     = 'l'
    TYPE_STRING     = '"'
    TYPE_REGEXP     = '/' # not implemented
    TYPE_ARRAY      = '['
    TYPE_HASH       = '{'
    TYPE_HASH_DEF   = '}' # not implemented
    TYPE_STRUCT     = 'S' # not implemented
    TYPE_MODULE_OLD = 'M' # not implemented
    TYPE_CLASS      = 'c' # not implemented
    TYPE_MODULE     = 'm' # not implemented
   
    TYPE_SYMBOL     = ':'
    TYPE_SYMLINK    = ';'
   
    TYPE_IVAR       = 'I' # not implemented
    TYPE_LINK       = '@' # not implemented

    __Version = "\x04\x08"
    __io = None
    __symbols = []
   
    @staticmethod
    def generate(io):
        pass
   
    @staticmethod
    def dump(object, io):
        pass
   
    @staticmethod
    def load(io):
        RubyMarshal.__io = io
        try:
            major = RubyMarshal.__r_byte()
            minor = RubyMarshal.__r_byte()
            if (major != RubyMarshal.MARSHAL_MAJOR or minor != RubyMarshal.MARSHAL_MINOR):
                raise "incompatible marshal file format (can't be read)\n\
                \tformat version %d.%d required; %d.%d given" %\
                (RubyMarshal.MARSHAL_MAJOR, RubyMarshal.MARSHAL_MINOR, major, minor)
            obj = RubyMarshal.__r_object()
        except:
            raise
        finally:
            RubyMarshal.__io = None
            RubyMarshal.__symbols = []
        return obj
   
    @staticmethod
    def __r_object():
        objectType = chr(RubyMarshal.__r_byte())
        print "type: " + str(objectType)
       
        if objectType == RubyMarshal.TYPE_LINK:
            index = RubyMarshal.__r_long()
            try:
                return RubyMarshal.__symbols[index]
            except:
                raise "dump format error (unlinked %d of %d at 0x%x)" %\
                      (index, len(RubyMarshal.__symbols), RubyMarshal.__io.tell())
            pass
       
        if objectType == RubyMarshal.TYPE_NIL:
            return None
       
        if objectType == RubyMarshal.TYPE_TRUE:
            return True
       
        if objectType == RubyMarshal.TYPE_FALSE:
            return False
       
        if objectType == RubyMarshal.TYPE_FIXNUM:
            return RubyMarshal.__r_long()
       
        if objectType == RubyMarshal.TYPE_BIGNUM:
            sign = (RubyMarshal.__r_byte() == '+')
            data = RubyMarshal.__r_bytes()
            result = 0
            while length > 0:
                shift = 0
                for i in xrange(4):
                    value |= data[i] << shift
                    shift += 8
                length -= 1
            if not sign:
                result = -result
            RubyMarshal.__r_entry(result)
            return result
       
        if objectType == RubyMarshal.TYPE_STRING:
            result = RubyMarshal.__r_bytes()
            RubyMarshal.__r_entry(result)
            return result
       
        if objectType == RubyMarshal.TYPE_ARRAY:
            result = RubyMarshal.__r_array()
            RubyMarshal.__r_entry(result)
            return result
       
        if objectType == RubyMarshal.TYPE_HASH:
            result = RubyMarshal.__r_hash()
            RubyMarshal.__r_entry(result)
            return result
       
        if objectType == RubyMarshal.TYPE_USERDEF:
            #try:
            result = RubyMarshal.__r_unique()
            result._load(RubyMarshal.__io)
            #RubyMarshal.__r_entry(result)
            return result
            #except:
            #    raise "class %s needs to have method '_load'" % klass
            #pass
       
        if objectType == RubyMarshal.TYPE_OBJECT:
            print "__symbols: " + str(RubyMarshal.__symbols)
            result = RubyMarshal.__r_unique()
            print "result: " + str(result)
            length = RubyMarshal.__r_long()
            attributes = {}
            while (length > 0):
                print "get key"
                key = RubyMarshal.__r_symbol()
                print "get value"
                value = RubyMarshal.__r_object()
                print "key, value: " + str(key) + " " + str(value)
                attributes[key] = value
                length -= 1
            print str(attributes)
            for symbol in attributes.keys():
                setattr(result, symbol.replace("@", ""), attributes[symbol])
            RubyMarshal.__r_entry(result)
            return result
       
        if objectType == RubyMarshal.TYPE_SYMBOL:
            result = RubyMarshal.__r_symreal()
            RubyMarshal.__r_entry(result)
            return result
       
        if objectType == RubyMarshal.TYPE_SYMLINK:
            result = RubyMarshal.__r_symlink()
            RubyMarshal.__r_entry(result)
            return result
       
        raise "dump format error(0x%x at 0x%x)" % (ord(objectType), RubyMarshal.__io.tell())
   
    @staticmethod
    def __r_byte():
        return ord(RubyMarshal.__io.read(1))
   
    @staticmethod
    def __r_bytes():
        return RubyMarshal.__r_bytes0(RubyMarshal.__r_long())
   
    @staticmethod
    def __r_bytes0(length):
        if (length == 0):
            return ''
        return RubyMarshal.__io.read(length)
   
    @staticmethod
    def __r_array():
        length = RubyMarshal.__r_long()
        result = []
        while (length > 0):
            result.append(RubyMarshal.__r_object())
            length -= 1
        return result
   
    @staticmethod
    def __r_hash():
        length = RubyMarshal.__r_long()
        result = {}
        while (length > 0):
            key = RubyMarshal.__r_object()
            value = RubyMarshal.__r_object()
            try:
                result[key] = value
            except TypeError:
                result[tuple(key)] = value
            length -= 1
        return result
   
    @staticmethod
    def __r_long():
        c = RubyMarshal.__r_byte()
        if c > 127:
            c -= 256
        if (c == 0):
            return 0
        if (c > 0):
            if (4 < c and c < 128):
                return (c - 5)
            result = 0
            for i in xrange(c):
                result |= RubyMarshal.__r_byte() << (8 * i)
            return result
        if (-129 < c and c < -4):
            return (c + 5)
        c = -c
        result = -1
        for i in xrange(c):
            result &= ~(0xFF << (8 * i))
            result |= RubyMarshal.__r_byte() << (8 * i)
        return result
   
    @staticmethod
    def __r_symreal():
        symbol = RubyMarshal.__r_bytes()
        print "symreal: " + str(symbol)
        RubyMarshal.__r_entry(symbol)
        return symbol
   
    @staticmethod
    def __r_symlink():
        index = RubyMarshal.__r_long()
        if index >= len(RubyMarshal.__symbols):
            raise "bad symbol (0x%x)" % RubyMarshal.__io.tell()
        print "symlink: " + str(index) + " " + str(RubyMarshal.__symbols[index])
        return RubyMarshal.__symbols[index]
   
    @staticmethod
    def __r_unique():
        return RubyMarshal.__id2name(RubyMarshal.__r_symbol())
   
    @staticmethod
    def __r_symbol():
        if chr(RubyMarshal.__r_byte()) == RubyMarshal.TYPE_SYMLINK:
            return RubyMarshal.__r_symlink()
        return RubyMarshal.__r_symreal()
   
    @staticmethod
    def __r_entry(value):
        RubyMarshal.__symbols.append(value)
        return value
   
    @staticmethod
    def __id2name(name):
        print "idtoname: " + str(name)
        return eval(name.replace("::", ".") + "()")

   
       


Title: Re: Reading Ruby's Marshal Format
Post by: G_G on February 16, 2014, 06:52:26 pm
Mmmmm. This will actually help quite a bit. Thanks for the help guys!
Title: Re: Reading Ruby's Marshal Format
Post by: Ryex on February 16, 2014, 06:54:28 pm
just keep in mind that this was broken when we abandoned it. it work for some files but if it had a table it it it crashed.

EDIT:
also, holly crap, when I rememberd I had kept RMPY on my external drive before I did I format of it, I went and ran recurva on it to see what I could find. I quicly found our old ruby marsh in pyhton back form december 2010. but there was a lot of shit hidden on that drive that I've been missing. like my old FL song projects and my work on PNO. as in old PNO.
Title: Re: Reading Ruby's Marshal Format
Post by: G_G on February 16, 2014, 08:58:38 pm
lol nice. And I'm sure the table can be read somehow. Don't we just have to unpack the data and then read it as if it were a ruby object? Here's vgvgf's Table rewrite that I'd always use when loading RMXP data with Ruby or IronRuby.

class Table
  def initialize(x, y = 1, z = 1)
     @xsize, @ysize, @zsize = x, y, z
     @data = Array.new(x * y * z, 0)
  end
  def [](x, y = 0, z = 0)
     @data[x + y * @xsize + z * @xsize * @ysize]
  end
  def []=(*args)
     x = args[0]
     y = args.size > 2 ? args[1] :0
     z = args.size > 3 ? args[2] :0
     v = args.pop
     @data[x + y * @xsize + z * @xsize * @ysize] = v
  end
  def _dump(d = 0)
     s = [3].pack('L')
     s += [@xsize].pack('L') + [@ysize].pack('L') + [@zsize].pack('L')
     s += [@xsize * @ysize * @zsize].pack('L')
     for z in 0...@zsize
        for y in 0...@ysize
           for x in 0...@xsize
              s += [@data[x + y * @xsize + z * @xsize * @ysize]].pack('S')
           end
        end
     end
     s
  end
  def self._load(s)
     size = s[0, 4].unpack('L')[0]
     nx = s[4, 4].unpack('L')[0]
     ny = s[8, 4].unpack('L')[0]
     nz = s[12, 4].unpack('L')[0]
     data = []
     pointer = 20
     loop do
        data.push(*s[pointer, 2].unpack('S'))
        pointer += 2
        break if pointer > s.size - 1
     end
     t = Table.new(nx, ny, nz)
     n = 0
     for z in 0...nz
        for y in 0...ny
           for x in 0...nx
              t[x, y, z] = data[n]
              n += 1
           end
        end
     end
     t
  end
  attr_reader(:xsize, :ysize, :zsize, :data)
end


And as far as I'm concerned, it still read the data just fine.
Title: Re: Reading Ruby's Marshal Format
Post by: Ryex on February 16, 2014, 09:29:44 pm
ya I think the problem was just in our understanding of how tables were stored.
Title: Re: Reading Ruby's Marshal Format
Post by: Blizzard on February 17, 2014, 02:02:02 am
Yeah, we did a mistake somewhere in the beginning of the data, in the first few bytes or so. If I remember right, Table has first the number of dimensions stored, the the respective x, y and z sizes and finally the whole size as x*y*z. And I think these 4-bytes weren't even marshalled but directly dumped.
Title: Re: Reading Ruby's Marshal Format
Post by: Ryex on February 17, 2014, 02:08:45 am
Ya, I'm pretty sure our error was in the table rewrite we made.


class Table
 attr_accessor :data
 def initialize(x, y = 1, z = 1)
    @xsize, @ysize, @zsize = x, y, z
    @data = Array.new(x * y * z, 0)
 end
 def [](x, y = 0, z = 0)
    @data[x + y * @xsize + z * @xsize * @ysize]
 end
 def []=(*args)
    x = args[0]
    y = args.size > 2 ? args[1] :0
    z = args.size > 3 ? args[2] :0
    v = args.pop
    @data[x + y * @xsize + z * @xsize * @ysize] = v
 end
 def _dump(d = 0)
    s = [3].pack('L')
    s += [@xsize].pack('L') + [@ysize].pack('L') + [@zsize].pack('L')
    s += [@xsize * @ysize * @zsize].pack('L')
    for z in 0...@zsize
       for y in 0...@ysize
          for x in 0...@xsize
             s += [@data[x + y * @xsize + z * @xsize * @ysize]].pack('S')
          end
       end
    end
    s
 end
 def self._load(s)
    size = s[0, 4].unpack('L')[0]
    nx = s[4, 4].unpack('L')[0]
    ny = s[8, 4].unpack('L')[0]
    nz = s[12, 4].unpack('L')[0]
    data = []
    pointer = 20
    loop do
       data.push(*s[pointer, 2].unpack('S'))
       pointer += 2
       break if pointer > s.size - 1
    end
    t = Table.new(nx, ny, nz)
    n = 0
    for z in 0...nz
       for y in 0...ny
          for x in 0...nx
             t[x, y, z] = data[n]
             n += 1
          end
       end
    end
    t
 end
 attr_reader(:xsize, :ysize, :zsize, :data)
end
class Color
 def initialize(r, g, b, a = 255)
    @red = r
    @green = g
    @blue = b
    @alpha = a
 end
 def set(r, g, b, a = 255)
    @red = r
    @green = g
    @blue = b
    @alpha = a
 end
 def color
    Color.new(@red, @green, @blue, @alpha)
 end
 def _dump(d = 0)
    [@red, @green, @blue, @alpha].pack('d4')
 end
 def self._load(s)
    Color.new(*s.unpack('d4'))
 end
 attr_accessor(:red, :green, :blue, :alpha)
end
class Tone
 def initialize(r, g, b, a = 0)
    @red = r
    @green = g
    @blue = b
    @gray = a
 end
 def set(r, g, b, a = 0)
    @red = r
    @green = g
    @blue = b
    @gray = a
 end
 def color
    Color.new(@red, @green, @blue, @gray)
 end
 def _dump(d = 0)
    [@red, @green, @blue, @gray].pack('d4')
 end
 def self._load(s)
    Tone.new(*s.unpack('d4'))
 end
 attr_accessor(:red, :green, :blue, :gray)
end


all I remember was that reading some tables but not all our stream pointer in the file would get off soon after reading the table dump.


Here's the ruby implementation of the RPG module if you need it


module RPG
  class Actor
    def initialize
      @id = 0
      @name = ""
      @class_id = 1
      @initial_level = 1
      @final_level = 99
      @exp_basis = 30
      @exp_inflation = 30
      @character_name = ""
      @character_hue = 0
      @battler_name = ""
      @battler_hue = 0
      @parameters = Table.new(6,100)
      for i in 1..99
        @parameters[0,i] = 500+i*50
        @parameters[1,i] = 500+i*50
        @parameters[2,i] = 50+i*5
        @parameters[3,i] = 50+i*5
        @parameters[4,i] = 50+i*5
        @parameters[5,i] = 50+i*5
      end
      @weapon_id = 0
      @armor1_id = 0
      @armor2_id = 0
      @armor3_id = 0
      @armor4_id = 0
      @weapon_fix = false
      @armor1_fix = false
      @armor2_fix = false
      @armor3_fix = false
      @armor4_fix = false
    end
    attr_accessor :id
    attr_accessor :name
    attr_accessor :class_id
    attr_accessor :initial_level
    attr_accessor :final_level
    attr_accessor :exp_basis
    attr_accessor :exp_inflation
    attr_accessor :character_name
    attr_accessor :character_hue
    attr_accessor :battler_name
    attr_accessor :battler_hue
    attr_accessor :parameters
    attr_accessor :weapon_id
    attr_accessor :armor1_id
    attr_accessor :armor2_id
    attr_accessor :armor3_id
    attr_accessor :armor4_id
    attr_accessor :weapon_fix
    attr_accessor :armor1_fix
    attr_accessor :armor2_fix
    attr_accessor :armor3_fix
    attr_accessor :armor4_fix
  end
end

module RPG
  class Class
    def initialize
      @id = 0
      @name = ""
      @position = 0
      @weapon_set = []
      @armor_set = []
      @element_ranks = Table.new(1)
      @state_ranks = Table.new(1)
      @learnings = []
    end
    attr_accessor :id
    attr_accessor :name
    attr_accessor :position
    attr_accessor :weapon_set
    attr_accessor :armor_set
    attr_accessor :element_ranks
    attr_accessor :state_ranks
    attr_accessor :learnings
  end
end

module RPG
  class Class
    class Learning
      def initialize
        @level = 1
        @skill_id = 1
      end
      attr_accessor :level
      attr_accessor :skill_id
    end
  end
end

module RPG
  class Skill
    def initialize
      @id = 0
      @name = ""
      @icon_name = ""
      @description = ""
      @scope = 0
      @occasion = 1
      @animation1_id = 0
      @animation2_id = 0
      @menu_se = RPG::AudioFile.new("", 80)
      @common_event_id = 0
      @sp_cost = 0
      @power = 0
      @atk_f = 0
      @eva_f = 0
      @str_f = 0
      @dex_f = 0
      @agi_f = 0
      @int_f = 100
      @hit = 100
      @pdef_f = 0
      @mdef_f = 100
      @variance = 15
      @element_set = []
      @plus_state_set = []
      @minus_state_set = []
    end
    attr_accessor :id
    attr_accessor :name
    attr_accessor :icon_name
    attr_accessor :description
    attr_accessor :scope
    attr_accessor :occasion
    attr_accessor :animation1_id
    attr_accessor :animation2_id
    attr_accessor :menu_se
    attr_accessor :common_event_id
    attr_accessor :sp_cost
    attr_accessor :power
    attr_accessor :atk_f
    attr_accessor :eva_f
    attr_accessor :str_f
    attr_accessor :dex_f
    attr_accessor :agi_f
    attr_accessor :int_f
    attr_accessor :hit
    attr_accessor :pdef_f
    attr_accessor :mdef_f
    attr_accessor :variance
    attr_accessor :element_set
    attr_accessor :plus_state_set
    attr_accessor :minus_state_set
  end
end

module RPG
  class Item
    def initialize
      @id = 0
      @name = ""
      @icon_name = ""
      @description = ""
      @scope = 0
      @occasion = 0
      @animation1_id = 0
      @animation2_id = 0
      @menu_se = RPG::AudioFile.new("", 80)
      @common_event_id = 0
      @price = 0
      @consumable = true
      @parameter_type = 0
      @parameter_points = 0
      @recover_hp_rate = 0
      @recover_hp = 0
      @recover_sp_rate = 0
      @recover_sp = 0
      @hit = 100
      @pdef_f = 0
      @mdef_f = 0
      @variance = 0
      @element_set = []
      @plus_state_set = []
      @minus_state_set = []
    end
    attr_accessor :id
    attr_accessor :name
    attr_accessor :icon_name
    attr_accessor :description
    attr_accessor :scope
    attr_accessor :occasion
    attr_accessor :animation1_id
    attr_accessor :animation2_id
    attr_accessor :menu_se
    attr_accessor :common_event_id
    attr_accessor :price
    attr_accessor :consumable
    attr_accessor :parameter_type
    attr_accessor :parameter_points
    attr_accessor :recover_hp_rate
    attr_accessor :recover_hp
    attr_accessor :recover_sp_rate
    attr_accessor :recover_sp
    attr_accessor :hit
    attr_accessor :pdef_f
    attr_accessor :mdef_f
    attr_accessor :variance
    attr_accessor :element_set
    attr_accessor :plus_state_set
    attr_accessor :minus_state_set
  end
end

module RPG
  class EventCommand
    def initialize(code = 0, indent = 0, parameters = [])
      @code = code
      @indent = indent
      @parameters = parameters
    end
    attr_accessor :code
    attr_accessor :indent
    attr_accessor :parameters
  end
end

module RPG
  class Weapon
    def initialize
      @id = 0
      @name = ""
      @icon_name = ""
      @description = ""
      @animation1_id = 0
      @animation2_id = 0
      @price = 0
      @atk = 0
      @pdef = 0
      @mdef = 0
      @str_plus = 0
      @dex_plus = 0
      @agi_plus = 0
      @int_plus = 0
      @element_set = []
      @plus_state_set = []
      @minus_state_set = []
    end
    attr_accessor :id
    attr_accessor :name
    attr_accessor :icon_name
    attr_accessor :description
    attr_accessor :animation1_id
    attr_accessor :animation2_id
    attr_accessor :price
    attr_accessor :atk
    attr_accessor :pdef
    attr_accessor :mdef
    attr_accessor :str_plus
    attr_accessor :dex_plus
    attr_accessor :agi_plus
    attr_accessor :int_plus
    attr_accessor :element_set
    attr_accessor :plus_state_set
    attr_accessor :minus_state_set
  end
end

module RPG
  class Armor
    def initialize
      @id = 0
      @name = ""
      @icon_name = ""
      @description = ""
      @kind = 0
      @auto_state_id = 0
      @price = 0
      @pdef = 0
      @mdef = 0
      @eva = 0
      @str_plus = 0
      @dex_plus = 0
      @agi_plus = 0
      @int_plus = 0
      @guard_element_set = []
      @guard_state_set = []
    end
    attr_accessor :id
    attr_accessor :name
    attr_accessor :icon_name
    attr_accessor :description
    attr_accessor :kind
    attr_accessor :auto_state_id
    attr_accessor :price
    attr_accessor :pdef
    attr_accessor :mdef
    attr_accessor :eva
    attr_accessor :str_plus
    attr_accessor :dex_plus
    attr_accessor :agi_plus
    attr_accessor :int_plus
    attr_accessor :guard_element_set
    attr_accessor :guard_state_set
  end
end

module RPG
  class Enemy
    def initialize
      @id = 0
      @name = ""
      @battler_name = ""
      @battler_hue = 0
      @maxhp = 500
      @maxsp = 500
      @str = 50
      @dex = 50
      @agi = 50
      @int = 50
      @atk = 100
      @pdef = 100
      @mdef = 100
      @eva = 0
      @animation1_id = 0
      @animation2_id = 0
      @element_ranks = Table.new(1)
      @state_ranks = Table.new(1)
      @actions = [RPG::Enemy::Action.new]
      @exp = 0
      @gold = 0
      @item_id = 0
      @weapon_id = 0
      @armor_id = 0
      @treasure_prob = 100
    end
    attr_accessor :id
    attr_accessor :name
    attr_accessor :battler_name
    attr_accessor :battler_hue
    attr_accessor :maxhp
    attr_accessor :maxsp
    attr_accessor :str
    attr_accessor :dex
    attr_accessor :agi
    attr_accessor :int
    attr_accessor :atk
    attr_accessor :pdef
    attr_accessor :mdef
    attr_accessor :eva
    attr_accessor :animation1_id
    attr_accessor :animation2_id
    attr_accessor :element_ranks
    attr_accessor :state_ranks
    attr_accessor :actions
    attr_accessor :exp
    attr_accessor :gold
    attr_accessor :item_id
    attr_accessor :weapon_id
    attr_accessor :armor_id
    attr_accessor :treasure_prob
  end
end

module RPG
  class Enemy
    class Action
      def initialize
        @kind = 0
        @basic = 0
        @skill_id = 1
        @condition_turn_a = 0
        @condition_turn_b = 1
        @condition_hp = 100
        @condition_level = 1
        @condition_switch_id = 0
        @rating = 5
      end
      attr_accessor :kind
      attr_accessor :basic
      attr_accessor :skill_id
      attr_accessor :condition_turn_a
      attr_accessor :condition_turn_b
      attr_accessor :condition_hp
      attr_accessor :condition_level
      attr_accessor :condition_switch_id
      attr_accessor :rating
    end
  end
end

module RPG
  class Troop
    def initialize
      @id = 0
      @name = ""
      @members = []
      @pages = [RPG::Troop::Page.new]
    end
    attr_accessor :id
    attr_accessor :name
    attr_accessor :members
    attr_accessor :pages
  end
end

module RPG
  class Troop
    class Member
      def initialize
        @enemy_id = 1
        @x = 0
        @y = 0
        @hidden = false
        @immortal = false
      end
      attr_accessor :enemy_id
      attr_accessor :x
      attr_accessor :y
      attr_accessor :hidden
      attr_accessor :immortal
    end
  end
end

module RPG
  class Troop
    class Page
      def initialize
        @condition = RPG::Troop::Page::Condition.new
        @span = 0
        @list = [RPG::EventCommand.new]
      end
      attr_accessor :condition
      attr_accessor :span
      attr_accessor :list
    end
  end
end

module RPG
  class Troop
    class Page
      class Condition
        def initialize
          @turn_valid = false
          @enemy_valid = false
          @actor_valid = false
          @switch_valid = false
          @turn_a = 0
          @turn_b = 0
          @enemy_index = 0
          @enemy_hp = 50
          @actor_id = 1
          @actor_hp = 50
          @switch_id = 1
        end
        attr_accessor :turn_valid
        attr_accessor :enemy_valid
        attr_accessor :actor_valid
        attr_accessor :switch_valid
        attr_accessor :turn_a
        attr_accessor :turn_b
        attr_accessor :enemy_index
        attr_accessor :enemy_hp
        attr_accessor :actor_id
        attr_accessor :actor_hp
        attr_accessor :switch_id
      end
    end
  end
end

module RPG
  class State
    def initialize
      @id = 0
      @name = ""
      @animation_id = 0
      @restriction = 0
      @nonresistance = false
      @zero_hp = false
      @cant_get_exp = false
      @cant_evade = false
      @slip_damage = false
      @rating = 5
      @hit_rate = 100
      @maxhp_rate = 100
      @maxsp_rate = 100
      @str_rate = 100
      @dex_rate = 100
      @agi_rate = 100
      @int_rate = 100
      @atk_rate = 100
      @pdef_rate = 100
      @mdef_rate = 100
      @eva = 0
      @battle_only = true
      @hold_turn = 0
      @auto_release_prob = 0
      @shock_release_prob = 0
      @guard_element_set = []
      @plus_state_set = []
      @minus_state_set = []
    end
    attr_accessor :id
    attr_accessor :name
    attr_accessor :animation_id
    attr_accessor :restriction
    attr_accessor :nonresistance
    attr_accessor :zero_hp
    attr_accessor :cant_get_exp
    attr_accessor :cant_evade
    attr_accessor :slip_damage
    attr_accessor :rating
    attr_accessor :hit_rate
    attr_accessor :maxhp_rate
    attr_accessor :maxsp_rate
    attr_accessor :str_rate
    attr_accessor :dex_rate
    attr_accessor :agi_rate
    attr_accessor :int_rate
    attr_accessor :atk_rate
    attr_accessor :pdef_rate
    attr_accessor :mdef_rate
    attr_accessor :eva
    attr_accessor :battle_only
    attr_accessor :hold_turn
    attr_accessor :auto_release_prob
    attr_accessor :shock_release_prob
    attr_accessor :guard_element_set
    attr_accessor :plus_state_set
    attr_accessor :minus_state_set
  end
end

module RPG
  class Animation
    def initialize
      @id = 0
      @name = ""
      @animation_name = ""
      @animation_hue = 0
      @position = 1
      @frame_max = 1
      @frames = [RPG::Animation::Frame.new]
      @timings = []
    end
    attr_accessor :id
    attr_accessor :name
    attr_accessor :animation_name
    attr_accessor :animation_hue
    attr_accessor :position
    attr_accessor :frame_max
    attr_accessor :frames
    attr_accessor :timings
  end
end

module RPG
  class Animation
    class Frame
      def initialize
        @cell_max = 0
        @cell_data = Table.new(0, 0)
      end
      attr_accessor :cell_max
      attr_accessor :cell_data
    end
  end
end

module RPG
  class Animation
    class Timing
      def initialize
        @frame = 0
        @se = RPG::AudioFile.new("", 80)
        @flash_scope = 0
        @flash_color = Color.new(255,255,255,255)
        @flash_duration = 5
        @condition = 0
      end
      attr_accessor :frame
      attr_accessor :se
      attr_accessor :flash_scope
      attr_accessor :flash_color
      attr_accessor :flash_duration
      attr_accessor :condition
    end
  end
end

module RPG
  class Tileset
    def initialize
      @id = 0
      @name = ""
      @tileset_name = ""
      @autotile_names = [""]*7
      @panorama_name = ""
      @panorama_hue = 0
      @fog_name = ""
      @fog_hue = 0
      @fog_opacity = 64
      @fog_blend_type = 0
      @fog_zoom = 200
      @fog_sx = 0
      @fog_sy = 0
      @battleback_name = ""
      @passages = Table.new(384)
      @priorities = Table.new(384)
      @priorities[0] = 5
      @terrain_tags = Table.new(384)
    end
    attr_accessor :id
    attr_accessor :name
    attr_accessor :tileset_name
    attr_accessor :autotile_names
    attr_accessor :panorama_name
    attr_accessor :panorama_hue
    attr_accessor :fog_name
    attr_accessor :fog_hue
    attr_accessor :fog_opacity
    attr_accessor :fog_blend_type
    attr_accessor :fog_zoom
    attr_accessor :fog_sx
    attr_accessor :fog_sy
    attr_accessor :battleback_name
    attr_accessor :passages
    attr_accessor :priorities
    attr_accessor :terrain_tags
  end
end

module RPG
  class CommonEvent
    def initialize
      @id = 0
      @name = ""
      @trigger = 0
      @switch_id = 1
      @list = [RPG::EventCommand.new]
    end
    attr_accessor :id
    attr_accessor :name
    attr_accessor :trigger
    attr_accessor :switch_id
    attr_accessor :list
  end
end

module RPG
  class System
    def initialize
      @magic_number = 0
      @party_members = [1]
      @elements = [nil, ""]
      @switches = [nil, ""]
      @variables = [nil, ""]
      @windowskin_name = ""
      @title_name = ""
      @gameover_name = ""
      @battle_transition = ""
      @title_bgm = RPG::AudioFile.new
      @battle_bgm = RPG::AudioFile.new
      @battle_end_me = RPG::AudioFile.new
      @gameover_me = RPG::AudioFile.new
      @cursor_se = RPG::AudioFile.new("", 80)
      @decision_se = RPG::AudioFile.new("", 80)
      @cancel_se = RPG::AudioFile.new("", 80)
      @buzzer_se = RPG::AudioFile.new("", 80)
      @equip_se = RPG::AudioFile.new("", 80)
      @shop_se = RPG::AudioFile.new("", 80)
      @save_se = RPG::AudioFile.new("", 80)
      @load_se = RPG::AudioFile.new("", 80)
      @battle_start_se = RPG::AudioFile.new("", 80)
      @escape_se = RPG::AudioFile.new("", 80)
      @actor_collapse_se = RPG::AudioFile.new("", 80)
      @enemy_collapse_se = RPG::AudioFile.new("", 80)
      @words = RPG::System::Words.new
      @test_battlers = []
      @test_troop_id = 1
      @start_map_id = 1
      @start_x = 0
      @start_y = 0
      @battleback_name = ""
      @battler_name = ""
      @battler_hue = 0
      @edit_map_id = 1
    end
    attr_accessor :magic_number
    attr_accessor :party_members
    attr_accessor :elements
    attr_accessor :switches
    attr_accessor :variables
    attr_accessor :windowskin_name
    attr_accessor :title_name
    attr_accessor :gameover_name
    attr_accessor :battle_transition
    attr_accessor :title_bgm
    attr_accessor :battle_bgm
    attr_accessor :battle_end_me
    attr_accessor :gameover_me
    attr_accessor :cursor_se
    attr_accessor :decision_se
    attr_accessor :cancel_se
    attr_accessor :buzzer_se
    attr_accessor :equip_se
    attr_accessor :shop_se
    attr_accessor :save_se
    attr_accessor :load_se
    attr_accessor :battle_start_se
    attr_accessor :escape_se
    attr_accessor :actor_collapse_se
    attr_accessor :enemy_collapse_se
    attr_accessor :words
    attr_accessor :test_battlers
    attr_accessor :test_troop_id
    attr_accessor :start_map_id
    attr_accessor :start_x
    attr_accessor :start_y
    attr_accessor :battleback_name
    attr_accessor :battler_name
    attr_accessor :battler_hue
    attr_accessor :edit_map_id
  end
end

module RPG
  class System
    class Words
      def initialize
        @gold = ""
        @hp = ""
        @sp = ""
        @str = ""
        @dex = ""
        @agi = ""
        @int = ""
        @atk = ""
        @pdef = ""
        @mdef = ""
        @weapon = ""
        @armor1 = ""
        @armor2 = ""
        @armor3 = ""
        @armor4 = ""
        @attack = ""
        @skill = ""
        @guard = ""
        @item = ""
        @equip = ""
      end
      attr_accessor :gold
      attr_accessor :hp
      attr_accessor :sp
      attr_accessor :str
      attr_accessor :dex
      attr_accessor :agi
      attr_accessor :int
      attr_accessor :atk
      attr_accessor :pdef
      attr_accessor :mdef
      attr_accessor :weapon
      attr_accessor :armor1
      attr_accessor :armor2
      attr_accessor :armor3
      attr_accessor :armor4
      attr_accessor :attack
      attr_accessor :skill
      attr_accessor :guard
      attr_accessor :item
      attr_accessor :equip
    end
  end
end

module RPG
  class System
    class TestBattler
      def initialize
        @actor_id = 1
        @level = 1
        @weapon_id = 0
        @armor1_id = 0
        @armor2_id = 0
        @armor3_id = 0
        @armor4_id = 0
      end
      attr_accessor :actor_id
      attr_accessor :level
      attr_accessor :weapon_id
      attr_accessor :armor1_id
      attr_accessor :armor2_id
      attr_accessor :armor3_id
      attr_accessor :armor4_id
    end
  end
end

module RPG
  class AudioFile
    def initialize(name = "", volume = 100, pitch = 100)
      @name = name
      @volume = volume
      @pitch = pitch
    end
    attr_accessor :name
    attr_accessor :volume
    attr_accessor :pitch
  end
end

module RPG
  class Map
    def initialize(width, height)
      @tileset_id = 1
      @width = width
      @height = height
      @autoplay_bgm = false
      @bgm = RPG::AudioFile.new
      @autoplay_bgs = false
      @bgs = RPG::AudioFile.new("", 80)
      @encounter_list = []
      @encounter_step = 30
      @data = Table.new(width, height, 3)
      @events = {}
    end
    attr_accessor :tileset_id
    attr_accessor :width
    attr_accessor :height
    attr_accessor :autoplay_bgm
    attr_accessor :bgm
    attr_accessor :autoplay_bgs
    attr_accessor :bgs
    attr_accessor :encounter_list
    attr_accessor :encounter_step
    attr_accessor :data
    attr_accessor :events
  end
end

module RPG
  class MapInfo
    def initialize
      @name = ""
      @parent_id = 0
      @order = 0
      @expanded = false
      @scroll_x = 0
      @scroll_y = 0
    end
    attr_accessor :name
    attr_accessor :parent_id
    attr_accessor :order
    attr_accessor :expanded
    attr_accessor :scroll_x
    attr_accessor :scroll_y
  end
end

module RPG
  class Event
    def initialize(x, y)
      @id = 0
      @name = ""
      @x = x
      @y = y
      @pages = [RPG::Event::Page.new]
    end
    attr_accessor :id
    attr_accessor :name
    attr_accessor :x
    attr_accessor :y
    attr_accessor :pages
  end
end

module RPG
  class Event
    class Page
      def initialize
        @condition = RPG::Event::Page::Condition.new
        @graphic = RPG::Event::Page::Graphic.new
        @move_type = 0
        @move_speed = 3
        @move_frequency = 3
        @move_route = RPG::MoveRoute.new
        @walk_anime = true
        @step_anime = false
        @direction_fix = false
        @through = false
        @always_on_top = false
        @trigger = 0
        @list = [RPG::EventCommand.new]
      end
      attr_accessor :condition
      attr_accessor :graphic
      attr_accessor :move_type
      attr_accessor :move_speed
      attr_accessor :move_frequency
      attr_accessor :move_route
      attr_accessor :walk_anime
      attr_accessor :step_anime
      attr_accessor :direction_fix
      attr_accessor :through
      attr_accessor :always_on_top
      attr_accessor :trigger
      attr_accessor :list
    end
  end
end

module RPG
  class Event
    class Page
      class Condition
        def initialize
          @switch1_valid = false
          @switch2_valid = false
          @variable_valid = false
          @self_switch_valid = false
          @switch1_id = 1
          @switch2_id = 1
          @variable_id = 1
          @variable_value = 0
          @self_switch_ch = "A"
        end
        attr_accessor :switch1_valid
        attr_accessor :switch2_valid
        attr_accessor :variable_valid
        attr_accessor :self_switch_valid
        attr_accessor :switch1_id
        attr_accessor :switch2_id
        attr_accessor :variable_id
        attr_accessor :variable_value
        attr_accessor :self_switch_ch
      end
    end
  end
end

module RPG
  class Event
    class Page
      class Graphic
        def initialize
          @tile_id = 0
          @character_name = ""
          @character_hue = 0
          @direction = 2
          @pattern = 0
          @opacity = 255
          @blend_type = 0
        end
        attr_accessor :tile_id
        attr_accessor :character_name
        attr_accessor :character_hue
        attr_accessor :direction
        attr_accessor :pattern
        attr_accessor :opacity
        attr_accessor :blend_type
      end
    end
  end
end

module RPG
  class EventCommand
    def initialize(code = 0, indent = 0, parameters = [])
      @code = code
      @indent = indent
      @parameters = parameters
    end
    attr_accessor :code
    attr_accessor :indent
    attr_accessor :parameters
  end
end

module RPG
  class MoveRoute
    def initialize
      @repeat = true
      @skippable = false
      @list = [RPG::MoveCommand.new]
    end
    attr_accessor :repeat
    attr_accessor :skippable
    attr_accessor :list
  end
end

module RPG
  class MoveCommand
    def initialize(code = 0, parameters = [nil] )
      @code = code
      @parameters = parameters
    end
    attr_accessor :code
    attr_accessor :parameters
  end
end