.arc Data Format

[Blizzard]

Should we publicly release the specs of the format? I don't see much bad in that.

[Ryex]

I guess we could. there is not real reason to not release it. is there any particular reason to release it?

[Blizzard]

Mostly for reference. We have both ARC::Data::load and ARC::Data::save implemented. Maybe people would like to make some sort of encryption or something. Or they would like to load some specific data in a custom environment. This would allow them to implement their own reader/writer of the .arc format.

[Ryex]

Ya lets go ahead and release the specification.

[Blizzard]

I just realized that it would be REALLY good to include a header tag. I have updated all code (hopefully) to include the additional processing of the excepted ARCD header. I have also updated all .arc data files located in the SVN. I have tested only the C++ version (not the Python version and not the Ruby version) and everything works. Ryex, make sure that the Ruby and Python code work as well. Let me know when we will update the project converter and the builds, because we should do it at the same time to prevent problems for people who download the beta or the development builds.

EDIT: And I also noticed a bug. Empty arrays will always be created a new arrays which is wrong. If several objects reference one and the same array and it's empty, the data will be saved/loaded incorrectly. This can be easily fixed and I don't think it affects the data we are already using. In any case, I will fix that bug in C++, Ruby and Python. Again, Ryex, you will have to test this in Ruby and Python.

EDIT: There's even more wrong. e.g. Two arrays with the same contents will be considered the same array. I have to fix that. :/ I will have to compare object identity I guess. :/

EDIT: BTW, Ryex, do you think that we should check only for the corresponding _arc_dump and _arc_load methods or that we should check for both of them? Technically if you have _arc_dump defined, you don't really have to have _arc_load defined and vice versa, but logically one would expect that you have to have both defined if you want to dump and/or load data.

[Ryex]

In the ruby and python code I only check for the method existence when I need to use it I think that's fine

[Blizzard]

Yes, purely technically it is fine. But I was wondering if it was fine by design. I think we can leave it like this though. If somebody uses one method, they don't have to necessarily use the other as well.

EDIT: Shit, I broke it all. :/ I'll have to go fix all of this stuff. Writing never really worked in C++ actually (I never tested it) and the redesign leaves me with weird data. Now I'm not sure whether the data has been corrupted during conversion from RMXP to ARC or whether it is not being loaded properly in C++. :/ I'll take a look at this tomorrow, it's late.

EDIT: Lol, I didn't break it after all. The problem was in April, not in our libs. But the C++ version of dumping doesn't work after all.

Ryex, I have updated the Ruby code as well as the Python code for ARC Data. I have also changed the usage of the "is" operator in Python as it compares identity, not equality. This could have caused many unforeseen problems. I will commit everything soon. If can fix the broken C++ dump code, then I will commit it as well.

EDIT: Alright, I got the dumper to work, but for some reason it's not creating files that are like the original. I have to investigate this further. It's probably just some additional junk data that's being stored that has been rendered obsolete with the final definition of ARC Data v1.0.

EDIT: I will add sorting of instance variables names. This will make files more consistent and changes won't be caused just by two variables switching places.

EDIT: I found something in RMXP that is pretty fucked up. In Animations.rxdata there are Tables that have one of the sizes set to 0 (xsize in the particular example I found). :/ I have implemented in ARC that xsize, ysize and zsize are always at least 1. When we convert data from RMXP, we're not handling that case so the converted files can have xsize, ysize and/or zsize set to 0 without us noticing. Obviously this means no data will be saved at all. If the file is loaded in the engine, the sizes are corrected to at least 1 and when they are saved again, these changes are reflected in the new file.

You can look it up easily. This hex code basically designates the Table entry in Animations.rxdata: "75 3b 0c 19". The easiest way to find it is to search for "02 00 00 00 00 00 00 00 08 00 00 00 01 00 00 00 00 00 00 00" which basically means 2 dimensions, 0 xsize, 8 ysize, 1 zsize, 0 elements.

I will implement an identical behavior in ARC. If one of the sizes is specified as 0 or less than 0, it will be clamped to 0. Obviously all data access will return nil and technically the Table is useless. But if Enterbrain implemented it that way, so will we.