[RESOLVED] (massive knowledge included) Extracting data from an .rxdata...

[shdwlink1993]

Hey. I've been working on this script that takes the information from a .rxdata file, copys it, and decompresses it in another folder as a .txt file.

So, i've been having a few problems with it:

1) First up, is there any way to get an array to come out correctly, so instead of this: 123456791015, it looks like this: [1, 2, 3, 4, 5, 6, 7, 9, 10, 15] or something that people would read easier.

2) I've been getting the information as to what to extract from the Help File (which I don't think was really made for people who are starting out with the program), and I can get Animation, but I can't figure out how to get Animation::Frame. Also, where are the odd things like Weather?

3) How the crud do you get the scripts out?

4) Is there a way to reverse the process, and import the information from a text file?

Thanks to anyone who can answer any of my questions.

[Blizzard]

1) ARRAY.inspect, the inspect method will write it out exactly like that.

2) Find RPG::Weather for the built-in weather script. If I'm not wrong, an animation has the array "frames" (animation.frames) which is an array of Animation::Frame instances.

3) They are compressed with a compression method, most probably zip. It's a nasty job to decode that, but it can be done. If I'm not wrong, the basic is an array of compressed strings, then you'd have to decompress the strings afterwards.

4) You just need a correct reader of the text file so you can import the text file data into RMXP. Actually, that's what the rxdata format is for: to compress the data into a special format, then the files get smaller. You can use Marshal.dump(FILE, ARRAY) (or was it Marshal.dump(ARRAY, FILE)...?) for the dumping into rxdata format.

I hope this was helpful.

[shdwlink1993]

1) OK, slight problem: I don't know where in this line of code that would go:

file.write("    when #{i}\n"
             {...}
"  @element_set = '#{item[i].element_set}'\n

Specifically, the last line.

2) I've been trying to figure out how to get that animation.frames thing out of animation. Probably the same as #1, right?

3) Doesn't Zlib::Inflate have something do to with that?

4) Ah. Marshal.dump. OK.

5) (Because I forgot to mention it in the first post) All of the maps (mapxyz.rxdata) have different file names. Is there any way to get it to automatically run through (000, 001, 002, 003, etc.) and if the file exists, make a new text file for each one with the information inside it? As opposed to copy and pasting the exact same phrase and changing two numbers a lot of times.

[Blizzard]

1)

file.write("    when #{i}\n"
             {...}
"  @element_set = '#{item[i].element_set.inspect}'\n"

2) Yeah, but Animation::Frame instances won't work directly. You'll have to copy the attributes (variables and stuff). Here an example of that. I had to iterate through the frames to resize the animations for Bizz-ABS.

  def self.animations_size_down
    # iterate through all animations
    $data_animations[1, $data_animations.size-1].each {|animation|
        # iterate through all frames and all cells
        animation.frames.each {|frame| (0...frame.cell_data.xsize).each {|i|
            # if cell contains image
            if frame.cell_data[i, 0] != nil && frame.cell_data[i, 0] != -1
              # size down x position, y position and zoom by half
              (1..3).each {|j| frame.cell_data[i, j] /= 2}
            end}}}
  end

or more readable:

  def self.animations_size_down
    # iterate through all animations
    for animation in $data_animations[1, $data_animations.size-1]
      # iterate through all frames
      for frame in animation.frames
        # iterate through all cells
        for i in 0...frame.cell_data.xsize
          # if cell contains image
          if frame.cell_data[i, 0] != nil && frame.cell_data[i, 0] != -1
            # size down x position, y position and zoom by half
            for j in 1..3
              frame.cell_data[i, j] /= 2
            end
          end
        end
      end
    end
  end

3) I think yes, I haven't tried that, though.

5) MapInfos.rxdata is a dumped hash. Simply load the file and use .keys method on the loaded data and you will have an array of all existing map IDs.

[Blizzard]

A couple of posts were lost...

1) Works perfectly, thanks!

2) I'm still not sure how to get that into a text document. But then again, maybe trying to get an animation file into a text file may be a bad idea... Like, how would I get the enemy actions into the file. I'm pretty sure that's in the same file as the enemies, but I can't get it out.

3) Nevermind. I got something together.

5) So i'd use keys on it to get the array, and then... use the array as the file names?

6) (These things don't stop, do they?) The way I got the scripts out is rather... annoying. Lemme show you:

s1 = load_data('Data\Scripts.rxdata')
a = s1[1]
b = s1[2]
...(etc)
file = File.open('Extract/Script1.txt', 'w')
  file.write("#{a}\n")
file.close
file = File.open(Extract/Script2.txt', 'w')
  file.write("#{b}\n")
file.close
...(etc)

Is there any way to do this without using half a ton of code and forcing people to comment out the stuff they don't need? I do know there is a limit to the number of scripts you can have (remember the bunch of scripts I had in the same project a while back?), but I don't think that someone using it really wants to comment every line they aren't using.

2) I agree, better you just move it into the other file.

5) The array contains map IDs, just use the same call for each ID like in Game_Map#setup (the line with load_data and sprintf).

6) You can try if false/if true branches instead.

[shdwlink1993]

Oh, thanks for getting that post back! I was worried when almost five minutes after it gave me the maintenance mode thing.

2) "move it into the other file"? What other file did I mention?

3) Nevernevermind, it works on about two scripts and then it gives me garbled stuff, like poor Window_InputNumber over here:

Spoiler: ShowHide

78627028Window_InputNumberxœ½WmoÓ0þŽÄ8µLJÛ4´!V˜o• 
âÃ,,"7qZ£Ä	¶£nüzÎ/i"¶,,šEíßù|oÏÝyýð¨Ïý{}áãI±Žæ¼¬Ô‡*_PŒñQ­	¾¬˜,,µQø-˜Ö©_7Š¥,,'š[IšàfµbÔŠšr*%YRwF€´cÇãþ½8#R	<«‰/‰Ds Ž"sáã⠍Ì9SŒdìQ¬à-©Ÿ,,-™'QN®afWX"Ð"ãÌ ÞVó@s
ζ,,
Ã×f9KèÃÉâ*#ŠB\	‰©_³D­ Eîrï(Þd|ôgE2GЀ(
©Œ´SŸTy~-˜ÊI‰Z^š--€Óµ÷èÔ‡G§µ•VWd
[b¢×*'Ú»Þ¤7ìž<Ý×$L¢ÔrdURáM|ÀO[Á°-h[ž<v¶Hš¥A\pE¹'m›­ôØì_Q¶\)³h
þ,Qgø4hEIb¦n¶Q>G€ÒëíZÐTP¹²‹ªL0~'3X Î4ò¤#,¿¥
ênÒ	<ùæpí©ª¯q×­cŸÛŽ™Êtž
¦³aXg^‡žýìÖÛÕ•}Et~...>L'ºÇ79†)òßIw1»°5‰¨#|™QøjÐØQŒC½.Á-5Ty®h†íZÞ¯îM[é.JoÉï$4FV÷ç¹æ
æGdž®ãƒÃ »ð'tÁaM$"ˆœËV†¥`æc hI‰zá™ÕlöõÓ@Ÿs÷êã·³ -èn$o¤¢9¦"*3rã=@ûHMtñ--t#Ó7}éÛ( DLëâÓs"^!¼(à`TEÝÁm
])x;µ€_‹Ž{ë'zè¤OPºæ×¼qˆ[‡vÃÖ@I'$0õuL@eµJîÓ¦ã˜˜Ìé·tOæì/2­@÷±ðä¶ëÓè€O­9âàofº­j¡'Ö호œ¿}÷ecîjæày§5ëÿPÑ}u=»°žï^Mì{'Ñô/Ž¼ýæ®ý؁ë?{u×ÿr‹'W
ƒ­»Tg"ˆCŒ_p•z2òBä$‹ÌÒmFª,ã*õz'"aÒó›'õëšpé1ÿ¯à¥&AÈ@[w"È:ÒWLíÎÌéc3_ì]Õ yÅüé÷A+æÇüù
뉙

No clue why that happened. It got the first two right OK.

5) Since i'm still a newb at this sort of thing, I'm not sure how exactly to do that.

6) By annoying, I primarily meant that I have to have 1000 different files maximum. You saw how I was doing that.

a = s1[1]
b = s1[2]

1000 scripts... and most of them won't even display correctly for reasons unknown. That was what I meant by annoying.

And yes, i'll need to use if false/true branches. Thanks.

Sorry if this is getting annoying. The rate this is going, you'd deserve as much credit for it as me.

[Blizzard]

2) Nvm, I misread your post. Enemies have an array called "actions" with instances of RPG::Enemy::Action (I think). Just iterate through it.

3)

map_data = load_data('MapInfos.rxdata')
maps = []
for id in map_data.keys
  maps.push(load_data(sprintf("Data/Map%03d.rxdata", id)))
end

or

map_data = load_data('MapInfos.rxdata')
maps = []
map_data.each_key {|id|
    maps.push(load_data(sprintf("Data/Map%03d.rxdata", id)))
}

"maps" will contain an array of all maps then.

6) This is a little bit complicated if you don't know Ruby's Marshal format. I was able to figure the format out recently, I could post a topic explaining it.

The thing is that the first byte, if it's less than 128, can be taken as direct value as well as 0, except for 1, 2, 3 and 4. If it's 1, 2, 3 or 4, it shows how many of the folllowing bytes are part of the number. Before an integer, there is an 'i' in the file. The byte order is "little endian". Before an array there is a [ in the file followed by an integer with the number of elements in the array. The file always starts with the version of the format in two numbers (4 and 8 in RMXP) This is all still fairly simple until it comes to the method of compression (strings are marked with s, then comes the length, the comes the string). I think I should really post a topic about that...

7) As I said, I figured out the format when I was working on the configuration application for Blizz-ABS so I could display game data in an external application. When I post the topic, I might even post the C# code pieces of my partial decoder (I didn't need a full decoder, so I made a partial decoder only).

[shdwlink1993]

2) The RPG::Enemy::Action thing was what I meant. I just didn't get what you meant by other file.

5) I can get that working, but I'm not sure how to get this written into a bunch of .txt files. My attempt at that was shot down by flaming legions of error messages

6) Interesting... You probably should post about that. A lot of it goes over my head , but theres some people who could use that.

[Blizzard]

2) You are trying to export the stuff from the files, right? I meant the output file. Nvm if I got you wrong earlier.

5) Just open a file in writing mode and use .write on the instance. This writes a string argument into the file.

6) It's not really complicated if you have it all there. Even figuring out the format isn't that complicated, it took me less than a day to figure out the format and make a partial reader.

[shdwlink1993]

2) Oh. I should have realized that. >.< In that case, I was meaning that I don't know what syntax to get it out of there. I know that it's RPG::Enemy::Actions, and I'm fairly certain it's in enemies.rxdata, but I don't know exactly what to tell it to get that I need that.

5) OK. I've been trying that, but I'm not sure exactly how to write that in. At all.

6) And you're using that for your game, chances are, right? See? You're really smart with this kind of stuff. I'm horrible, but I'm trying :.

7) (Don't worry, this is a lot easier.) On the title screen, I have it write something on the screen (version). How do I get it to go away after the title screen? It stays there while I'm playing the game, and it's fairly annoying.

[Blizzard]

2) Well, if you put it into a text file, you will need to parse the file to get the data out again. I suggest using a text file only if you want to print out data, it's not really suited for data storage. You'd have to create a new format if you want to save the stuff and load it again. But creating one isn't as easy, especially if you work from scratch, so I suggest you keep using the Marshal format, you don't have to use .rxdata as extension, though.

5)

file = File.open('NAME', 'w')
file.write('Hello file!')
file.close

The stuff will be written into the file AFTER you use .close.

6) Actually for the configuration application for Blizz-ABS. You're not horrible, you're still learning. I couldn't figure out that format earlier either, because I had no idea where to start from. When you have more experience (points, LOL), you will figure out half of it by just looking at some saved data in a hex editor.

7) Are you using a separate sprite or window? You need to call the .dispose methods for such, so they get removed from memory. Don't use a global variable for that (variable with $ before the name), but use an instance variable (with a leading @ instead) in Scene_Title#main. Create the sprite/window and draw the version before the "Graphics.transition" and "loop do" and after the "Graphics.freeze" dispose it.

[shdwlink1993]

2) OK. The entire point of the script is just to see how it does stuff. That's it.

5) I meant that I don't know how to specifically write in that string you meant. I can get something like 'Hello file' in easy. I mean the string you mentioned.

6) That's part of the problem: I don't earn Experience Points in games, usually I try and makethe game give me the points (read: usually I hack the crap out of it)

7) Worked perfectly. Thanks.

[Blizzard]

5) Oh, you mean how to put the string together? Well, you can do something like

...
for animation in $data_animations
  if animation != nil
    str = animation.id.to_s + ' - '
    str += animation.name + '; '
    str += 'Frames: ' + animation.frames.size.to_s
    ...
  else
    str = 'NIL'
  end
  file.write(str)
end
file.close

Just put in there the information you need. Don't forget to use .to_s for non-string values or you'll get an error.

6) Just lol!

[Sally]

are you doing this to let us be able to cheat in our games?

[shdwlink1993]

@Susys: I'm not sure how you'd be able to do it. Ideally, the goal is that if you are going on an extended vacation or something like that and you will have a computer but not RPG Maker XP (like when I go to school. Those computers are darn near under lock and key), you can change text files and have RPG Maker XP reincrypt them into marshall format so you can edit them.

@Blizzard:
5) OK. Slight thing: that's not the string i'm looking for.

I'm looking for something that would get animation.frames.cell_max or enemy.action.basic. Things like that. And how exactly to get the maps thing working properly. Then that might be applicable to the scripts thing, since it's kinda similar yea, right....

6) I also think that posting about that Marshall format would be helpful to a lot of people here.

[Fantasist]

I also think that posting about that Marshall format would be helpful to a lot of people here.

Seconded. I've learnt some things from this topic, thanks guys *powers up Bliz and shdwlink*

[Blizzard]

5) Do you have the english help file? You have the complete structure in there. I can upload the file if you don't.

[shdwlink1993]

5) The only reason I know all this stuff is because I had the English help file. I also have the Japanese help file and the translated help file (that for some reason isn't finished). Screw it, i've even got the VX English and Japanese help files! It's just that none of them say how to get the information from a "subsubclass" of RPG. Actually, they don't say anything about getting info. I've just been getting lucky guesses because most of the info is named after a file. I just don't know how to get info out of something that
a) does not have a file.rxdata named after it, like weather, or
b) is a "subclass", like how I can get to Class, but not Class::Learning.

[Zeriab]

For the Scripts.rxdata :3

script_data = load_data("Data/Scripts.rxdata")
scripts = []
for raw_script in script_data
 scripts << Zlib::Inflate.inflate(raw_script[2])
end
for script in scripts
  p script
end

I am sure you can figure out the rest ^^

[Fantasist]

Wow, a little change to that code and I got very interesting results =D