I am having a bit of a problem that I can't seem to figure out. I am using IronRuby's Zlib Library in a .NET application through embedded to compress strings. Here's a sample of a script I am using, and calling through the application:
load_assembly 'IronRuby.Libraries', 'IronRuby.StandardLibrary.Zlib'
def load(path)
file = File.open(path, 'rb')
object = Marshal.load(file)
file.close
return object
end
def save(object, path)
file = File.open(path, 'wb')
Marshal.dump(object, file)
file.close
end
def decompress(string)
return Zlib::Inflate.inflate(string)
end
def compress(string)
zlib = Zlib::Deflate.new(Zlib::BEST_COMPRESSION)
compressed = zlib.deflate(string, Zlib::FINISH)
return compressed
end
This works perfectly fine. It also works for inflating objects that were Marshalled with Ruby 1.8.6. The problem arises when I compress and Marshal the string with IronRuby, then attempt to read the data in Ruby 1.8.6. The compressed data is different, and I get "incorrect data check" errors. It still loads fine in IronRuby, though, so I'm not sure if it has something to do with different versions of the Zlib library, so if is simply an incompatibility between the languages. I have tried compressing the data in various ways as well, using a "--Zlib::MAX_WBITS", and every possible compression level, all still to no avail. Its getting rather frustrating... :???:
Any ideas?
I remember having issues with this. I finally got it working after downloading a different version of IronRuby. If it helps you can use the libraries from my Price Fix app I posted awhile ago. It has the IronRuby libraries packed with it. http://forum.chaos-project.com/index.php/topic,9777.0.html Hope it'll help.
Yeah, if it cannot be read by Ruby 1.8.6, then it's a new version of Marshall/Zlib for Ruby that's being used by Iron Ruby. Using an older Iron Ruby version should do the trick.
Using IronRuby 1.0.0.0 with .NET 2.0 is still not doing the trick. It just seems to compress the data differently. I have also noticed that the compressed data is significantly larger than standard Ruby's. Either way, the Ruby Zlib library fails at reading it. It still throws the "incorrect data check" exception.
I'll just say what I'm trying to do. I created an external RMXP/RMVX script editor. It has a lot of neat features that the standard one does not have. Its not a full blown IDE, but it is a definite improvement. So as you can see, the ability to save the Scripts.rxdata file back into a format that can be read is a pretty pivotal function. :P
Everything else seems to work perfectly, except when you open RMXP's script editor, all the text for the scripts is blank. Stranger yet, even if you write new text into them and save it, the data still does not show up. At first I though perhaps I screwed up the sorta "ID' that RMXP assigns to each script, defined in the first element of each scripts array (in Scripts.rxdata), bur even when I tried loading the file from another project, you can clearly see the data as it is compressed, but any attempt to use Zlib::Inflate on it wil just throw the error. It doesn't do it when you start the editor, so I imagine Enterbrain must have included some catch in it to bypass any thrown Exceptions on the decompressing process.
Either way, I am very disappointed. It seems my only choice now is to try and find some Zlib library that is not native to .NET, and try to interface it. :???:
Nah, it's most probably just a version conflict. Enterbrain never did anything smarter than the little bit of encryption in the RGSS.dll. If I remember right, in Ruby 1.8.6 the Marshall version is "48".
Can you run the game with the modified Scripts?
I also just noticed that RMXP uses Ruby 1.8.1. I knew that before, but for some stupid reason, I have been thinking it is 1.8.6.
The Marshal version is still 4.8, though.
What gets me is that both the IronRuby Zlib version, and the RMXP Zlib version are both 0.6.0.
Is it possible I am messing something up with the "id". To be honest, I am unsure exactly how it is used. I am just assigning a random integer to it. I know of the existence of "magic numbers" that Ruby uses, but I have never looked into how they are used.
EDIT:
@GG: No, it does nothing, same as if you erased all the scripts.
Scratch that. Forget this post.
No, Scripts.rxdata is an array of arrays. Each inner array is a three element array: [ID, "TITLE", "STRING"]
EDIT:
Hahah, I saw your fail before you could delete it :V:
What was it, what was it? D:
Yeah, you're right. RMXP uses 1.8.1. I keep forgetting that and always keep thinking it's 1.8.6.
Ok, I am seriously at a loss. I have been trying for 2 days to find why the hell this won't work. Both the Marshal and Zlib versions are exactly the same, and still nothing seems to work. It has nothing to do with the ID, I eliminated that possibility. Its not a problem with the string encoding, both are in UTF-8.
Has anybody successfully used Zlib::Inflate in RMXP on a string that was deflated with IronRuby's Zlib? I'm sorry for the repeated whining, its just I have never been so thoroughly stumped on something that, from what I can see, should work.
Deflate the same string in both versions and compare them. If they differ, something's off.
RMXP:s = "this is a string"
z = Zlib::Deflate.deflate(s)
p z
(http://dl.dropbox.com/u/20787370/Pics/Ruby181Test.png)
IronRuby:var test = _engine.Execute(@"
load_assembly 'IronRuby.Libraries', 'IronRuby.StandardLibrary.Zlib'"
s = 'this is a string'
z = Zlib::Deflate.deflate(s)
return z", _scope);
MessageBox.Show(test.ToString());
(http://dl.dropbox.com/u/20787370/Pics/IronRuby100Test.png)
Now that appears to be different string encodings. I'm not sure if the #ToString() method is messing with the text or not, so to be sure, I Marshall.dump the compressed string, and then Marshall.load it in RMXP to see what it looks like.
IronRuby Code_engine.Execute(@"
load_assembly 'IronRuby.Libraries', 'IronRuby.StandardLibrary.Zlib'
file = File.open('test.dat', 'wb')
s = 'this is a string'
z = Zlib::Deflate.deflate(s)
Marshal.dump(z, file)
file.close", _scope);
test = load_data('test.dat')
p test
(http://dl.dropbox.com/u/20787370/Pics/RMXPTest2.png)
And when I attempt to use Zlib::Inflate on the string...
RMXP Coderesult = Zlib::Inflate.inflate(test)
p result
(http://dl.dropbox.com/u/20787370/Pics/error.png)
The only thing left to do now that I could see that would make a difference is testing all the different compression levels. I really don't see how that would make a difference, though, since Zlib::Inflate should inflate the string regardless of its compression level.
Yeah, the encoding in the message boxes is different, but both strings loaded in Ruby look different when deflated. You'll have to research some more.
Well Gzip seems to work fine back and fourth between the two. I can compress a string with IronRuby using it and load it up in RMXP just fine. Unfortunately, this doesn't help with my issue...
EDIT:
Well, after spending about 5 MORE hours researching Zlib, posting on the IronRuby developer's forum, looking at one or two other projects that messed with the Scripts.rxdata file, and researching string encoding, I have come to the conclusion that currently IronRuby's Zlib library is not compatible with standard Ruby's library.
I have come up with a solution that will allow me use my .NET application to achieve what I need, but it is the most low-down dirty coding I have ever done. I am going to try it and hide it so that I never have to face the shame of it. It will work, though, and at this point, that's good enough for me...
Don't hide it. This kind of coding will be unavoidable in some real life scenarios. e.g. While we were using Python, PyOgre and CEGUI, we had to do a couple of nasty hacks ourselves to make it work properly. :/ That's the disadvantage when you have to work with an existing system that has design flaws.
let me guess. you went the long way around. calling in an third party perhaps exploited a bug or two, did something that really should not of worked had the system been designed properly. all in the name of success?
I've had to do something similar when getting pyglet to render to a WXPython windows opengl context. the result is stable but i really shouldn't of had to go about it the way I did. far as I know I'm the first person to actually do it. or at least the first person to SHARE how they did it.
I'll never tell.
Here is what I did:
- Using my .NET app, I output each script's text into a file, whose name is its unique id (0 - int::MAX), and save it to a single directory
- I then write a "keyfile" that is basically a list where each key looks something like this: "SCRIPT_ID - SCRIPT_TITLE
- The last line of the keyfile is the directory of Scripts.rxdata/Scripts.rvdata for the current project
- On open, the application unpacks RGSS102E.dll, Game.exe, Game.ini, and Game.rgssad to a local directory. The script within it is written to parse the keyfile, load the script, compress, set to an array, and save to the passed directory
- In my application, it starts Game.exe as a hidden process, and the actual save is done by RMXP
- After save, all the files are deleted, and the the RMXP files are deleted when the application closes.
Could always make a few ruby scripts and use ocra to convert them to executables.
http://ocra.rubyforge.org/
Run them in the background, do a process watch or something, and so on and so fourth. I've done it this way before and it works pretty good.
Nah, I thought of that, but then I would have to embed the Ruby interpreter, and the filesize would shoot up. This way, its only about 800 KB difference.