ARCed.NET

Started by ForeverZer0, June 30, 2012, 01:50:36 pm

Previous topic - Next topic

ForeverZer0

As was talked about in the other topic, I created a .NET based variation of the editor. I am committing all my work sometime today, but in the meantime I can tell you two a bit about it.

Dependencies (End-user needs installed)

  • Microsoft .NET 4.0 ("Full" for developers. I haven't tested, but users should only require the Client Profile)

  • Windows XP or higher



External Native Libraries (Included)

  • 7z.dll or 7z64.dll, the libraries used for 7zip

  • SciLexer or SciLexer64, the native Scintilla libraries



.NET Assemblies (Included)

  • Dejavu.dll, a undo/redo framework I have yet to implement, but it makes that pretty easy.

  • SevenZipSharp.dll, a wrapper for the native 7zip libraries

  • Microsoft.Dynamic.dll, assembly that allows for "dynamic" types that basically allows you to work with data of unknown types without compiler issues

  • Microsoft.Xna.Framework.dll, the primary XNA library for rendering

  • Microsoft.Xna.Framework.Graphics.dll, a secondary XNA containing various types used for rendering



With the exception of the .NET Framework needing installed on the host machine, the application is completely portable, and will run off a USB stick without need for further installation. I would still like to have it as an installed program though, since if you are running portably, file associations cannot be made without modifying the Registry. This means there would be no "click project to open" or associated icons, the user would need to manually open the editor and load a project, or drop a project on top of the editor icon. BTW, I have setting file associations and icons done, they can only be made if the editor is run as Administrator obviously, which it also checks for. Making the app portable for the end-user will be nothing more than copy-pasting the install directory to a USB stick or whatever though, so thats a plus. The application stores data in Appdata/Roaming/ARCed.NET", so to create 100% portability, I will need to include a setting to offer the choice to store settings locally in the program directory. Small thing there.

Features (Implemented)

  • Ryex's cool idea for using "templates" to create projects. I am keeping the template files compressed, and using 7zip to extract them if they are chosen from the "New Project" dialog. In the "File" tab of the main menu strip, there is an option to "Save as Template", which will compress and create a copy of the current project to create a new template to use for new projects. There is a "default" project that is embedding in the program, so that will always be available, and there won't be any worries of screwing up new project creation.

  • Script templates. I went a step farther and also included script templates, meaning you can do the same as above when creating new scripts. Pretty much like what VS does when you choose "New Class...", etc.

  • Backups (another Ryex idea ;) ), an automatic system for creating backups. There are various options on when to save (on play test, on every save, at intervals, etc), and options on what to save (scripts, database, maps, etc). The backups are compressed and stored locally with the project. They are given a GUID for a name. I would have worried about actual filenames, but GUIDs are much simpler, since the only real factor needed to differentiate is creation times of the backups. There is a small manager for it that allows for setting max backups created, as well as deleting and restoring. I dubbed it the "ARChive Utility".

  • I am using a free and open-source docking library I found on SourceForge called DockPanel Suite. I made various edits to support default windows sizes for floating windows, etc, and implemented a skin manager that allows for changing colors and fonts. It provides the VS type of docking functionality we want.

  • I am using Scintilla.NET again for the editor, same as I did for Gemini. I made various improvements to it, and it still have more plans for it, which I will list in the "Planned Features" section I am going to write after this one.

  • I am including settings for nearly everything. Most of these can be ignored, and are mainly aesthetic changes, but its a nice touch. Script style settings for fonts and colors of data types is fully configurable, same as it was in Gemini, although I made a different form for it.

  • Plugin system. The plugin system is extremely simple to use. I am storing all important data used in the editor and project within a few public static classes. All someone needs to do is create a form in any .NET language (C#, Visual C++, or Visual Basic, etc.), make a reference to the main editor, and have their form inherit from the dock form and IPlugin interface. All project, editor, windows, and settings that a plugin would have business altering are easily accessible through these class. They can simply do what they need through them. The editor will handle everything else. Just throw the finished dll or exe into the plugin folder and loads up when the editor is started. These plugins are "registered" with the editor, which can then pick them apart with Reflection to tell anything you need to know about them.

  • I have created loads of custom controls for the editor as well. Many of these are for reducing redundancies, but there are many that also provide a rich-GUI experience for the editor. I created a custom ColorPicker that replaces the boring default Windows one. It supports RGB and HSV, contains a built in capture picker that lets the user pick a color from the screen. There is also a custom FontPicker. The font picker was a bit of a necessity. I implemented loading and using fonts from a file so that they didn't need installed (the portability thing). GDI+ requires only OpenType fonts and the standard dialog does not contain filters for that. On top of that, even when you forced enumeration of "memory fonts", the dialog would throw an error when you selected them and chose "OK". I researched this a bit, and according to Microsoft, they are aware of the issue, but have no plans on ever fixing it, since it lies so deep in the API that to change it would cause only more bugs. Its a small thing, and I have overcome it already, so no problem there.

  • Serialization. This was a bitch to do. I managed to implement this with the use of Reflection, and I am proud to say the results are outstanding. It is quite capable of reading and writing the ARC data format with great speed. After I dropped IronRuby, I knew this would be one of the largest obstacles to overcome, and I finally got it working flawlessly after a 2 days. I still need to alter the Table class to what Blizz mentioned about 0 sizes. It cannot currently load Animations.arc due to that, but it has no problem reading and dumping Tables under other circumstances. I just haven't got around to it yet.

  • RTP and local resources have been full implemented. Both audio and graphics are loaded as a "GameResource", and there is a central class that handles accessing them. There are many different methods and properties to get all resources of certain types, whether it be all local, all rtp, all graphics, all local audio, all local SE, all RTP characters, etc, etc. You get the point. In the "picker" dialogs when users select them, they are given markers, just like in RMXP, to differentiate their location. Local resources are listed first, same as RMXP. I also implemented FileSystemWatchers to detect changes to these directories and refresh itself accordingly without having to worry about it.

  • There are loads of other minor things I have not mentioned here. I have built it to be extremely dynamic as a whole. Practically nothing "hard-coded" in, and where it is they are mainly placeholders. This will allow for expansion into whatever is needed. For example, instead of have 5 slots for actor equipment, the slots are created at runtime from reading a configuration (which is a simple placeholder for now). The configuration simply tells what label and "type" to include in its slot, and the rest is done automatically from there. If we decide we want to have 3 equippable accessories, all it takes is to simple add a some extra configuration. All the logic to add to the form and automatically alter the data is already there without having to code anything else. Same thing with parameters. You basically just add a configuration that says X's label is SPD, its associated data type to effect is a property called "spd", and everything else is done. The forms adds a numeric up-down box labeled "SPD" to the form, and when changed, it alters a parameter of the current game object that is named "spd". There would be obvious caveats that would need addressed if this configuration was open to the user, such as memory corruption, but it makes altering things for a dev to be a simple matter of 5 seconds.



Planned Features (not/partially implemented)("*" Denotes a requirement before completion)

  • Tilemap editor, although I already know exactly how to do it. Its not just implemented yet. I have already created a C# version a while back that uses GDI+ when I made the MapConverter project. It will be quite simple to use the same logic in an embedded XNA panel. (Already implemented embedding XNA panels into WinForms, so no problem there) (*)

  • Snippet manager for the script editor

  • Creation of a full plugin manager that allows for viewing information about and control of plugins, etc.

  • XAL wrapper for using the library within the editor. (*)

  • Finish the last few database forms (Tilemap, Animation, and System)(*)

  • Implement an "EventBuilder" class that creates RPG.EventCommands from code values and parameter arrays. (Already finished that parser that does the obvious to display in the event editor windows)(*)

  • Improve handling of creating/saving/loading/renaming script files

  • Make some improvements to make a few things more modular that I omitted

  • There is a more detailed TODO.txt file that I am committing, as well as plenty of TODO comments throughout the code I will be committing.


There are a number of things I am sure I am forgetting to mention, or need explained fuller, but I just wanted to give you guys a general idea of what's going on so far, so that I after I commit, you have some type of initial knowledge of what the hell is going on. If you have any questions or concerns, please don't hesitate to say so. I have been solo on this without having a second person there to call out major flaws I may be overlooking, so anything you have to say is important.
I am done scripting for RMXP. I will likely not offer support for even my own scripts anymore, but feel free to ask on the forum, there are plenty of other talented scripters that can help you.

Blizzard

July 01, 2012, 06:22:40 am #1 Last Edit: July 01, 2012, 06:26:06 am by Blizzard
Comments:

1. I love the fact that you added plugins after all.
2. The serialization of the Table class shouldn't be much of a problem. You can check the Ruby/C++ code for reference. Maybe even better, check the SVN log for the revision where I fixed that problem and you will see what exactly I did so you can apply an equivalent patch in your code.
3. So we will ship ARC with all necessary binaries except .NET 4.0, right?

Suggestions:

1. On first run, check registry for .arcproj extension binding and add it if it doesn't exist. That way a portable app can still be associated with arcproj files. Of course only if we want to do this. The point of a portable application is that it doesn't edit registry or anything after all. Thoughts?
2. Do we need XAL in the C# editor for audio or is there a way (there should be) to play audio files in C#? I know that I have implemented a localization dialogue editor for voice overs at Cateia for our German publisher of Kaptain Brawe and I have used both special audio controls (to play wav files, I still had to read them manually) and a PictureBox for the waveform display and visual play position indicator.

What do you guys think about me announcing to the community that we will soon have an editor beta?
Check out Daygames and our games:

King of Booze 2      King of Booze: Never Ever
Drinking Game for Android      Never have I ever for Android
Drinking Game for iOS      Never have I ever for iOS


Quote from: winkioI do not speak to bricks, either as individuals or in wall form.

Quote from: Barney StinsonWhen I get sad, I stop being sad and be awesome instead. True story.

ForeverZer0

Yes, the only external dependency required by the end-user is to install .NET 4.0, which many will already have. With the popularity if .NET increasing, I doubt that will be issue to anyone.

As for file associations, like I said I am at a bit of a crossroads. Actually setting the association is no problem, but as you said, editing the registry defies the purpose of being portable. There is also the fact that if the program is sitting on the Desktop when the association is made, and then is moved to "My Documents" later on, the association is broken. I will likely end up having a setting for it. If portable, no association is made and settings are stored in the program directory instead of AppData, or choose the "normal" way of placing it a stationary location and setting associations.

As for audio, back when I made the "MCI Player" script for RMXP, it was based off a C# version I made. I actually already have a fully working audio player with seek functions, etc., not to mention XNA provides functions for playback, as well as even normal functions built in to the "System.Media" namespace if .NET. My only concern was accuracy of sound with XAL. It is difficult to find a good audio library for C# that has all the features I am looking for. NAudio, Bass.NET, Alvas, etc. are the probably the largest, and none of them meet the following criteria:


  • Free and/or free for commercial use

  • Support for MP3, OGG, WAV, and MIDI. The MIDI is the main issue. I know we are no currently supporting MIDI, but I don't want someday its inclusion in the engine to require me to have to totally redo audio in the editor.

  • Pitch changing

  • Ability to get the data as arrays of floats that can be converted to waveforms



I have yet to find a library that has all that. I was looking into SWIG, which is can somewhat generate a wrapper for you, but to be honest, it kinda confuses me. I didn't spend a whole lot of time worrying about it yet, but I was a bit in over my head when I was looking at it a while ago. Manipulating audio programmatically and working with raw data is not my thing. I admit I don't understand half of it. :(

As for the announcement, I personally have no problem with it.
I am done scripting for RMXP. I will likely not offer support for even my own scripts anymore, but feel free to ask on the forum, there are plenty of other talented scripters that can help you.

Blizzard

July 01, 2012, 12:05:19 pm #3 Last Edit: July 01, 2012, 12:12:27 pm by Blizzard
Yeah, working with audio seems simple at first, but when you realize that you have a bunch of different things (sampling rate, number of channels, bits-per-sample etc.), you realize that it can become quite complicated. I don't mind you wrapping XAL, I was just thinking that you have a quicker solution present already. After all, if I can get MIDI working, it will be implemented in XAL as well.

EDIT: BTW, as far as I remember, you can very easily create an installation for the editor which you can configure to have .NET 4.0 as a prerequisite. Then if the user does not have .NET 4.0 installed, our installer will download and install it for him.
Check out Daygames and our games:

King of Booze 2      King of Booze: Never Ever
Drinking Game for Android      Never have I ever for Android
Drinking Game for iOS      Never have I ever for iOS


Quote from: winkioI do not speak to bricks, either as individuals or in wall form.

Quote from: Barney StinsonWhen I get sad, I stop being sad and be awesome instead. True story.

ForeverZer0

I have never used it, but I have seen the options in VS to do that.
I guess I won't be able to say "I never used it" for long, lol.
I am done scripting for RMXP. I will likely not offer support for even my own scripts anymore, but feel free to ask on the forum, there are plenty of other talented scripters that can help you.

Ryex

I'm not sure if you'll understand my relief but they way you went on about this it gave me the impression that it was better and had more features than the Python version. when you first \told me about it you had me wanting to pull my hair out and go "NOOOOOOOOOOOOO YOU CAN'T HAVE REPLICATED AND BESTED MY WORK IN ONLY 2 MONTHS!"  needless to say it both pissed me off and made me extremely jealous.

now, I know and knew that none of that was you intention so don't feel guilty or anything I just had to get that out there. 

I'm fine with an announcement so long as we are clear that this editor version is only for the legacy edition and the Full version of ARC will have a largely similar but different editor built in python.
I no longer keep up with posts in the forum very well. If you have a question or comment, about my work, or in general I welcome PM's. if you make a post in one of my threads and I don't reply with in a day or two feel free to PM me and point it out to me.<br /><br />DropBox, the best free file syncing service there is.<br />

ForeverZer0

No, its almost impossible this will ever take the place of THE editor for ARC. At the very most it will likely only be a secondary option for Windows. It's mostly built in a manner that would require only a minor edit internally to offer non-Legacy support, but that is not its primary purpose, more of just a byproduct of keeping it very modular.

To put any lingering doubts of yours at ease, I will go far as to say this will never be cross-platform.
I am done scripting for RMXP. I will likely not offer support for even my own scripts anymore, but feel free to ask on the forum, there are plenty of other talented scripters that can help you.

Blizzard

Don't feel bad, Ryex. The productivity you can achieve with .NET is just insane. e.g. The first version of the Blizz-ABS config application took me 4-5 days. I improved the internal structure and data handling through later refactorings, but the initial version was still made in only 4-5 days.
Check out Daygames and our games:

King of Booze 2      King of Booze: Never Ever
Drinking Game for Android      Never have I ever for Android
Drinking Game for iOS      Never have I ever for iOS


Quote from: winkioI do not speak to bricks, either as individuals or in wall form.

Quote from: Barney StinsonWhen I get sad, I stop being sad and be awesome instead. True story.

Ryex

when I load the solution
---------------------------
Microsoft Visual Studio
---------------------------
'C:\Users\Ryexander\Desktop\ARC\editor\ARCed.NET\Documentation\Documentation.shfbproj' cannot be opened because its project type (.shfbproj) is not supported by this version of the application.

To open it, please use a version that supports this type of project.
---------------------------
OK   Help   
---------------------------


this is in the output when the solution loads
C:\Users\Ryexander\Desktop\ARC\editor\ARCed.NET\PluginTestForm\PluginTestForm.csproj : error  : Unable to read the project file 'PluginTestForm.csproj'. 
C:\Users\Ryexander\Desktop\ARC\editor\ARCed.NET\PluginTestForm\PluginTestForm.csproj: The project file could not be loaded. Could not find a part of the path 'C:\Users\Ryexander\Desktop\ARC\editor\ARCed.NET\PluginTestForm\PluginTestForm.csproj'.



also
I'm getting build errors because of missing files

Error 1 Unable to create a manifest resource name for "EventBuilder\CmdConditionDialog,cs.resx". Could not find file 'C:\Users\Ryexander\Desktop\ARC\editor\ARCed.NET\ARCed.NET\EventBuilder\CmdConditionDialog.cs'. ARCed
I no longer keep up with posts in the forum very well. If you have a question or comment, about my work, or in general I welcome PM's. if you make a post in one of my threads and I don't reply with in a day or two feel free to PM me and point it out to me.<br /><br />DropBox, the best free file syncing service there is.<br />

ForeverZer0

Damn it, the first error is because the documentation project is not uploaded. You can either install Sandcastle, or open the .csproj file instead. I'll upload the documentation after work, and get a link for Sandcastle.

The second error it seems is that I missed committing a file... I'll fix that to.

Sorry ...
I am done scripting for RMXP. I will likely not offer support for even my own scripts anymore, but feel free to ask on the forum, there are plenty of other talented scripters that can help you.

Blizzard

Tortoise SVN should display a list of files that were changed and that list also includes files that have not been added. It's always a good idea to scroll through that list before big commits with lots of files.
Check out Daygames and our games:

King of Booze 2      King of Booze: Never Ever
Drinking Game for Android      Never have I ever for Android
Drinking Game for iOS      Never have I ever for iOS


Quote from: winkioI do not speak to bricks, either as individuals or in wall form.

Quote from: Barney StinsonWhen I get sad, I stop being sad and be awesome instead. True story.

ForeverZer0

I need to manually add the files. For some reason it is only listing files that are inversioned or modified. The list of unversioned is quite large, since VS likes to generate so much debug stuff.
I am done scripting for RMXP. I will likely not offer support for even my own scripts anymore, but feel free to ask on the forum, there are plenty of other talented scripters that can help you.

Blizzard

You have the "ignore" property for that. You can try it out. Right click on the engine folder, select Tortoise SVN, then "Properties". There you can see an example of how I added the bin folder to be ignored as well as some other user-dependent VS stuff. I recommend using that feature for files that you know you will never commit.
Check out Daygames and our games:

King of Booze 2      King of Booze: Never Ever
Drinking Game for Android      Never have I ever for Android
Drinking Game for iOS      Never have I ever for iOS


Quote from: winkioI do not speak to bricks, either as individuals or in wall form.

Quote from: Barney StinsonWhen I get sad, I stop being sad and be awesome instead. True story.

ForeverZer0

As you may have already noticed, I updated added/removed/ignored files in SVN.
One of the things I added was the documentation project, which requires SandCastle to build. Its not set to build with in the solution, but you will need a couple things installed.

http://sandcastle.codeplex.com/
http://shfb.codeplex.com/

If you are unfamiliar with SandCastle, it is the equivalent of what RDoc is to Ruby or whatever the help builder for Python is called, but for .NET. You comment your code using XML tags, which it parses and generates a help file of. It takes quite some time to build, so I wouldn't suggest having it build each time the project is ran. Its pretty cool, though. :)

All the commit issues should be ok now. Let me know if you have any more problems.
I am done scripting for RMXP. I will likely not offer support for even my own scripts anymore, but feel free to ask on the forum, there are plenty of other talented scripters that can help you.

Blizzard

I remember Sandcastle. We used it at college to generate documentation files.
Check out Daygames and our games:

King of Booze 2      King of Booze: Never Ever
Drinking Game for Android      Never have I ever for Android
Drinking Game for iOS      Never have I ever for iOS


Quote from: winkioI do not speak to bricks, either as individuals or in wall form.

Quote from: Barney StinsonWhen I get sad, I stop being sad and be awesome instead. True story.

ForeverZer0

Well good new and bad news. Actually its the same news, just depends on your perspective. I'll start with that "bad".

I spent tonight doing some testing inside some freshly installed Windows XP virtual machines to test dependencies. I come to find the following: The editor in its current state requires a XNA 4.0 Game Studio. By that I don't mean the small redistributable package either, I mean the 50 MB developer framework. What's better, in order to even install that, the host machine needs Visual Studio installed, so double the fun. On a side note, a dependency of those two requires a full .NET installation, not client profile, but when you are installing almost a GB of bullshit to run a 5 MB application, what's 10 more MB, right? Obviously, this does not even come close to being in the realm of "acceptable", not in anyone's wildest imaginations. I would tell someone to shove ARC up their ass if they told me I had to install all that to run it. So I now I come to the "good" news.

I know how to fix it. Its not really even that difficult, I just need to change much of the current structure of the project. As it sits now, everything is pretty much in one project, all built into one project. All that really needs done is to move some of these namespaces into separate projects and assemblies so that they can be built using a different profile. I know of a few places where I will need to change the actual code around to avoid circular dependency within the solution, but all in all, this is something that needs done anyway. After it is done, the solution will be much more tidy and better organized, as well as much easier to maintain. Instead of having a single EXE, there is going to be a few more DLLs, etc, and a smaller main app, but in the end, I see this as a good thing. It may take me a whole night to get everything to where it needs to be, but in the end result is going to be better.
I am done scripting for RMXP. I will likely not offer support for even my own scripts anymore, but feel free to ask on the forum, there are plenty of other talented scripters that can help you.

Ryex

Sound like the exact same thing I have to go through to get the Cython version of ARCed to build and run with out missing dependencies.  good luck.
I no longer keep up with posts in the forum very well. If you have a question or comment, about my work, or in general I welcome PM's. if you make a post in one of my threads and I don't reply with in a day or two feel free to PM me and point it out to me.<br /><br />DropBox, the best free file syncing service there is.<br />

ForeverZer0

I am done scripting for RMXP. I will likely not offer support for even my own scripts anymore, but feel free to ask on the forum, there are plenty of other talented scripters that can help you.

Blizzard

The .NET dependency is so-so, but all the rest is really kind of pointless. So you're sure that splitting the code would actually remove the XNA and VS requirements?
Check out Daygames and our games:

King of Booze 2      King of Booze: Never Ever
Drinking Game for Android      Never have I ever for Android
Drinking Game for iOS      Never have I ever for iOS


Quote from: winkioI do not speak to bricks, either as individuals or in wall form.

Quote from: Barney StinsonWhen I get sad, I stop being sad and be awesome instead. True story.

ForeverZer0

Well, I discovered something quite embarrassing.
I discovered the source of the XNA Developer requirement, and it has nothing to do with separating namespaces and building the assemblies in parts. XNA uses something called the "Content Pipeline" to load textures, fonts, audio, and other resources. These resources are added to a project, and compiled into .xnb files when the project is built. My problem is, I am simply using an embedded panel on a WinForm, and there is no pipeline, nor any compiling of resources. At one point, I implemented compiling resources dynamically at run-time, which used the "Microsoft.Xna.Framework.Build" assembly. This is not part of the basic framework, and requires a full install, which is why I abandoned the idea (I have since created an alternative method to do I wanted).

Well, guess who forgot to remove the reference from the project? That is why it wouldn't run. It was trying to make a reference to an assembly that I wasn't even using. :facepalm:

Oh well, I am almost done with the reorganization, and I am pleased with it so far. The solution is much more tidy, although I am going to have to go back and make a few modifications to get everything back t 100%. I haven't had a lot of time to work on things that past couple nights, but I got a lot done tonight. I am going to get everything to how it should be before committing, it will be much easier than constantly tracking so many added/deleted/moved files/folders. It will hopefully be tomorrow night.
I am done scripting for RMXP. I will likely not offer support for even my own scripts anymore, but feel free to ask on the forum, there are plenty of other talented scripters that can help you.

Blizzard

:troll:

BTW, I noticed that you committed the binaries as well (bin and obj directory). I think you can safely put these on the ignore list.
Check out Daygames and our games:

King of Booze 2      King of Booze: Never Ever
Drinking Game for Android      Never have I ever for Android
Drinking Game for iOS      Never have I ever for iOS


Quote from: winkioI do not speak to bricks, either as individuals or in wall form.

Quote from: Barney StinsonWhen I get sad, I stop being sad and be awesome instead. True story.