Chaos Project

Featured Projects => Advanced RPG Creator => ARC Welder => Topic started by: Ryex on April 08, 2011, 07:02:50 pm

Title: Setting up your environment to Work on the editor
Post by: Ryex on April 08, 2011, 07:02:50 pm
Well I think it's time to update this. I'm upgrading to python 2.7 I really can't figure out why I was using 2.6 to begin with probably some library I'm not using anymore besides I keep finding features that were added in 2.7 that I want to use but can't.


Windows


first off install Python 2.7.2
http://www.python.org/download/releases/2.7.2/

then install wxpython 2.8.12.1 for python 2.7
http://www.wxpython.org/download.php#stable

then install PIL (Python Imaging Library) 1.1.7 for python 2.7
http://www.pythonware.com/products/pil/#pil117

next install Pyglet 1.1.4  
http://www.pyglet.org/download.html (http://www.pyglet.org/download.html)

then numpy for python 2.7
http://sourceforge.net/projects/numpy/files/NumPy/1.6.1/numpy-1.6.1-win32-superpack-python2.7.exe/download

and last install Rabbyt for python 2.7 I had to build this myself as the site only has it up to 2.6
http://dl.dropbox.com/u/20709011/ARC/Rabbyt/Rabbyt-0.8.3.win32-py2.7.exe

I use Visual Studio with Python Tools http://pytools.codeplex.com/

alternatively you can use Eclipse and Pydev

Linux




#install python_setuptools
sudo apt-get install python_setuptools

#install pip (python package manager, not truly necessary but helpful)
sudo easy_install pip

#get python header files if we dont have them
sudo apt-get install python-dev

#install build_essential (gcc and the like)
sudo apt-get install build_essential

#install numpy
sudo pip install numpy

#install wxpython
sudo apt-get install python-wxgtk2.8

#install pyglet download source (pip tries to get the docs if you just give it the name, weird)
sudo pip install http://pyglet.googlecode.com/files/pyglet-1.1.4.zip

#install pyrex (c-extention compiler used by rabbyt)
sudo pip install pyrex

#install cython (c-extention compiler, the successor to pyrex and uses by us for building the binary verison or ARCed)
sudo pip install cython

# install OpenGL Headers
sudo apt-get install mesa-common-dev
sudo apt-get install libgl1-mesa-dev
sudo apt-get install libglu1-mesa-dev

#install rabbyt
sudo pip install rabbyt

#install pillow
sudo pip install pillow


Mac OS X



10.7 Lion and up

#install xcode
#install xcode command line tools

#install homebrew
ruby <(curl -fsSkL raw.github.com/mxcl/homebrew/go)
#add line ot .bash_profile
export PATH=/usr/local/bin:$PATH
#run brew doctor to make sure it's all working
brew doctor

#install python
brew install python --framework --universal
#add line to .bash_profie
export PATH=/usr/local/share/python:$PATH
# update the python install
cd /System/Library/Frameworks/Python.framework/Versions
sudo rm Current
ln -s /usr/local/Cellar/python/2.7.3/Frameworks/Python.framework/Versions/Current

#install wxpython
#install 2.9.x from dmg download
#add line to .bash_profile
export PYTHONPATH="/usr/local/lib/wxPython/lib/python2.7/site-packages/wx-2.9.1-osx_cocoa:/usr/local/lib/wxPython/lib/python2.7/site-packages:$PYTHONPATH"

#add system links
cd /usr/local/lib/python2.7
ln -s ../wxPython/lib/python2.7/wxhack.pth .
ln -s ../wxPython/lib/python2.7/wxhack.py .

#install numpy
sudo pip install numpy


#install pyglet download source (pip tries to get the docs if you just give it the name, weird)
sudo pip install http://pyglet.googlecode.com/files/pyglet-1.1.4.zip

#install pyrex (c-extention compiler used by rabbyt)
sudo pip install pyrex

#install cython (c-extention compiler, the successor to pyrex and uses by us for building the binary verison or ARCed)
sudo pip install cython

#install rabbyt
sudo pip install rabbyt
Title: Re: Setting up your environment to Work on the editor
Post by: ForeverZer0 on April 08, 2011, 07:11:41 pm
I already have Python 2.8 installed. I know that you are sticking with an older, more mature version of it, but it won't be an issue just to stick with it for now will it? It will be highly unlikely that something won't work in this version that didn't in 2.2.
Title: Re: Setting up your environment to Work on the editor
Post by: Ryex on April 08, 2011, 07:24:55 pm
there is not python 2.8  only 2.7
and there are things that work in 2.7 that don't in 2.6
Title: Re: Setting up your environment to Work on the editor
Post by: ForeverZer0 on April 08, 2011, 07:37:41 pm
Sorry, your right, I just checked my version of wxPython. I have 2.7 though.
I understand that things from new versions won't work in older versions in some cases, but not vice-versa for the most part (there may be an occasion). Most of these are rarely used, and it won't be difficult to code in a "2.2 compatible way".
Title: Re: Setting up your environment to Work on the editor
Post by: Ryex on April 08, 2011, 08:03:41 pm
the main reason is that some of the libraries only work with version 2.6
Title: Re: Setting up your environment to Work on the editor
Post by: nathmatt on April 08, 2011, 08:06:43 pm
yea i noticed that i have been meaning to uninstall 2.7 and install 2.6
Title: Re: Setting up your environment to Work on the editor
Post by: ForeverZer0 on April 08, 2011, 08:07:18 pm
Just noticed that when I tried to install PIL.
Switching to 2.2.6...

;)
Title: Re: Setting up your environment to Work on the editor
Post by: Ryex on April 08, 2011, 09:09:12 pm
you mean 2.6.6?
Title: Re: Setting up your environment to Work on the editor
Post by: nathmatt on April 08, 2011, 09:16:51 pm
Quote from: Ryex on April 08, 2011, 07:02:50 pm
first off install Python 2.2.6 http://www.python.org/download/releases/2.6.6/ (http://www.python.org/download/releases/2.6.6/)
Title: Re: Setting up your environment to Work on the editor
Post by: Ryex on April 08, 2011, 09:20:43 pm
well you'll notice the link lead to 2.6.6. :P sorry about that edited first post you need 2.6.6
Title: Re: Setting up your environment to Work on the editor
Post by: Blizzard on April 09, 2011, 04:02:23 am
I'm having troubles installing PIL. It keeps telling me that Python 2.6 is not installed and could not be found in the registry.
Title: Re: Setting up your environment to Work on the editor
Post by: Ryex on April 09, 2011, 04:08:30 am
there is a link somewhere on the PIL page about registering python... here it is
http://effbot.org/zone/python-register.htm (http://effbot.org/zone/python-register.htm)
but I never had problems like that. in fact the link to fix that problem is only there for versions older than 1.1.7
Title: Re: Setting up your environment to Work on the editor
Post by: Blizzard on April 09, 2011, 04:54:51 am
It's possible that this happened because I installed the 64 bit version of Python. I'm about to try again with the 32 bit version.

EDIT: Yup. That was it.
Title: Re: Setting up your environment to Work on the editor
Post by: ForeverZer0 on April 09, 2011, 01:37:49 pm
I had the same problem. I hadn't installed the 32 bit version yet, but I assumed that was likely the problem

EDIT:
Got everything, but if I attempt to debug, it tells me I am missing "numpy".
Title: Re: Setting up your environment to Work on the editor
Post by: Ryex on April 09, 2011, 04:20:18 pm
oops this true, numpy is a very important library

numpy for python 2.6 http://sourceforge.net/projects/numpy/files/NumPy/1.5.1/ (http://sourceforge.net/projects/numpy/files/NumPy/1.5.1/) the file is named "numpy-1.5.1-win32-superpack-python2.6.exe"
Title: Re: Setting up your environment to Work on the editor
Post by: ForeverZer0 on April 09, 2011, 05:02:50 pm
Working beautifully now. Sexy it is.  ;)
Title: Re: Setting up your environment to Work on the editor
Post by: G_G on April 09, 2011, 05:32:01 pm
So 32-bit install will work fine on 64-bit systems?
Title: Re: Setting up your environment to Work on the editor
Post by: ForeverZer0 on April 09, 2011, 06:03:21 pm
Yes. I used the 64 bit version first, which is preferable, but some of the libraries don't work with it. They are stored differently in the registry or something.
Title: Re: Setting up your environment to Work on the editor
Post by: Ryex on April 09, 2011, 06:35:59 pm
specifically wxpython dosn't work with 64 bit python. at least on mac
Title: Re: Setting up your environment to Work on the editor
Post by: Ryex on April 21, 2011, 02:28:26 am
update. pygame is needed now.
Title: Re: Setting up your environment to Work on the editor
Post by: Blizzard on April 21, 2011, 02:31:59 am
I noticed when I was trying out the dimming of the layers. xD
Title: Re: Setting up your environment to Work on the editor
Post by: Ryex on April 21, 2011, 02:52:03 am
Za? I only added pygame dependency just now. until my commit a few seconds ago (magical 200) all pygame stuff was commented out...
Title: Re: Setting up your environment to Work on the editor
Post by: Blizzard on April 21, 2011, 02:59:08 am
When I tried running wxgltext.py the other day, I got some error messages of not being able to import some pygame stuff so I figured that I had to install it. Once I did, everything worked fine. True, it took a while until it started (probably Python byte code compilation), but the point is that it worked after I installed pygame.
Title: Re: Setting up your environment to Work on the editor
Post by: Ryex on April 21, 2011, 03:02:43 am
I just realized that the RPGutil.py in the test/code/rmxp folder had be copied out before I commented out all the pygame stuff. I fixed that yesterday :P
but I added the code back in to the cache.py file as I needed the hue rotation. I actually have no Idea if it works though...

*goes to try a test*
Title: Re: Setting up your environment to Work on the editor
Post by: Blizzard on April 21, 2011, 03:07:47 am
Will we be able to make an release build without pygame dependency?
Title: Re: Setting up your environment to Work on the editor
Post by: Ryex on April 21, 2011, 03:35:23 am
we could If I can figure out how to do the hue rotation using PIL instead of pygame.

but then we would be stuck using pyglet to play sounds which need the user to have AVbin to play files other than wav

Pygame doesn't require AVbin but it can only really play ogg and wav. mp3 can crash if the format isn't exactly right

so really, I would say if we can we should but I can't really figure out how to do the same hue rotation in PIL at the moment. the second I do I'll remove Pygame completely
Title: Re: Setting up your environment to Work on the editor
Post by: Blizzard on April 21, 2011, 04:11:39 am
If there is no way to do it automatically, we will simply implement it manually.
Title: Re: Setting up your environment to Work on the editor
Post by: Ryex on April 21, 2011, 04:48:54 am
oh yes I know, I'm working on it now.
Title: Re: Setting up your environment to Work on the editor
Post by: Blizzard on April 21, 2011, 05:55:16 am
Alright. Best you check out how it's done, i.e. how the coloring system with hue-saturation-brightness works.
Title: Re: Setting up your environment to Work on the editor
Post by: Ryex on April 21, 2011, 05:55:59 am
Ok I've implemented the change hue function in PIL the function is close to 3 time slower than pygame though (0.308 seconds rather than 0.167 seconds) but at the same time we don't need to distribute with  3.5 MB library that we are only using for one ten line piece of code. :/

EDIT: assembla is having server problems ... :( it's committed now, yay!


EDIT2:
ok this is freaking epic I just managed to DRASTICALLY reduce the amount of time needed to rotate the hue of an image with PIL and woudn't you know it it take twenty times a much code... oh and numpy :P.
Spoiler: ShowHide
(http://img148.imageshack.us/img148/931/inprovment.png)

the times on the left are from my new PIL function, the right the old loop and set each pixel separately PIL function and as you can see its even faster than the pygame version.

how awesome am I? :P
Title: Re: Setting up your environment to Work on the editor
Post by: Blizzard on April 25, 2011, 01:21:08 pm
Sometimes code can be optimized heavily if you separate some specific cases and process them in a more optimized manner rather than doing all of it in a general way so you end up with more code. If that makes any sense. xD
Title: Re: Setting up your environment to Work on the editor
Post by: Ryex on April 25, 2011, 05:47:45 pm
actually I did it by doing the color calculation manually. before I was usieng the ability of the pyhton colorsys module to do the RGB to HSV calculation which meant I could only do pixel at a time which meant I had to do a loop through every pixel.

with the exspanded version I did the calculation my self and uses numpy to to the operation element wise on the c level. much faster that way.
Title: Re: Setting up your environment to Work on the editor
Post by: ForeverZer0 on April 25, 2011, 05:50:19 pm
Using a script to iterate pixels is always slow. That is why I didn't bother trying to implement a Ruby version for transitions when I wrote that custom resolution script.
Title: Re: Setting up your environment to Work on the editor
Post by: Ryex on October 13, 2011, 07:49:17 pm
On F0's Discovery of http://pytools.codeplex.com/
I have abandoned Eclipse and created a Solution in visual studio using python tools.

it WAY better.
Title: Re: Setting up your environment to Work on the editor
Post by: ForeverZer0 on October 13, 2011, 08:56:48 pm
Quote from: Ryex on October 13, 2011, 07:49:17 pm
On F0's Discovery of http://pytools.codeplex.com/
I have abandoned Eclipse and created a Solution in visual studio using python tools.

it WAY better.


I was ecstatic after finding it last night, lol.
The improvement is huge, and it feels good to be back in the IDE I am familiar with.
Title: Re: Setting up your environment to Work on the editor
Post by: Blizzard on October 14, 2011, 02:18:01 am
I'll remember this. I never liked Eclipse.
Title: Re: Setting up your environment to Work on the editor
Post by: ForeverZer0 on October 23, 2011, 09:57:55 pm
I've decided to use matplotlib for generating and setting curves for parameters. I'm sure we can find some other neat uses for it as well. The license is based on the same one that Python uses, so thats good, too.

I've updated the main post, but here's the link to save you time:

http://sourceforge.net/projects/matplotlib/files/matplotlib/matplotlib-1.1.0/

Download the file:  "matplotlib-1.1.0.win32-py2.6.exe"
Title: Re: Setting up your environment to Work on the editor
Post by: Ryex on October 23, 2011, 10:03:38 pm
Sounds good.
Title: Re: Setting up your environment to Work on the editor
Post by: Ryex on October 25, 2011, 06:52:56 pm
In VS Tools>Options>Python>Interperter Options there is a button called "generate completion database" This will get auto-completion working properly in Python Tools
Title: Re: Setting up your environment to Work on the editor
Post by: Ryex on November 03, 2011, 10:23:22 pm
I'm moving to python 2.7 see first post for details
Title: Re: Setting up your environment to Work on the editor
Post by: ForeverZer0 on November 03, 2011, 10:33:46 pm
Thank you!

I was wanting to use ordered dictionaries just yesterday, and saw they weren't implemented until 2.7. It really wasn't worth creating my own, but now I won't have to :)
Title: Re: Setting up your environment to Work on the editor
Post by: Ryex on November 04, 2011, 12:17:50 am
the sire for Rabbyt only has the  installers for python up to 2.6 I built it myself http://dl.dropbox.com/u/20709011/ARC/Rabbyt/Rabbyt-0.8.3.win32-py2.7.exe tell me if it doesn't work

I've tested everything we have so far and it works in 2.7 so were good to go.
Title: Re: Setting up your environment to Work on the editor
Post by: ForeverZer0 on November 04, 2011, 12:39:30 am
Actually, I am having numerous problems with this. Pyglet wouldn't install, to 2.7, but I managed that by installing it from source. I set the project to use 2.7, and also VS's settings to use 2.7, and I have some strange bugs. First and foremost, the app won't start.  No how, no way, and with no error. I run it, the console pops up for a second, then it exits. It doesn't matter if I put a 'print "hello"" at the top of __main__, it doesn't display, and nothing happens. I CAN get a single file that is just a panel with snippet at the end to add it to a frame and run a simple app, but ONLY if I right-click and choose "Start without Debugging". If I choose to debug, I get the same problem. I can't even set it as the startup file and press F5.

Now, I didn't uninstall 2.6 beforehand, so that may be an issue. I already edited the environment variables to 2.7, and if I simply type "python" into cmd.exe, it starts up 2.7, so that isn't the issue, although I yet checked PYTHONPATH. I may just uninstall both 2.6 and 2.7, then reinstall only 2.7, and all the packages again.

That's gonna be for tomorrow, though, I'm tired and going to bed. :P
Title: Re: Setting up your environment to Work on the editor
Post by: Ryex on November 04, 2011, 12:46:16 am
ya I had to install pyglet form source too.. hmm well I HAVE made some changes to the source so i'll commit them and we'll see what happens.

some errors that python tools caches are harmless and are actually part of the normal start up of the libraries.

I didn't uninstall 2.6 before hand either but my stuff worked with no problems, did you set 2.7 as the new default interpreter in python tools?
Title: Re: Setting up your environment to Work on the editor
Post by: ForeverZer0 on November 04, 2011, 09:29:29 am
Yeah, everything in VS is set to use 2.7.
Title: Re: Setting up your environment to Work on the editor
Post by: ForeverZer0 on November 04, 2011, 05:32:09 pm
Alright, I don't know WTF is going on with this. I did the following:



Other possible factors, though I doubt they are of any relevance:


Here is something strange that I imagine has something to with it, though I am unsure how to really fix it. When I choose "Python 2.7 Interactive" from VS's tools menu:

Python interactive window.  Type $help for a list of commands.
Failed to launch REPL process
Traceback (most recent call last):
  File "C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\1.1\visualstudio_py_repl.py", line 1050, in <module>
    _run_repl()
  File "C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\1.1\visualstudio_py_repl.py", line 1036, in _run_repl
    BACKEND.connect(int(options.port))
  File "C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\1.1\visualstudio_py_repl.py", line 474, in connect
    ReplBackend.connect(self, port)
  File "C:\Program Files (x86)\Microsoft Visual Studio 10.0\Common7\IDE\Extensions\Microsoft\Python Tools for Visual Studio\1.1\visualstudio_py_repl.py", line 101, in connect
    self.conn.connect(('127.0.0.1', port))
  File "C:\Python27\lib\socket.py", line 224, in meth
    return getattr(self._sock,name)(*args)
socket.error: [Errno 10013] An attempt was made to access a socket in a way forbidden by its access permissions
>>>
Title: Re: Setting up your environment to Work on the editor
Post by: Ryex on November 04, 2011, 06:02:23 pm
we don't need pygame anymore. but this is really weird. my setup works fine... one thing to note I installed python well before I installed pytools
Title: Re: Setting up your environment to Work on the editor
Post by: ForeverZer0 on November 04, 2011, 06:08:06 pm
I figured it out... although I'm not sure why it should keep the application from opening. I noticed in the trace that it was having a socket error. I'm using NOD32 firewall, set to interactive mode, meaning every time something tries to connect, it shows me a dialog where I can create a rule to permanently allow/block, temporarily allow/block, or once only allow/block.  It wasn't popping up, so I restarted my PC, tries again, and it showed the dialog so I could create a rule. Since that, everything seems to work good.
Title: Re: Setting up your environment to Work on the editor
Post by: ForeverZer0 on November 21, 2011, 03:31:29 pm
Updated the topic, namely the addition of Scipy, and the demise of matplotlib. I discovered wx has native support for graph drawing, that although is not as feature rich as matplotlib, is simpler to use and more than enough for our needs.
Title: Re: Setting up your environment to Work on the editor
Post by: Ryex on January 12, 2013, 11:48:11 pm
updated 1st post with info for setting up on Lynix
Title: Re: Setting up your environment to Work on the editor
Post by: Ryex on January 15, 2013, 11:45:12 am
and now we have set up for OS X 10.7 and up.

my god I hate Mac for making this so dam complicated seriously I HAVE OT FREAKING REBUILD PYTHON AND THEN LINK IT IT! oh and because of that the python paths aren't standard so I have to link Wxpython back to the new python to get THAT to work

oh, and in case you didn't know pythonw is a completely different animal on OSX, because Mac is stupid and normal python can create windows apps because it isn't a framework that had to make a framework shim that starts up a python process inside it. what does that mean? I have to set up the paths correctly just so this special shim can find the python standard library.

seriously, why does it seem like while OS X is extremely nice and friendly to the average user if you want to ANYTHING complicated it makes your life hell?
did you know that if you internet connection breaks in the middle of the appstore downloading something it cant just pause the download? that it will instead have 1 hash check fail and scrap everything? do you have any idea how difficult this makes downloading 3 GB files like Xcode?

Oh, and I ended up having to compile GCC 4.2 to build python with which in retrospect I didn't even need to because apple only recently fixed their shit but still, HOURS of my life wasted on this bull.
Title: Re: Setting up your environment to Work on the editor
Post by: Blizzard on January 15, 2013, 05:30:43 pm
Welcome to Mac development. Apple hopes that you had a nice cactus stuck up your ass on your way here.
Title: Re: Setting up your environment to Work on the editor
Post by: Ryex on January 16, 2013, 08:01:11 pm
Turns out I forgot to list somethings for Linux

namely ensuring you have python headers and opengl headers

and I ended up having to use pyglet 1.2alpha1 for Mac 10.7 as well Mac 10.7 only supports 64 bit and pyglet 1.1.4 has som major issues with 64bit that were solved in 1.2
Mac's version of PIL also has some weird ishues with PIL because of PIL's dual usage of

import Image

and

from PIL import Image

being the same module but Mac python can't tell for some reason and loads the binnary twice. if both commands are used in two different places you end up with a hash collision error that crashes python outright so I had to go into Pyglet and get rind of the try statement that uses import Image in an attempt to be backwards compatible.
I suppose I could also just use Pillow instead of PIL (it's a actively maintained drop in replacement for PIL as PIl was last updated in 2009) as it only supports the from PIL import Image format and the dual import would of been avoided.