Using certain technology for the wrong purposes

Started by Blizzard, March 20, 2010, 03:51:21 pm

Previous topic - Next topic

Blizzard

March 20, 2010, 03:51:21 pm Last Edit: March 20, 2010, 03:56:56 pm by Blizzard
I just had a conversation about how using certain technology for the wrong purposes is bad. This might sound vague and abstract, but it really isn't. Imagine that, as a programmer, you make a 3D renderer in Java that you intend to use. That's just plain stupid. Java is a high level programming language intended for cross-platform application development. Sure, you can use a C++ written renderer with Java to make a game and that's actually not such a bad idea, but making the renderer itself in Java is just stupid. An analogy would be using a hammer to drill a hole. You just don't do that. This is pretty much the summary of the chat log below where I criticize the usage of MySQL (a database engine) for dynamic data. The name of the other person in that chatlog has been removed to protect his/her privacy.

Spoiler: ShowHide
Quote[20:43] <***> oo... I also made a few changes to the engine to register users themselves as "online" in the SQL, haven't tested it but...
[20:44] <***> It'd work better than the log, in my opinion
[20:44] <@Blizzard> what log?
[20:45] <@Blizzard> oh, that you mean
[20:45] <***> that thing u made
[20:45] <***> xd
[20:45] <@Blizzard> well
[20:45] <***> that counts the clients connected
[20:45] <@Blizzard> putting online status in an SQL database is just plain stupid, you do realize that?
[20:45] <***> many games do that. But it's so that I can easily say... "These people are online" from a webpage
[20:46] <***> =)
Session Start (www.centralchat.net:#chaosproject): Sat Mar 20 20:48:05 2010 0100
[20:48] *** Blizzard has joined #chaosproject
[20:48] *** Topic on #chaosproject is: #chaosproject Soul Silver <3
[20:48] *** Topic set by NAMKCOR (12.03.2010 at 06:54)
[20:48] *** CC-Bot sets mode +qo Blizzard Blizzard
[20:48] <***> welcome back
[20:50] <@Blizzard> thx
[20:53] <***> right so...
[20:53] <***> I only added the "online" thing, to tell ppl on a webpage "Who, and how many" are online
[20:53] <***> even if it glitches, and forgets to change "online" to "offline"
[20:54] <***> its stil a good guess
[20:54] <@Blizzard> still
[20:54] <@Blizzard> online status is dynamic data
[20:54] <@Blizzard> and should not be in a storage data container like SQL
[20:54] <***> But that dynamic data cant be accessed in PHP
[20:55] <***> xd
[20:55] <@Blizzard> it can
[20:55] <***> not unless u use a ruby script
[20:55] <***> with ur php
[20:55] <@Blizzard> jc made a simple PHP script that easily accesses the dumped online list and displays who's online on his website
[20:55] <***> oh?
[20:56] <@Blizzard> see? (Link: http://pkmnneon.com/)http://pkmnneon.com/
[20:56] <@Blizzard> I actually made the online user plugin so he could make that PHP script to get the online data
[20:57] <***> oh i see the game info box
[20:57] <***> nice
[20:58] <***> I'll have to take another look
[20:58] <***> at the server code. xd
[20:59] <***> But no, the main reason  is cuz i dont have the game running off the computer with the website
[20:59] <***> that shows the status
[20:59] <@Blizzard> still not a real problem
[20:59] <@Blizzard> make a php request to the server
[20:59] <***> Well for me, its easier to connect to the SQL on a seperate computer
[20:59] <@Blizzard> html*
[20:59] <@Blizzard> http* #_#
[20:59] <***> xd
[20:59] <@Blizzard> and let the html on the other side simply send you the online list
[21:00] <***> hmm..
[21:00] <***> I could do that...
[21:00] <***> I still like putting everything through SQL
[21:00] <***> xd
[21:01] <***> its my uhh... preferred language.
[21:01] <***> xd
[21:02] *** Aqua has joined #chaosproject
[21:02] *** CC-Bot sets mode +o Aqua
[21:02] <@Aqua> MALE CHICKENS ARE COCKS!!!
[21:03] <***> no really?
[21:03] <@Aqua> AND I LOVE EATING CHIKCEN
[21:03] <@Aqua> HENCE...
[21:03] <@Aqua> :x
[21:03] <***> lol
[21:03] <@Blizzard> NOWAI
[21:03] <@Blizzard> @***: SQL is a bad choice for that kind of data
[21:03] <@Blizzard> a very bad one
[21:03] <@Blizzard> SQL access is very CPU expensive


From here the actual criticism begins.

Spoiler: ShowHide
Quote[21:06] <***> tell that to the guy that has a whole site stored in SQL. ;)
[21:06] <@Blizzard> omg
[21:06] <@Blizzard> no, you didn't do that
[21:06] <@Blizzard> *heavy facepalm*
[21:06] <***> yes, yes i did
[21:06] <***> ^^
[21:06] <@Blizzard> how much coding experience did you say, you have?
[21:06] <***> lawl
[21:07] <***> Lets see... 12 years
[21:07] <***> xd
[21:07] <@Blizzard> no offense, but that's wasted 12 years if you think putting everything into SQL is a good thing / idea
[21:07] <***> xd
[21:08] <@Blizzard> I have less than 5 years and even I know that's a bad thing to do :/
[21:08] <***> lawl
[21:08] <***> I wanted to give it a try
[21:08] <***> and i made it so if it doesn't work i can still move it to the PHP files
[21:08] <***> xd
[21:08] <@Blizzard> that's what the makers of Matrix 2 and 3 said as well :V
[21:08] <***> all i know is that on my servers...
[21:08] <@Blizzard> BTW, :V is sarcasm xD
[21:09] <***> This engine works fine
[21:09] <***> xd
[21:09] <@Blizzard> Netplay+ works fine as well
[21:09] <@Blizzard> until people started complaining that it crashes and that it's slow
[21:09] <***> heh
[21:09] <***> xd
[21:09] <@Blizzard> and lags like hell
[21:10] <***> I'll test it on bad computers... But thats for a later date
[21:10] <***> right now i want to finish my engine
[21:10] <***> Xd
[21:10] <@Blizzard> in any case
[21:10] <***> (its also the reason i learned how to shorten a 4000 line code into 800 lines)
[21:10] <***> xd
[21:10] <@Blizzard> using one single technology for everything is a sign of insufficient experience
[21:10] <@Blizzard> like one of my professors once said
[21:11] <@Blizzard> "an inexperienced programmer thinks that there is an algorithm that will solve everything"
[21:11] <***> I have no qualms with that saying
[21:11] <@Blizzard> "an experienced programmer knows that there is no such thing"
[21:11] <@Blizzard> it's not about algorithms
[21:11] <@Blizzard> it's generally about using one thing to do everything
[21:11] <@Blizzard> one tool for everything you do
[21:12] <@Blizzard> you wouldn't drill a hole with a hammer, would you?
[21:12] <@Blizzard> just because they are both hammer and drill are tools used in architectures?
[21:12] <***> No
[21:12] <***> but I still like to use PHP and mySQL for a website
[21:12] <***> xd
[21:13] <@Blizzard> PHP and SQL is fine
[21:13] <@Blizzard> but not if you stuff everything in the SQL database
[21:13] <@Blizzard> especially not dynamic data
[21:13] <***> lawl
[21:13] <@Blizzard> sessions are pretty much the only exception that's reasonable
[21:13] <***> naw...
[21:14] <***> the only things that are in the SQL database are the data... and the PHP files that'd normally be PHP files...
[21:14] <***> ;)
[21:14] <@Blizzard> putting any other dynamic data into SQL database is like...
[21:14] <@Blizzard> IDK, using a hammer to drill holes
[21:14] <***> hehehe
[21:14] <@Blizzard> again, no offense
[21:14] <@Blizzard> but I just love it when somebody comes along and is all like
[21:15] <@Blizzard> "I made this and that using just this single technology"
[21:15] <***> it sounded like a fun project. I know why people dont do it
[21:15] <***> i just wanted to try
[21:15] <@Blizzard> and I'm like "now that you have fed your ego because of your small penis, I hope you won't seriously consider using this in practice"
[21:15] <@Blizzard> try is ok, but
[21:15] <***> xd
[21:15] <@Blizzard> still
[21:16] <@Blizzard> xD
[21:16] <***> i do know people that would actually use it... Not that I plan on giving my engine out... I just like doing things no1 would think of
[21:17] <@Blizzard> just because you can use something or do something doesn't mean it's a good idea
[21:17] <@Blizzard> I'm not saying it can't be used
[21:17] <@Blizzard> I'm saying that it's a bad idea to use it
[21:17] <@Blizzard> just keep dynamic data out of SQL, ok?
[21:17] <***> Hasn't done much to me yet
[21:17] <@Blizzard> YET
[21:18] <***> Im tellin you... xd
[21:18] <@Blizzard> do whatever you like
[21:18] <@Blizzard> it's your time that you're wasting
[21:18] <@Blizzard> just as I had to revamp Blizz-ABS 2 times
[21:18] <***> I made it work 2 ways so its fine
[21:18] <***> xd
[21:19] <***> If the SQL doesn't get the reviews i want... it can still go to PHP, with the same engine
[21:19] <***> xd
[21:19] <@Blizzard> just keep in mind that people will prefer good software over yours
[21:20] <***> Its not like u dont already do things like my engine does
[21:20] <***> Imaging having a forum... and every post is like...
[21:21] <@Aqua> what engine are you making?
[21:21] <***> 10000 characters
[21:21] <***> or more
[21:21] <***> the SQL loads that quite fast
[21:21] <@Blizzard> because that's static data
[21:21] <@Blizzard> you don't change that data every few minutes
[21:21] <@Blizzard> most post don't ever change
[21:21] <***> right
[21:22] <***> And i have
[21:22] <@Blizzard> do you actually know how SQL works?
[21:22] <***> PHP files copied into the SQL
[21:22] <@Blizzard> do you know what concurrent data access is?
[21:22] <***> you dont edit that stuff
[21:22] <***> that often either
[21:22] <@Blizzard> table locks, database locks, attribute locks, entity locks?
[21:23] <@Blizzard> technically
[21:23] <@Blizzard> if you have php files in the SQL database
[21:23] <@Blizzard> if 100 clients would try to load your side
[21:23] <@Blizzard> one client will get the data immediately
[21:23] <@Blizzard> the next one after that
[21:23] <@Blizzard> another one after that
[21:24] <@Blizzard> the last client would have to wait quite some time until he gets the page to actually show up
[21:24] <@Blizzard> there are delays between locks and unlocks of data
[21:24] <@Blizzard> and SQL data often isn't cached
[21:24] <@Blizzard> and on top of all that
[21:24] <@Blizzard> it's completely unecessary
[21:24] <@Blizzard> there is no need for such concurrent access control for a file that never changes
[21:25] <@Blizzard> but you don't have the knowledge yet to configure the database so it works fine
[21:25] <@Blizzard> or do you know how database systems actually work?
[21:25] <***> i know
[21:26] <@Blizzard> I can't say how much I learned in college
[21:26] <@Blizzard> but I definitely learned to NOT do a specific amount of stupid things and mistakes
[21:26] <@Blizzard> mixing concepts and using inappropriate technologies was one of them
[21:27] <@Blizzard> I mean, you wouldn't use an SQL database to make 3D rendered game, would you?
[21:27] <@Blizzard> even though it's technically actually possible
[21:27] <***> xd
[21:28] <***> havent tried that one before... But no
[21:28] <***> I wouldnt do that
[21:28] <***> that'd kill a computer
[21:28] <***> xd
[21:28] <@Blizzard> see my point?
[21:28] <@Blizzard> you just have to learn that being a bit off with using techology and being a lot off with using technology is the same
[21:28] <@Blizzard> it's both being off using technology
[21:29] <@Blizzard> how much it's off will mostly affect the performance you get
[21:29] <@Blizzard> I've seen 3D software renderers made in Java
[21:29] <@Blizzard> that generally being a very stupid idea
[21:29] <@Blizzard> it served its purpose
[21:30] <@Blizzard> but it was still a completely stupid idea
[21:30] <@Blizzard> I mean, it took SECOND to render just one frame
[21:30] <@Blizzard> SECONDS*
[21:30] <***> i have a good reason to start off with everything in the mySQL
[21:30] <@Blizzard> static data, yes
[21:30] <@Blizzard> storage data, yes
[21:30] <@Blizzard> DYNAMIC DATA?!
[21:30] <@Blizzard> god forbid, NO
[21:31] <***> and im sorry i am taking a while to respond
[21:31] <***> i am playing a gmae
[21:31] <***> xd
[21:31] <@Blizzard> lol
[21:31] <@Blizzard> meh, I need to get going anyway
[21:31] <***> ah
[21:31] <***> kk
[21:31] <@Blizzard> since I see there's no point in telling you anything
[21:31] <***> lawl
[21:31] <***> xd
[21:32] <***> i keep at it
[21:31] <***> when i get fixed on an idea
[21:32] <***> ;)
[21:32] <@Blizzard> another bad trait of an inexperienced programmer :P
[21:32] <@Blizzard> I'm out, cya


People simply don't understand the complexity that's behind some of the technologies and that makes those technologies unsuited for certain usages. SQL is one of the complex technologies that are very simple to use. i.e. Did you know that during a simple UPDATE SQL query actually a lot of data is written? There are change logs that are maintained so first there is an entry added in the log for "undo", then the actual data is added and finally an entry in the "redo" log is added. Now, if at any point the system crashes, using undo and redo logs it's possible to reconstruct the data that was lost during the crash.
Between that there is lots of resource locking and unlocking involved to disallow concurrent data access. A simple example why this is done is shown below.

- 2 queries are executed (each has its own local variable for x; y is data on a common location accessible and shared by both queries
Q1: read into x from location y
Q2: read into x from location y
Q1: x = x + 10
Q2: x = x + 20
Q1: write x to location y
Q2: write x to location y


What happened? The result of Q1 was overwritten as if it never happened. And this is just one way of how things can go wrong if there is no proper handling of concurrent data access. Obviously this kind of data access requires a system to handle. Since exclusive resource locking of, let's say, the entire database defeats the complete purpose of concurrent access. So a solution in between is used. This is a topic that I will not discuss here any further as it's too broad (there are books about concurrent data access!).

The point of this is that you shouldn't use SQL to store dynamic data (obviously data that changes often during runtime) or trivially simple data that doesn't belong into a database. It is a mistake done in Netplay+ (the online player list was stored into the database) and avoided in RMX-OS.
Check out Daygames and our games:

King of Booze 2      King of Booze: Never Ever
Drinking Game for Android      Never have I ever for Android
Drinking Game for iOS      Never have I ever for iOS


Quote from: winkioI do not speak to bricks, either as individuals or in wall form.

Quote from: Barney StinsonWhen I get sad, I stop being sad and be awesome instead. True story.

winkio

Agree with you 100%.  There's a large difference between being pioneering and creative, and being an idiot.

Trainer Zydragon


Fantasist

This is one thing I learnt from you over the years, mate, to use the right tools for the right thing. I was actually excited when I saw a 3D engine built on RMXP back then.

Quote from: Arceus on March 20, 2010, 04:45:35 pm
Agree with you 100%.  There's a large difference between being pioneering and creative, and being an idiot.


That's true, but we can only hope something good might come off such endeavors, even if it's one in a million.
...
Wait, I withdraw that statement. 1 in a million is just not worth it.
Do you like ambient/electronic music? Then you should promote a talented artist! Help out here. (I'm serious. Just listen to his work at least!)


The best of freeware reviews: Gizmo's Freeware Reviews




fugibo

Not as much using a hammer to drill a hole as using a toy drill to drill a hole