Extensive Evaluation of Dolphin 7.x.x JavaScript

DosDawg posted 12th of September 2010 in Community Voice. 15 comments.

Evaluation of JavaScript on Dolphin 7.x.x

 

In my attempt at searching for a better performance level, I have been very busy attempting to understand how to get the best performance out of the server as well as the application. Knowing we had to extensively modify our server configuration in order to accommodate the Dolphin 7.x.x environment, we have started looking at other ways to squeeze performance out of this application.

I started looking at the javascript, reason being, is basically every analytical evaluation tool, complains about javascript. I know that dolphin is rather heavy with javascript, some just randomly tossed into the files.

What I am evaluating and where I am getting my data is from the following resources:

  • JSMin is a conservative compressor, written several years ago by Douglas Crockford. It is considered safe (especially if you verify your code with JSLint first-- an excellent thing to do anyway) because it doesn't attempt to change any variable names.
  • Dojo shrinksafe is a very popular Java based JavaScript compressor that parses the JavaScript using the rhino library and crunches local variable names.
  • Packer (Version 3.1) by Dean Edwards, is also a very popular JavaScript compressor, that can go beyond regular compression and also add advanced on-the-fly decompression with a JavaScript runtime piece.
  • the YUI Compressor (Version 2.4.2) is a newer compressor written by Julien Lecomte, that aims to combine the safety of JSMin with the higher compression levels acheived by Dojo Shrinksafe. Like Dojo shrinksafe, it is written in Java and based on the rhino library.
  • Also writings from Crockford author of JSMin http://javascript.crockford.com/code.html Code Conventions of the JavaScript Programming Language.

Ok, so yes I am new to all of this, and trust the resources on the internet in regards to javascript and compression. So here is where I start, and this could prove to be elongated. Andrew Boon, this is not an attempt to thwart the application, this is something that needs to be done, and should have been done, and in order for optimal performance, we are looking for resolution to such issues as javascript compression out of the box on future releases.

 

I have compiled a list of errors or tweaks that need to be looked at on this next release. Most of the issues are simple to fix, missing semicolons, removing implied globals, removing the (eval) instances, changing some directives on syntax. removing variables that are not being used. Just a general clean up of the JavaScript, so we can start to implement a compression agent for the dolphin environment.

I have had to put this PDF file in a zip format, because i was unable to get it to read from FF.

 

JavaScript Eval by Terabyte Hosting Solutions

 

Guys, on the jslint site there are multiple options that can be selected, my evaluation was ran from the viewpoint that we need to assess exactly what we are dealing with. Please feel free to run your own evaluation on these files. Understanding this is not a complete list of all of the javascript files that are part of the Dolphin 7.x.x environment. We really need all of the javascript to be looked at. My next adventure is to evaluate the CSS, and see how this can be reduced.

 

Guys, the other large item that needs to be addressed, is that javascript should not be top-loaded, can we please get this addressed so that our sites are loading optimally.

As you know, there were suggestions provided in regards to server configuration. This same server configuration was never mentioned from the onset. Server configs were working perfectly, safely and securely with Dolphin 6.x.x, so if there were known server config changes that were required, these should have been part of the documentation.

 

With that being said, Terabyte Hosting Solutions has been making those adjustments, and there has been some improvement in server CPU load, but what we see is that the load switched from CPU to RAM. So now, this can become a problem, as there is no known notation on what RAM requirements we should have for the Dolphin 7.x.x?

On a non-busy (non-productive) site, 2GB of RAM does seem to suffice, but what happens when an account actually does produce some traffic?

 

AlexT, would be much appreciated if you would look into the syntax errors produced from jslint, and see what can be done to get our javascript files up to the coding convention standards.

 

Granted, we are not looking for a perfect set of files, but I have compiled 30 pages of syntax errors out of the limited number of javascript files that i have evaluated.

 

Thanks,

DosDawg

 

 

 
Comments
·Oldest
·Top
Please login to post a comment.
AlexT
Thank you for your investigation.

We have tried to implement js gzip + js minifying in Dolphin 7.0.3, but have implemented js gzip only.
Because of the following:

1) js syntax accept code lines ending with semicolon (;) and without it, but minifying scripts require that all lines are ended with semicolon. Dolphin has a lot of js files with both variants - this is a lot of code to change and we postpone it for the future versions.

2) there are already some minified scripts (like jQuery) see more in Dolphin and it can be some problems with minyfying one file where non-minified and minified js scrips are mixed. There is no solution for it right now, after 1) is resolved - I hope some solution will be found.

Now js gzip is implemented and it saves more bandwidth than js minifying. Of cause it is better if both variants are implemented.
DosDawg
AlexT,
if you looked at the source code issues that were pointed out from the evaluation, this amounts to poor and lazy coding. Why would you not use one coding convention, so that you are not creating these performance problems? This was not just about the semi-colon (;), its about much more. There are non-declared variables, against convention. There are variables that are used before they are declared, against coding convention, there is syntax that is just incorrect.

Yes it is a lot of work, see more and just by you saying that, you should realize that selling this product for the amount that it is sold for, is totally ludicrous. The code for the most part can be cleaned up, one coding convention should be used, strict adherence to the coding convention standard should be followed and enforced. Yes there is a lot of javascript within the dolphin application, and that is even more a reason that it should be cleaned and optimized.

The ability to compress the jscript is important for performance, not just bandwidth. JS gzip may as you state save bandwidth, js-minify, increases overall performance, and that is what is being sought after here is a performance increase.

The solution AlexT, is code convention standards, that would resolve the issue with the javascript. Yes its a lot of work, work that should have been done from the onset, and not released and sold as a functional product.

I will point out one issue that was annoying to me, the jscript for IE 5.5 and 6, now honestly, why is that source code still being used, do you know of one soul who is using Dolphin 7 and IE 5.5 or IE 6, doubtful.

at any rate Alex, time-consuming or not, these issues need to be fixed, else, the product should not have been released to the public if its not ready for public consumption.

Regards,
DosDawg
DosDawg
one other thing to note here, is there a valid reason why this post is not showing up on the blogs on IE?
umeshtmi1
@ AlexT

I would like to request you to consider point 1 for this release only. I can see that it is a lot of work, but if done then it will generate a lot of happy faces. At least to me, high load time is the single biggest problem right now.

For all other businesses: I am currently on a dedicated server. I moved on to this server thinking that I will have a faster performing site. We did a lot of tweaking on the server. There was some improvement in the performance but not to the extent I see more invested into it, and expected. From my experience I would like to suggest all businesses, who wants to consider dedicate server for better performance that, if you have money invest it in working on the script for yourself than on the server first.

Can anybody post the date of the next release.
AlexT
This problem has nothing related to the server performance, except saving some bandwidth, but js gzip can did this.
We will implement this as soon as more critical things will be done.
DosDawg
i would tend to consider that malformed and useless code to be a deprivation on server performance. and this is critical, should have been done before the release. I am also posting in the next few days my evaluation of the CSS. specifically issues like mispelled directives on the file cmts.css where there is a directive calling for lovercase now, where things may function that way down under, there is no such name as l o v e r c a s e when considering css syntax.

again, you may not consider this see more to be a valid concern, but if this specific instance of something in style is not working, and clearly this could never have worked, because there is no l o v e r c a s e, then why even have it in the files? there are several instances of incorrect syntax on the style sheets and those will be posted in the next few days. yes this is critical, its critical because it is clear that these files were never looked at, and they should be. if as the developing entity, you are releasing something at a cost to the general public, then by all means this should have been gone over, and we as the end-user should not be finding these types of mistakes?

FYI ---- everything readable says this is a performance hit, and not just a bandwidth hit. malformed syntax, and top loaded javascript are a performance hit. so this is critical.

Regards,
DosDawg
AlexT
@DosDawg

You don't understand basic things about performance and code syntax.
DosDawg
oh but i do, and when an index page on a non-used dolphin site spits out 158 database queries, i know enough to know that you would never get performance out of that if you had a decent sized site. so if i as one person calling the index.php from the about_us page, generates 158 queries on the database, well enough said about that one, you can do the math i suppose.
Andrew Boon
JS syntax is not malformed. Some of the minifiers require certain approach, which doesn't necessarily mean it's the only right way. Please, don't blow it out of proportion. Cleaning and unifying JSs is a good thing of course, but in this particular case performance gains would be negligible, so we have to take other priorities.

For example, when Apple released OS X Leopard, it had GIGABYTES of unused code, and still they didn't consider the issue critical, and released the OS with great success. see more Later they came up with Snow Leopard and cleaned it for the most part. It's not an issue of poor/lazy coding - it's an issue of software that's in development for almost 10 years, and by hundreds of people.

We have made a lot of tests in the last few weeks, and realised that cleaning code would only give us 5-50% performance increases depending on files. Proper server configurations, however, give 500% or more improvement.

Now, we created a ticket, admitted that there's a room for improvement and planned it out for when we can work on this. Why would you keep persisting and making it sound like there's a cholera decease packed with Dolphin? You found misspellings and unused code, great, why don't you contribute your specific findings and fixes?


TO EVERYONE WHO'S READING THIS:

Calm down, please. This is NOT a critical issue. This issue does NOT significantly hinder Dolphin performance. Dolphin 7.0.3 is coming soon with great performance improvements and 7.1 will be an even better step. Following our server configuration guidelines would give you 10-20 time higher performance boost compared to even super-clean JSs. Choose good hosting and pursue proper configuration and your Dolphin site will fly. We have recently examined an active site working on a properly configured SHARED (cheap) hosting with 50,000+ active members working great.
DosDawg
@Andrew,
sir did you bother to read what was attached in that file. sir that is malformed incorrect javascript syntax. not according to me, according to the conventions of syntax for javascript. i am contributing by providing the results to you so you can have those who lazily coded this application, never verifying their syntax, never testing what they had written, and released it to the public. i have contributed, the attachment will reveal as much.
dolphinbugfixing
I agree with using js gzip is just saving bandwidth, internet users with slower internet connection would benefit this.

@Andrew and @AlexT
Correct me if I'm wrong I think the biggest factor with regards to performance is the number of online logged in members at any given time, just for example if there are 100 online members thats equivalent to 100 mysql connection every 5secs is that right?
CodeSatori
@All: Javascript wouldn't cause server performance issues on its own of course, any more than any other static files served out by Apache. (Though it can cause serious browser performance issues.) If the javascript dynamically formed and served out, or if it makes further server calls as it runs, then of course it could cause undue server loads, if not properly optimized.


@DolphinBugFixing: The amount of MySQL connections isn't as critical as the amount of MySQL queries. The XML.php file discussed see more earlier causes 4 MySQL queries every 5 seconds by default. On top of that, you will of course have an additional (and larger) amount of queries per page request as users browse your site — anywhere between ten to a few dozen (which is on the excessive side). Here's quick calculation for MySQL query loads per second:

1 user = (1 pageload x 18 queries / 120 sec interval) + (1 XML.php x 4 queries / 5 sec interval) = 0,15 qps + 0,8 qps = 0,95 qps

Then, 100 users would cause an average of 95 queries per second, which is on top of the Apache+PHP server load caused by the software. I wouldn't expect that to fly on a shared hosting account, unless the host is okay with your hogging up way more than you pay for. The RAM and CPU impact of the queries of course depends on how optimized they (and the database) are. (Aside possible MySQL caches etc., that shouldn't be relied on by programmers as primary solutions.)
dolphinbugfixing
@CodeSatori
based on your calculation and my conclusion regarding performance hog it doesn't matter if you have a few hundreds of members what matter most when they logged in or online at any given time. for example :

100,000 members with 10 users online at any given time VERSUS 500 members with 100 users online at any given time.

so basically online users eat more resources like you said "95 queries per second" and now the big question is how do we reduce that 95 queries or what see more do we need to add on the server to handle such large number of queries??
CodeSatori
@DolphinBugFixing: Exactly — what counts the most is the level of active usage. The performance difference between 100 members and 50,000 members outside active usage is marginal, as it's only data stored in the database and files stored in the filesystem. The effect of large tables in themselves is minimal in comparison to how they are accessed.

1. How to reduce the amount of queries per page/script access? By optimizing the code. And really, there's no shortcut for that — you really need see more to crack the files open, see how and why the queries are there, think how you can normalize / otherwise revise the database design and the queries to eliminate all unnecessary overhead, and introduce internal software caches as necessary to further eliminate recurring queries on unchanged data. (Even with a sophisticated MySQL cache setup, you will still have a bunch of possibly redundant PHP required to process the overhead data, again counting against your CPU.)

2. How to reduce the impact of the existing queries? By attempting to optimize your MySQL server settings to match your data situation, and installing accelerators/optimizers on your MySQL server.

Much of the talk regarding performance issues has been beating around the bush with #2 above, while #1, which is the real and lasting solution to all of this, has only received attention in passing at best. Don't think #1 is anything you'd want to get into yourself even with adequate skills, unless you intend to only shave off the critical bottlenecks, or unless you have a year plus of hobby time at your disposal. Optimization really isn't something software end-users should be worrying about.
dolphinbugfixing
@DolphinBugFixing: I think everybody is worried about this unless you have a server with dozens of CPU's and tons of RAM. Actually its a good side bad side situation... for me its a good side -- I can afford to put food in the table for my huge family fixing bugs LOL.
 
 
Below is the legacy version of the Boonex site, maintained for Dolphin.Pro 7.x support.
The new Dolphin solution is powered by UNA Community Management System.
PET:0.077399969100952