Changes between Initial Version and Version 1 of TutorialHowToCalculateLoadAndHardwareRequirements

Timestamp:

Nov 9, 2011, 1:58:25 AM (13 years ago)

Author:

AlexT

Comment:

--

TutorialHowToCalculateLoadAndHardwareRequirements

@@ +1 @@
+= How to calculate server load and hardware requirements for Dolphin =
+
+The [wiki: minimal requirements] is not always enough to choose proper server, especially if you expect site growing in the nearest time.
+This doc will help you to choose your [hosting] more adequately to your site size. 
+
+'''Please remember that this is very rough calculations for the default Dolphin setup with standard modules only. The real site can have many more modules from 3rd-party developers (which can affect on site load, space, etc) or maybe you site can have no 3rd-party modules and partially installed standard modules.'''
+
+
+
+== How to calculate load and choose proper hardware for web servers ==
+
+The load is very dependant on number, purpose, functionality and quality of installed modules and the site theme. Therefore it is almost impossible to predict it beforehand. 
+But if you have some sample setup of desired site, or pre-production setup on the webserver1 it is possible to calculate memory usage at least. 
+To do it install "Profiler" module, this module shows different timings at the bottom of every page for site administrator (by default). Try to open different pages and watch for memory usage and page execution time.
+
+[[Image(dolphin-replication-profiler.png)]]
+
+Try open different pages and measure average "Memory" and "Time" values. Lets assume that it is '''20Mb''' for memory usage and '''0.5 second''' for execution time. 
+Then if you have web server with 4G RAM (for example) the web server is capable to handle:
+
+{{{
+4G - 1G (for system usage) / 20 Mb ~ 150 simultaneous requests.
+}}}
+ 
+Please note that simultaneous requests isn't number of online users. 
+Users make requests to the server one time in several seconds or dozen of seconds. 
+
+
+
+
+If it is '''static site''' (without dynamically changed content like AJAX messenger, shoutbox, etc) 
+then it maybe one request in 30 seconds from one user. In this case the server is capable to handle the following number of online users:
+{{{
+30 (average seconds per one request from one user) / 0.5 (our average "Time" value) * 150 (simultaneous requests) = 9000 online users
+}}}
+
+
+If it is '''dynamic site''' (with messenger, live feeds, etc), Dolphin is more like this since it has 2 messengers, menu notifications, shoutbox, spy.
+It maybe 1 request from one user every second on average. In this case the server is capable to handle the following number of online users:
+{{{
+1 (average seconds per one request from one user) / 0.5 (our average "Time" value) * 150 (simultaneous requests) = 300 online users
+}}}
+
+As you can see dynamic site require '''30x''' more resources than static site.
+
+But it is possible to uninstall both messengers and disable updates for other modules and you will need only '''1 server''' instead of '''30 servers''' for your high load site!
+
+This is very rough calculation which consider only server's memory usage, '''CPU usage is not counted at all.''' It maybe situation when you have high CPU usage and low memory load and vice versa.
+For example video conversions can cause very hight CPU usage.
+
+
+
+== How to calculate load and choose proper hardware for DB server ==
+
+DB server is very dependant on many factors and can not be predicted, until the site will start functioning. 
+Just follow some advices:
+- tune MySQL settings for your needs, recheck settings as your database grow and the load increases. I suggest to use this tool to check and tune your MySQL settings: http://mysqltuner.pl/
+- install more RAM can help a lot, refer to the first advice to see if you need more RAM
+- use InnoDB DB engine instead of MyISAM if you have high load
+- store database tables on SSD disks, for the better performance you maybe will need to tune MYSQL especially for SSD disks
+- if nothing helps you may be need cluster for MySQL too
+
+
+== How to calculate calculate database and files storage ==
+
+Storage varies depending on number and types of installed modules. The better method is to start site and when you will have some number of users calculate average data size in DB and on Disk and multiply this values on some number of users you are going to predict in the future.  
+
+I'm separating DB and Disk storage for the case if your DB is located on separate server or you are going to use separate (for example SSD disk) for database storage.
+
+In the following calculations I'm using some sample Dolphin setup. The site has only 9 users, all standard modules are installed and almost every user have some content (text content, photos, videos and sounds) in different modules.
+
+Lest assume that we need to calculate future storage for 100 000 users.
+
+=== DB size ===
+
+- Current size: '''1.5 Mb''' (I subtracted size of languages table since it doesn't grow with your users database)
+- Number of users: '''9'''
+- Average DB space per user: 1.5 Mb / 9 ~ '''0.167 Mb'''
+
+Approximate DB Storage required for 100 000 users:
+100 000 * 0.167 Mb = 16700 Mb ~''' 16 Gb'''
+
+
+=== Disk storage ===
+
+- Current size:  '''374 Mb'''
+- Number of users: '''9'''
+- Average DB space per user: 374 Mb / 9 ~ '''41Mb'''
+
+Approximate Disk Storage required for 100 000 users:
+100 000 * 41Mb = '''4T'''