Scaling website seems to be a rigorous task, It can be rigorous if decisions are made in haste or if their are architectural faults due to bad development , incorrect server design or architecture or if their is bad database management. I will try to cover every aspect of running a scalable webserver
Lets hop in then…
- Database management : Relational Database (Oracle,Mysql,Postgresql etc) are resource intensive as compared to file level operation eg. Serving many users from database is expensive then serving users with an html page.
Here are some quick tips :
– Don’t use sword where needle is enough : For a light to medium traffic website you don’t have to use heavy database software like oracle ,you can go with mysql or postgresql. Using a big database software will mean you have to invest more on hardware , with the same cost saving you can use it to scale your hardware in future.
– Less Queries , Good Performance : Less number of queries means page will load quickly and will take less time . Calculate the query time of each query running on webpage and try to combine as many query as you can using JOINS . Sometimes JOINS can be expensive too over multiple queries , proper analysis should be done before trying to reducing the queries.
In ideal condition Time taken by single Join statement < Time taken by multiple queries
You can also avoid queries in many places and save data in a text file especially where relation of the data is not needed to be determined.eg. Poll/Voting
– Follow Latest Nosql system : If expertise is available you can go and select non relational database ie. MongoDB , CouchDB, Cassandra etc , which store data as a file and not in form of table . It will become a norm in future and soon RDBMS will go exctinct , Some companies like facebook have already evolved and FB is using cassandra to store their data.
- Effective Caching Mechanism : Effective caching mechanism can save you tons of load and save resources and will also help in quick page load time.It is the most effective way to cache the database data so that same data is not queried again.
– Object Caching vs Page level caching
Object caching caches different components of webpages in cache, it is modular kind of cache ,Objects can be functions, methods, subroutines ,class etc . eg. Suppose you are using a function to display the poll result on several page , if the same function is cached , then the query will not be called until the cache is cleared even if several pages are browsed. Memcache , APC , Xcache etc can be used to achieve object caching require some modification in code.
Page level caching is simlar to static html page , whole page is converted into static file which is shown to user when user accesses it. Same level of efficiency cannot be achieved as compared to object caching as if same function is called in different pages it will query the database each time different page is accessed , But it will work fine for small website or a website with low or medium traffic. Apache Mod_cache , Nginx Cache Purge Module , Varnish can be used to achieve page level caching.
– Caching Interval : No one wants to serve old data to the client , Caching interval is a prime factor for how long data needs to be cached , Caching can be static or dynamic , If static caching is selected ,manual cache will get cleared at a defined interval , cache automatically gets cleared during data change. Caching interval should be decided by calculating the ratio of data urgency vs server resources.
- Process data only when required : When a user opens a webpage most of the website also loads data which is not visible in the viewport and is only visible if user scrolls down , but not every user will scroll down the page and he/she may click on different menu link. To avoid this technique called lazy load should be used , which loads only the data visible in current view avoiding utilization of server resources which also enhances user experience by saving their bandwidth.
