ngx_pagespeed for Nginx web server
Google recently announced the release of Nginx Pagespeed, ngx_pagespeed module - the Nginx equivalent to Apache's mod_pagespeed. Reminder, it's still a beta release, so be sure to keep up to date with Nginx Pagespeed bugs and issues as well 'Using the PageSpeed Module' guide.
- What is ngx_pagespeed and why install the module ?
- Google I/O 2013 - Video
- Introduction to ngx_pagespeed on Centmin Mod
- How to install and configure ngx_pagespeed on Centmin Mod Nginx ?
- How to exclude specific urls, files, directories from ngx_pagespeed ? (added May 3rd, 2014)
- How to disable ngx_pagespeed integration in Centmin Mod Nginx ?
- How to upgrade ngx_pagespeed on Centmin Mod ?
- ngx_pagespeed config files
- ngx_pagespeed statistics page
- ngx_pagespeed PageSpeed Console (added July 18th, 2013)
What is ngx_pagespeed and why install the module ?
ngx_pagespeed speeds up your site and reduces page load time. This open-source nginx server module automatically applies web performance best practices to pages, and associated assets (CSS, JavaScript, images) without requiring that you modify your existing content or workflow.
ngx_pagespeed improves web page latency and bandwidth usage by changing the resources on that web page to implement web performance best practices. Each optimization is implemented as a custom filter in ngx_pagespeed, which are executed when nginx serves the website assets. Some filters simply alter the HTML content, and other filters change references to CSS, JavaScript, or images to point to more optimized versions.
Google I/O 2013 - Video
Ilya Grigorik presentation video at Google I/O 2013 explaing Google Pagespeed - Automating Performance Best Practices with PageSpeed. If after watching that video, you get the itch to test your own web site's PageSpeed scores, check out one of my blog articles titled - Web site page loading speed optimization online tools & tests, which lists a number of online page speed and analysis tools - many I use myself.
Introduction to ngx_pagespeed on Centmin Mod
I decided to add ngx_pagespeed module support to Centmin Mod v1.2.3-eva2000.01 beta release. There are 4 variables in centmin.sh which controls whether ngx_pagespeed module support is added. Note: Centmin Mod 123.08stable is not longer compatible since ngx_pagespeed 1.10+ release. Instructions on how to do a fresh install or switch 123.08stable to 123.09beta01 via centmin.sh menu option 23 submenu option 3 are outlined here. In latest 123.09beta01, instead of editing centmin.sh directly to control these variables do it via persistent config file - create or append to a persistent config file at /etc/centminmod/custom_config.inc and set variables:
- NGINX_PAGESPEEDGITMASTER='n'. Recommended lest to disabled = n. Setting to yes will override the below variable's version number and grab the lastest master source repository from https://github.com/pagespeed/ngx_pagespeed. This setting was added as some specific version builds had bugs which were not fixed in the specific version but were fixed in master repository. Setting this to yes would allow you to get the latest fixes which don't exist in version specific versions. If you have this set to yes and Nginx fails to compile, it could be a bug in master repository, so set it to no NGINX_PAGESPEEDGITMASTER=n and specify a specific version below and re-run centmin.sh menu option #4 to recompile Nginx
- NGINX_PAGESPEED='y'. Setting NGINX_PAGESPEED='n' will disable ngx_pagespeed module when you next run Nginx upgrade via menu option #4.
- NGXDYNAMIC_NGXPAGESPEED='y'. Setting NGXDYNAMIC_NGXPAGESPEED='y' will allow ngx_pagespeed module to be compiled as a nginx dynamic module instead of static module when you next run Nginx upgrade via menu option #4.
- NGXPGSPEED_VER='1.12.34.2-stable' - sets the ngx_pagespeed version to install that can override the version listed in centmin.sh by default. Only required if there is a newer ngx_pagespeed version than the one listed in centmin.sh.
- NGINX_PAGESPEEDPSOL_VER='1.12.34.2' - sets the Pagespeed Optimization Library version - which as of ngx_pagespeed 1.5.27.3 beta is no longer included with main ngx_pagespeed module download to the PSO version that can override the version listed in centmin.sh by default. Should make the version used by NGXPGSPEED_VER='1.12.34.2-stable' minus the ending
-stabletag. Only required if there is a newer ngx_pagespeed version than the one listed in centmin.sh.
While ngx_pagespeed module is installed, the default ngx_pagespeed state is turned off. After ngx_pagespeed module is added to Nginx, you'll need to enable it server wide by default via Centmin Mod command line shortcut within SSH telnet type:
pscontrol on
You'll see the nginx restart process below output after command is run:
service nginx restart nginx: the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok nginx: configuration file /usr/local/nginx/conf/nginx.conf test is successful Stopping nginx: [ OK ] Starting nginx: [ OK ]
If you do not want a particular Nginx vhost domain to have ngx_pagespeed enabled, you can disable ngx_pagespeed on a per vhost basis by editing your domain.com.conf vhost file and adding entry - for ngx_pagespeed earlier than 1.13.35.51, use:
pagespeed off;
for ngx_pagespeed version equal to or greater than 1.13.35.51, use the updated syntax
pagespeed unplugged;
For specific Nginx vhost domain only edit vhost config and add to the existing 4 lines of below - make sure to add it after include /usr/local/nginx/conf/pagespeed.conf line:
# ngx_pagespeed & ngx_pagespeed handler #include /usr/local/nginx/conf/pagespeed.conf; pagespeed unplugged; #include /usr/local/nginx/conf/pagespeedhandler.conf; #include /usr/local/nginx/conf/pagespeedstatslog.conf;
Then restart nginx server with service restart or command shortcut commands
service nginx restart
or
ngxrestart
Note: as at July 18, 2013 for ngx_pagespeed 1.6.29.3+ beta the verbose output on Nginx server restart has been hidden.
You may need to hard refresh your browser page to see changes made when ngx_pagespeed module is installed and pagespeed is enabled. You can also dynamically turn on and off ngx_pagespeed on a per url link basis by tagging on the end of urls the following line ?ModPagespeed=off
To manually enable ngx_pagespeed on per url link basis
yourdomain.com/?ModPagespeed=on
To manually disable ngx_pagespeed on per url link basis
yourdomain.com/?ModPagespeed=off
Checking if ngx_pagespeed is running
You can check to see if ngx_pagespeed module is working by checking the headers for X-Page-Speed header listed. Via SSH telnet using curl:
curl -I http://localhost/?ModPagespeed=on HTTP/1.1 200 OK Server: nginx Content-Type: text/html; charset=UTF-8 Connection: keep-alive Vary: Accept-Encoding Pragma: public Date: Thu, 01 Jan 1970 00:00:00 GMT Expires: Thu, 01 Jan 1970 00:00:00 GMT Cache-Control: max-age=0, no-cache X-Page-Speed: 1.7.30.4-3847
curl -I http://localhost/?ModPagespeed=off HTTP/1.1 200 OK Server: nginx Date: Sun, 28 Apr 2013 06:13:03 GMT Content-Type: text/html; charset=utf-8 Content-Length: 3796 Last-Modified: Sun, 28 Apr 2013 05:29:32 GMT Connection: keep-alive Vary: Accept-Encoding Expires: Sun, 05 May 2013 06:13:03 GMT Cache-Control: max-age=604800 Pragma: public Cache-Control: public, must-revalidate, proxy-revalidate Accept-Ranges: bytes
How to install and configure ngx_pagespeed on Centmin Mod Nginx ?
- You'll need to download latest announced 123.09beta01 beta release. Centmin Mod 123.08stable is not longer compatible since ngx_pagespeed 1.10+ release. Instructions on how to do a fresh install or switch 123.08stable to 123.09beta01 via
centmin.sh menu option 23 submenu option 3are outlined here. Out of the box, ngx_pagespeed is disabled by default, but can be enabled in 123.09beta01 via persistent config file/etc/centminmod/custom_config.incyou create or append to if already exists and set 2 variables in it and then runcentmin.sh menu option 4to recompile nginx 1.11.3+.NGXDYNAMIC_NGXPAGESPEED='y' NGINX_PAGESPEED='y'
- Either install Centmin Mod 123.09beta01 release on fresh CentOS server via
centmin.sh. Fresh installs will have their Nginx vhost created templates already populated with the correct files in server {} section to enable ngx_pagespeed. You just need to uncomment (remove hash # in front of the 3 include *.conf files. Then enable it via Centmin Mod command line shortcut within SSH telnet type:pscontrol on
# ngx_pagespeed & ngx_pagespeed handler #include /usr/local/nginx/conf/pagespeed.conf; #include /usr/local/nginx/conf/pagespeedhandler.conf; #include /usr/local/nginx/conf/pagespeedstatslog.conf;
Note: as at July 18, 2013 for ngx_pagespeed 1.6.29.3 beta, a third include line has been added to control ngx_pagespeed console statistics and logging at include /usr/local/nginx/conf/pagespeedstatslog.conf. If you installed prior older release less than Centmin Mod v1.2.3-eva2000.03 and where you upgraded to Centmin Mod v1.2.3-eva2000.03+ and higher, you may need to manually add the third include line to existing Nginx vhosts.
- For existing Centmin Mod users, run Centmin Mod v1.2.3's
centmin.shand select menu option #4 to upgrade Nginx - this will recompile Nginx with ngx_pagespeed module support and setup the appropriate configuration files. Only difference from fresh installs is that for existing Centmin Mod users upgrading, you will need to manually edit your existing Nginx vhost and add those 3 include *.conf file lines. Then enable it via Centmin Mod command line shortcut within SSH telnet type:pscontrol on
. Example below:server { listen 80 default_server; server_name localhost; root html; access_log /var/log/nginx/localhost.access.log main buffer=32k; error_log /var/log/nginx/localhost.error.log error; # ngx_pagespeed & ngx_pagespeed handler include /usr/local/nginx/conf/pagespeed.conf; include /usr/local/nginx/conf/pagespeedhandler.conf; include /usr/local/nginx/conf/pagespeedstatslog.conf; # limit_conn limit_per_ip 16; # ssi on; location / { # Enables directory listings when index file not found # autoindex on; } include /usr/local/nginx/conf/staticfiles.conf; include /usr/local/nginx/conf/php.conf; #include /usr/local/nginx/conf/phpstatus.conf; include /usr/local/nginx/conf/drop.conf; #include /usr/local/nginx/conf/errorpage.conf; }
How to exclude specific urls, files, directories from ngx_pagespeed ?
The Nginx PageSpeed User Guide outlines how to disallow elements/urls at https://developers.google.com/speed/pagespeed/module/restricting_urls. You would enter the pagespeed Disallow entries into your /usr/local/nginx/conf/pagespeed.conf outlined here. The pagespeed.conf can be accessed via nano text editor via the shortcut command = pscontrol edit. After adding and editing your pagespeed.conf, you would need to restart Nginx web server for the changes to take effect.
For example:
pagespeed Disallow "*/jquery-ui-1.8.2.custom.min.js"; pagespeed Disallow "*/js_tinyMCE.js";
You can enable Nginx Pagespeed Console and message logging to double check which files are excluded or processed by Nginx Pagespeed. Then once enabled, check your message log at yourdomain.com/ngx_pagespeed_message.
How to disable ngx_pagespeed integration in Centmin Mod Nginx ?
You can also disable ngx_pagespeed from Centmin Mod Nginx if you do not want to integrate ngx_pagespeed module with Nginx. To do this:
- Create or append to a persistent config file at
/etc/centminmod/custom_config.incthe below setting to disable ngx_pagespeedEditand change the ngx_pagespeed module variable NGINX_PAGESPEED from y to n.centmin.shNGINX_PAGESPEED=n # Install ngx_pagespeed
- Then run centmin.sh and select menu option #4 for Nginx upgrade and specifying the Nginx web server version to upgrade/recompile i.e. 1.11.3 or higher. Ensure NGINX_PAGESPEED=n is set if it wasn't already.
How to upgrade ngx_pagespeed on Centmin Mod ?
Upgrading ngx_pagespeed on Centmin Mod Nginx installs is easy and done by:
- Editing
centmin.shand changing the ngx_pagespeed module and Pagespeed Optimization Library versionsNGINX_PAGESPEED=y # Install ngx_pagespeed NGXPGSPEED_VER='1.7.30.1-beta' NGINX_PAGESPEEDPSOL_VER='1.7.30.1'
For example, ngx_pagespeed 1.11.33.3 beta was just released so to upgrade change centmin.sh variable via persistent config file - create or append to a persistent config file at/etc/centminmod/custom_config.incand set variables:NGINX_PAGESPEED=y # Install ngx_pagespeed NGXPGSPEED_VER='1.11.33.3-beta' NGINX_PAGESPEEDPSOL_VER='1.11.33.3'
- Running centmin.sh and selecting menu option #4 for Nginx upgrade and specifying the Nginx web server version to upgrade/recompile i.e. 1.11.3 or higher. Ensure NGINX_PAGESPEED=y is set if it wasn't already.
If the Nginx recompile fails with missing PSOL library, double check if there was an incompleted or broken tarball download by deleting the existing downloaded versions and then re-running menu option #4 by typing these commands:
Replace the version numbers in below 2 lines, 1.7.30.4 with the version you are trying to upgrade to but failed previously to upgrade.
rm -rf /svr-setup/ngx_pagespeed-release-1.9.32.1-beta rm -rf /svr-setup/release-1.9.32.1-beta.tar.gz
Then re-run centmin.sh menu option #4 to recompile Nginx.
If after upgrading, you do not see the X-Page-Speed header when you check your files' headers, you may need to enable ngx_pagespeed via Centmin Mod command line shortcut within SSH telnet type:
pscontrol on
Dealing with ngx_pagespeed upgrade errors
If during upgrading you run into errors at either Nginx configure, make or make install stages, Centmin Mod script has error checking routine which will abort at whichever stage is causing an error, so you can troubleshoot and find where the error is.
For instance, upgrading ngx_pagespeed module from 1.5.27.1 beta to 1.5.27.3 beta complained of missing Pagespeed Optimization Library which is no longer bundled with 1.5.27.3 beta and aborted at the Nginx configure stage. This is now fixed in updated Centmin Mod v1.2.3 beta.
checking for struct dirent.d_type ... found
checking for sysconf(_SC_NPROCESSORS_ONLN) ... found
checking for openat(), fstatat() ... found
configuring additional modules
adding module in ../ngx_pagespeed-release-1.5.27.3-beta
ngx_pagespeed: pagespeed optimization library not found:
You need to separately download the pagespeed library:
$ cd /path/to/ngx_pagespeed
$ wget https://dl.google.com/dl/page-speed/psol/1.5.27.3.tar.gz
$ tar -xzvf 1.5.27.3.tar.gz # expands to psol/
Or see the installation instructions:
https://github.com/pagespeed/ngx_pagespeed#how-to-build
*************************************************
Tue Apr 30 05:40:04 EST 2013
Error: 1, Nginx configure failed
*************************************************
Centmin Mod has automatic logging of Nginx upgrade routine when you run centmin.sh and upgrade Nginx via menu option #4. So you can troubleshoot any compilation errors you encounter with Nginx and ngx_pagespeed upgrades.
You can check the logs at /root/centminlogs.
i.e.
ls -lahrt /root/centminlogs/ total 5.5M -rw-r--r-- 18.5K Apr 28 15:16 centminmod_1.2.3-eva2000.01_280413-151022_yuminstall_centos6.log dr-xr-x---. 74.0K Apr 28 15:44 .. -rw-r--r-- 14.9M Apr 28 15:46 centminmod_1.2.3-eva2000.01_280413-151022_install.log -rw-r--r-- 1 24K Apr 30 05:36 centminmod_1.2.3-eva2000.01_300413-053539_nginx_upgrade.log drwxr-xr-x 24.0K Apr 30 05:39 . -rw-r--r-- 1632K Apr 30 05:59 centminmod_1.2.3-eva2000.01_300413-053939_nginx_upgrade.log
ngx_pagespeed config files
When you install or upgrade Nginx via menu option #4, 3 configuration files and 1 pscontrol file will be created.
- /usr/local/nginx/conf/pagespeed.conf - main ngx_pagespeed settings.
- /usr/local/nginx/conf/pagespeedhandler.conf - location directives to ensure pagespeed handler receives optimised pagespeed resources
- /usr/local/nginx/conf/pagespeedstatslog.conf - location directives and settings added in ngx_pagespeed 1.6.29.3 beta for pagespeed console
- /usr/bin/pscontrol - basic command line shortcut script to turn ngx_pagespeed support on or off server wide. Within SSH telnet just type the following to turn on or of ngx_pagespeed support server wide on Centmin Mod Nginx based servers. You can also edit the pagespeed.conf and pagespeedhandler.conf files via edit and handler options.
Note: as at July 18, 2013 for ngx_pagespeed 1.6.29.3 beta additional pscontrol options where added.
pscontrol handler pscontrol edit pscontrol editstats pscontrol on pscontrol off pscontrol statson pscontrol statsoff pscontrol admin
Contents of /usr/local/nginx/conf/pagespeed.conf
Main configuration file for ngx_pagspeed module. Here's where you'll tweak and tune your ngx_pagespeed settings and enable, disable or configure your ngx_pagespeed filters. There's a full list of ngx_pagespeed filter examples.
You can edit the pagespeed.conf file via Centmin Mod command line shortcut which will invoke nano text editor within SSH telnet by typing:
pscontrol edit
You can edit the Pagespeed console pagespeedstatslog.conf file via Centmin Mod command line shortcut which will invoke nano text editor within SSH telnet by typing:
pscontrol editstats
You can edit the Pagespeed Admin pagespeedadmin.conf file via Centmin Mod command line shortcut which will invoke nano text editor within SSH telnet by typing:
pscontrol admin
You can turn on/off Pagespeed console pagespeedstatslog.conf file via Centmin Mod command line shortcut which will invoke nano text editor within SSH telnet by typing:
pscontrol statson pscontrol statsoff
There is an option to use Memcached server to store Pagespeed resources instead of file cache. You'd need to have installed Memcached server(s) via Centmin Mod. The default install for Memcached server with Centmin Mod is on port 11211 for 1st Memcached server and if at install time you choose to install a 2nd Memcached server, it will be on port 11212, so localhost:11211 is correct within pagespeed.conf.
I've added a few more commonly tuned filters I used with mod_pagespeed for Apache but commented them out and added links to documentation, so feel free to enable them for testing.
Note: updated July 18, 2013 for ngx_pagespeed 1.6.29.5 beta changes.
pagespeed off;
#######################################################
# File cache settings
######################################
# needs to exist and be writable by nginx
pagespeed FileCachePath /var/ngx_pagespeed_cache;
# pagespeed FileCacheSizeKb 102400;
# pagespeed FileCacheCleanIntervalMs 3600000;
# pagespeed FileCacheInodeLimit 500000;
#######################################################
# Set it to 0 if you want to disable this feature.
# pagespeed MessageBufferSize 100000;
#######################################################
# By default, ngx_pagespeed adds an X-PageSpeed header with a value of the version of
# ngx_pagespeed being used. This directive lets you specify the value to use instead:
# pagespeed XHeaderValue "ngx_pagespeed";
#######################################################
# let's speed up PageSpeed by storing it in the super duper fast memcached
# Ensure Memcached server installed http://centmin.com/memcached.html
# default install for centmin mod is on port 11211, so localhost:11211 is correct
# uncomment - remove hash # in front of below 2 lines to enable
# timeout set at 100 milliseconds
# pagespeed MemcachedThreads 1;
# pagespeed MemcachedServers "localhost:11211";
# pagespeed MemcachedTimeoutUs 100000;
#######################################################
## 1.6.29.5 beta defaults
######################################
pagespeed AvoidRenamingIntrospectiveJavascript on;
pagespeed ImageInlineMaxBytes 3072;
pagespeed CssImageInlineMaxBytes 0;
pagespeed MaxInlinedPreviewImagesIndex -1;
pagespeed MinImageSizeLowResolutionBytes 3072;
#######################################################
## ngx_pagespeed filters settings below ##
######################################
# show half the users an optimized site, half the regular site
# change UA-XXXXXXXXXX-1 to your GA unique id
# uncomment - remove hash # in front of below 5 lines to enable
# pagespeed RunExperiment on;
# pagespeed AnalyticsID UA-XXXXXXXXXX-1;
# pagespeed ExperimentVariable 1;
# pagespeed ExperimentSpec "id=1;percent=50;level=CoreFilters;enabled=collapse_whitespace,remove_comments;";
# pagespeed ExperimentSpec "id=2;percent=50";
# Filter settings
# filters outlined at http://ngxpagespeed.com/ngx_pagespeed_example/
pagespeed RewriteLevel CoreFilters;
pagespeed EnableFilters collapse_whitespace,remove_comments;
# make_google_analytics_async
# https://developers.google.com/speed/docs/mod_pagespeed/filter-make-google-analytics-async
# pagespeed EnableFilters make_google_analytics_async;
# move_css_above_scripts
# https://developers.google.com/speed/docs/mod_pagespeed/filter-css-above-scripts
# pagespeed EnableFilters move_css_above_scripts;
# combine_css
# https://developers.google.com/speed/docs/mod_pagespeed/filter-css-combine
# pagespeed EnableFilters combine_css;
# Uncomment this if you want to prevent mod_pagespeed from combining files
# (e.g. CSS files) across paths
#
# pagespeed CombineAcrossPaths off;
# combine_javascript
# https://developers.google.com/speed/docs/mod_pagespeed/filter-js-combine
# pagespeed EnableFilters combine_javascript;
# extend_cache
# https://developers.google.com/speed/docs/mod_pagespeed/filter-cache-extend
# pagespeed EnableFilters extend_cache;
# rewrite_css
# https://developers.google.com/speed/docs/mod_pagespeed/filter-css-rewrite
# pagespeed EnableFilters rewrite_css;
# rewrite_javascript
# https://developers.google.com/speed/docs/mod_pagespeed/filter-js-minify
# pagespeed EnableFilters rewrite_javascript;
# lazyload_images
# https://developers.google.com/speed/docs/mod_pagespeed/filter-lazyload-images
# pagespeed EnableFilters lazyload_images;
# rewrite_images
# https://developers.google.com/speed/docs/mod_pagespeed/filter-image-optimize
# pagespeed EnableFilters rewrite_images;
pagespeed DisableFilters rewrite_images;
# pagespeed EnableFilters convert_png_to_jpeg;
# pagespeed EnableFilters convert_jpeg_to_webp;
# pagespeed EnableFilters convert_to_webp_lossless;
# insert_dns_prefetch
# https://developers.google.com/speed/docs/mod_pagespeed/filter-insert-dns-prefetch
# pagespeed EnableFilters insert_dns_prefetch;
#######################################################
Note: I've disabled rewrite_images filter by default, you can comment out or remove this line to re-enable. Reason for doing so is Centmin Mod is usually installed on low end VPS with limited cpu and memory resources and usually I'd optimise all my site images before uploading them to my server (something you folks should be doing as standard practice ;) ).
pagespeed DisableFilters rewrite_images;
Contents of /usr/local/nginx/conf/pagespeedhandler.conf
You can edit the pagespeedhandler.conf file via Centmin Mod command line shortcut which will invoke nano text editor within SSH telnet by typing:
pscontrol handler
Note: updated July 18, 2013 for ngx_pagespeed 1.6.29.3+ beta changes added /pagespeed_console location.
# Ensure requests for pagespeed optimized resources go to the pagespeed
# handler and no extraneous headers get set.
location ~ "\.pagespeed\.([a-z]\.)?[a-z]{2}\.[^.]{10}\.[^.]+" { add_header "" ""; }
location ~ "^/pagespeed_static/" { }
location ~ "^/ngx_pagespeed_beacon$" { }
location /ngx_pagespeed_statistics { allow 127.0.0.1; deny all; }
location /ngx_pagespeed_global_statistics { allow 127.0.0.1; deny all; }
location /ngx_pagespeed_message { allow 127.0.0.1; deny all; }
location /pagespeed_console { allow 127.0.0.1; deny all; }
location ~ ^/pagespeed_admin { allow 127.0.0.1; deny all; }
location ~ ^/pagespeed_global_admin { allow 127.0.0.1; deny all; }
Contents of /usr/local/nginx/conf/pagespeedstatslog.conf
# Pagespeed stats logging 1.6.29.5+ pagespeed Statistics on; pagespeed StatisticsLogging on; pagespeed LogDir /var/log/pagespeed;
Contents of /usr/bin/pscontrol
Note: updated July 18, 2013 for ngx_pagespeed 1.6.29.3+ beta changes. If you're more comfortable using a different linux text editor than nano, just edit /usr/bin/pscontrol to your liking.
#/bin/bash
# turn ngx_pagespeed on or off switch
pshandler() {
nano -w /usr/local/nginx/conf/pagespeedhandler.conf
}
psconf() {
nano -w /usr/local/nginx/conf/pagespeed.conf
}
psstatsconf() {
nano -w /usr/local/nginx/conf/pagespeedstatslog.conf
}
psoff() {
sed -i 's/pagespeed on;/pagespeed off;/g' /usr/local/nginx/conf/pagespeed.conf
/etc/init.d/nginx restart
}
pson() {
sed -i 's/pagespeed off;/pagespeed on;/g' /usr/local/nginx/conf/pagespeed.conf
/etc/init.d/nginx restart
}
psstatslogoff() {
sed -i 's/pagespeed Statistics on;/pagespeed Statistics off;/g' /usr/local/nginx/conf/pagespeedstatslog.conf
sed -i 's/pagespeed StatisticsLogging on;/pagespeed StatisticsLogging off;/g' /usr/local/nginx/conf/pagespeedstatslog.conf
/etc/init.d/nginx restart
}
psstatslogon() {
sed -i 's/pagespeed Statistics off;/pagespeed Statistics on;/g' /usr/local/nginx/conf/pagespeedstatslog.conf
sed -i 's/pagespeed StatisticsLogging off;/pagespeed StatisticsLogging on;/g' /usr/local/nginx/conf/pagespeedstatslog.conf
/etc/init.d/nginx restart
}
case "$1" in
handler)
pshandler
;;
edit)
psconf
;;
editstats)
psstatsconf
;;
on)
pson
;;
off)
psoff
;;
statson)
psstatslogon
;;
statsoff)
psstatslogoff
;;
*)
echo "pscontrol handler"
echo "pscontrol edit"
echo "pscontrol editstats"
echo "pscontrol on"
echo "pscontrol off"
echo "pscontrol statson"
echo "pscontrol statsoff"
;;
esac
exit
ngx_pagespeed statistics page
Update: January 29th, 2017: New revamped admin stats page in Nginx Pagespeed 1.9.32-1 beta, full outline with screenshots.
You can access ngx_pagespeed statistics page via SSH telnet at grabbing url link to localhost/pagespeed_global_admin/ or the preferred yourdomain.com/pagespeed_global_admin/ link which now has a separate tab section for ngx_pagespeed statistics. Public access to the url is blocked and only ip 127.0.0.1 is allowed access as per the default pagespeedhandler.conf settings. While /pagespeed_global_admin is not specifically defined in the location contexts below, it is protected and controlled by those location contexts so no need to specifically define a location context for /pagespeed_global_admin. You could add your own ip address (i.e. 192.168.0.99) and your server's ip (xxx.xxx.xxx.xxx) to the configuration so you can view ngx_pagespeed statistics via yourdomain.com/pagespeed_global_admin/
location /ngx_pagespeed_statistics { allow 127.0.0.1; allow 192.168.0.99; allow xxx.xxx.xxx.xxx; deny all; }
location /ngx_pagespeed_global_statistics { allow 127.0.0.1; allow 192.168.0.99; allow xxx.xxx.xxx.xxx; deny all; }
location /ngx_pagespeed_message { allow 127.0.0.1; allow 192.168.0.99; allow xxx.xxx.xxx.xxx; deny all; }
location /pagespeed_console { allow 127.0.0.1; allow 192.168.0.99; allow xxx.xxx.xxx.xxx; deny all; }
location ~ ^/pagespeed_admin { allow 127.0.0.1; allow 192.168.0.99; allow xxx.xxx.xxx.xxx; deny all; }
location ~ ^/pagespeed_global_admin { allow 127.0.0.1; allow 192.168.0.99; allow xxx.xxx.xxx.xxx; all
Example of using lynx command to view configuration
lynx -dump yourdomain.com/pagespeed_global_admin/config
Example of using lynx command to view statistics
lynx -dump yourdomain.com/pagespeed_global_admin/statistics
Example statistics output:
View Configuration, Statistics (with memcached Stats). Global Statistics cache_time_us: 0 cache_hits: 0 cache_misses: 0 cache_fallbacks: 0 cache_expirations: 0 cache_inserts: 0 cache_deletes: 0 instrumentation_filter_script_added_count: 0 cache_extensions: 0 not_cacheable: 0 critical_css_beacon_filter_script_added_count: 0 critical_css_no_beacon_due_to_missing_data: 0 critical_css_skipped_due_to_charset: 0 critical_images_beacon_filter_script_added_count: 0 css_file_count_reduction: 0 css_filter_blocks_rewritten: 0 css_filter_parse_failures: 0 css_filter_fallback_rewrites: 0 css_filter_fallback_failures: 0 css_filter_rewrites_dropped: 0 css_filter_total_bytes_saved: 0 css_filter_total_original_bytes: 0 css_filter_uses: 9 flatten_imports_charset_mismatch: 0 flatten_imports_invalid_url: 0 flatten_imports_limit_exceeded: 0 flatten_imports_minify_failed: 0 flatten_imports_recursion: 0 flatten_imports_complex_queries: 0 css_imports_to_links: 0 css_elements_moved: 0 domain_rewrites: 0 google_analytics_page_load_count: 0 google_analytics_rewritten_count: 0 image_file_count_reduction: 0 image_rewrites: 0 image_norewrites_high_resolution: 0 image_rewrites_dropped_intentionally: 0 image_rewrites_dropped_mime_type_unknown: 0 image_rewrites_dropped_server_write_fail: 0 image_rewrites_dropped_nosaving_resize: 0 image_rewrites_dropped_nosaving_noresize: 0 image_rewrites_dropped_due_to_load: 0 image_rewrites_squashing_for_mobile_screen: 0 image_rewrite_total_bytes_saved: 0 image_rewrite_total_original_bytes: 0 image_rewrite_uses: 0 image_inline: 0 image_webp_rewrites: 0 image_ongoing_rewrites: 0 image_webp_conversion_gif_timeouts: 0 image_webp_conversion_png_timeouts: 0 image_webp_conversion_jpeg_timeouts: 0 image_webp_alpha_timeouts: 0 image_webp_opaque_timeouts: 0 in_place_oversized_opt_stream: 0 in_place_uncacheable_rewrites: 0 inserted_ga_snippets: 0 javascript_blocks_minified: 36 javascript_libraries_identified: 0 javascript_minification_failures: 0 javascript_total_bytes_saved: 963 javascript_total_original_bytes: 9135 javascript_minify_uses: 27 javascript_reducing_minifications: 27 javascript_minification_disabled: 0 javascript_did_not_shrink: 9 javascript_failed_to_write: 0 js_file_count_reduction: 0 num_local_storage_cache_candidates_found: 0 num_local_storage_cache_stored_total: 0 num_local_storage_cache_stored_images: 0 num_local_storage_cache_stored_css: 0 num_local_storage_cache_candidates_added: 0 num_local_storage_cache_candidates_removed: 0 converted_meta_tags: 9 num_distributed_rewrite_successes: 0 num_distributed_rewrite_failures: 0 num_deadline_alarm_invocations: 0 url_trims: 0 url_trim_saved_bytes: 0 resource_url_domain_rejections: 63 rewrite_cached_output_missed_deadline: 0 rewrite_cached_output_hits: 27 rewrite_cached_output_misses: 0 resource_404_count: 0 slurp_404_count: 0 total_page_load_ms: 0 page_load_count: 0 resource_fetches_cached: 0 resource_fetch_construct_successes: 0 resource_fetch_construct_failures: 0 num_flushes: 9 num_fallback_responses_served: 0 num_conditional_refreshes: 0 ipro_served: 0 ipro_not_in_cache: 0 ipro_not_rewritable: 0 total_fetch_count: 0 total_rewrite_count: 9 num_rewrites_executed: 0 num_rewrites_dropped: 0 num_resource_fetch_successes: 0 num_resource_fetch_failures: 0 html-worker-queue-depth: 0 rewrite-worker-queue-depth: 0 low-priority-worked-queue-depth: 0 cache_batcher_dropped_gets: 0 critical_images_valid_count: 0 critical_images_expired_count: 0 critical_images_not_found_count: 0 critical_css_valid_count: 0 critical_css_expired_count: 0 critical_css_not_found_count: 0 critical_selectors_valid_count: 0 critical_selectors_expired_count: 0 critical_selectors_not_found_count: 0 pcache-cohorts-clientstate_deletes: 0 pcache-cohorts-clientstate_hits: 0 pcache-cohorts-clientstate_inserts: 0 pcache-cohorts-clientstate_misses: 0 cache_flush_count: 0 cache_flush_timestamp_ms: 0 memcache_timeouts: 0 memcache_last_error_checkpoint_ms: 0 memcache_error_burst_size: 0 file_cache_disk_checks: 0 file_cache_cleanups: 0 file_cache_evictions: 0 file_cache_bytes_freed_in_cleanup: 0 file_cache_deletes: 0 file_cache_hits: 0 file_cache_inserts: 0 file_cache_misses: 0 lru_cache_deletes: 0 lru_cache_hits: 0 lru_cache_inserts: 0 lru_cache_misses: 0 shm_cache_deletes: 0 shm_cache_hits: 0 shm_cache_inserts: 0 shm_cache_misses: 0 memcached_deletes: 0 memcached_hits: 27 memcached_inserts: 0 memcached_misses: 18 compressed_cache_corrupt_payloads: 0 compressed_cache_original_size: 0 compressed_cache_compressed_size: 0 serf_fetch_request_count: 0 serf_fetch_bytes_count: 0 serf_fetch_time_duration_ms: 0 serf_fetch_cancel_count: 0 serf_fetch_active_count: 0 serf_fetch_timeout_count: 0 serf_fetch_failure_count: 0 serf_fetch_cert_errors: 0 pcache-cohorts-beacon_cohort_deletes: 0 pcache-cohorts-beacon_cohort_hits: 0 pcache-cohorts-beacon_cohort_inserts: 0 pcache-cohorts-beacon_cohort_misses: 9 pcache-cohorts-dom_deletes: 0 pcache-cohorts-dom_hits: 0 pcache-cohorts-dom_inserts: 0 pcache-cohorts-dom_misses: 9 child_shutdown_count: 0 Histogram Name (click to view) Count Avg StdDev Min Median Max 90% 95% 99% Rewrite Latency Histogram 9 3.0 0.7 2 2 4 4 4 4 memcached_get_count 45 1.0 0.0 1 1 1 1 1 1 memcached_hit_latency_us 27 233.1 70.0 137 230 439 290 300 400 memcached_lookup_size_bytes 27 32.3 28.8 12 38 73 68 70 73 pcache-cohorts-beacon_cohort_get_count 9 1.0 0.0 1 1 1 1 1 1 pcache-cohorts-dom_get_count 9 1.0 0.0 1 1 1 1 1 1 [ 0, 20) 9 100.0% 100.0%
ngx_pagespeed PageSpeed Console
Update: October 16th, 2014: New revamped admin stats page in Nginx Pagespeed 1.9.32-1 beta, full outline with screenshots.
PageSpeed Console was added by ngx_pagespeed 1.6.29.3+ beta. To enable PageSpeed console, you need to follow these steps below:
- Edit pagespeed.conf (prior to 1.9.32.1) or edit pagespeedadmin.conf (from 1.9.32.1 onwards) and enable MessageBufferSize setting by uncommenting the option (remove hash # in front of it). You can use pscontrol file shortcut to quickly edit the pagespeed.conf
pscontrol edit
You can use pscontrol file shortcut to quickly edit the pagespeedadmin.confpscontrol admin
####################################################### # Set it to 0 if you want to disable this feature. # pagespeed MessageBufferSize 100000;
becomes
####################################################### # Set it to 0 if you want to disable this feature. pagespeed MessageBufferSize 100000;
- Enable PageSpeed statistics and statistics logging within pagespeedstatslog.conf. You can use pscontrol file shortcut to quickly enable or disable statistics and statistics logging within the pagespeedstatslog.conf file.
Enable
pscontrol statson
Disable
pscontrol statsoff
- Ensure your ISP IP address and your server IP address are added to allow list for pagespeedhandler.conf file. You can use pscontrol file shortcut to quickly edit the pagespeedhandler.conf file
pscontrol handler
For instance, if my ISP IP address is 192.168.0.99 and server IP address is xxx.xxx.xxx.xxx, then I will add allow 192.168.0.99 and xxx.xxx.xxx.xxx to existing config to end up as follows:location /ngx_pagespeed_statistics { allow 127.0.0.1; allow 192.168.0.99; allow xxx.xxx.xxx.xxx; deny all; } location /ngx_pagespeed_global_statistics { allow 127.0.0.1; allow 192.168.0.99; allow xxx.xxx.xxx.xxx; deny all; } location /ngx_pagespeed_message { allow 127.0.0.1; allow 192.168.0.99; allow xxx.xxx.xxx.xxx; deny all; } location /pagespeed_console { allow 127.0.0.1; allow 192.168.0.99; allow xxx.xxx.xxx.xxx; deny all; } location ~ ^/pagespeed_admin { allow 127.0.0.1; allow 192.168.0.99; allow xxx.xxx.xxx.xxx; deny all; } location ~ ^/pagespeed_global_admin { allow 127.0.0.1; allow 192.168.0.99; allow xxx.xxx.xxx.xxx; all - Then restart Nginx server
- Then access PageSpeed Console from your web browser at yourdomain.com/pagespeed_global_admin
The PageSpeed Console statistics will be logged into the pagespeedstatslog.conf defined log directory at /var/log/pagespeed within a file called /var/log/pagespeed/stats_log_global.
Output contents of /var/log/pagespeed/stats_log_global:
timestamp: 1374115752310 cache_hits: 0 cache_misses: 0 cache_backend_hits: 0 cache_backend_misses: 0 cache_expirations: 0 css_combine_opportunities: 0 css_file_count_reduction: 0 css_filter_blocks_rewritten: 0 css_filter_parse_failures: 0 css_filter_total_bytes_saved: 0 flatten_imports_charset_mismatch: 0 flatten_imports_invalid_url: 0 flatten_imports_limit_exceeded: 0 flatten_imports_minify_failed: 0 flatten_imports_recursion: 0 image_rewrites: 0 image_norewrites_high_resolution: 0 image_rewrites_dropped_intentionally: 0 image_rewrites_dropped_nosaving_resize: 0 image_rewrites_dropped_nosaving_noresize: 0 image_rewrites_dropped_due_to_load: 0 image_rewrite_total_bytes_saved: 0 image_ongoing_rewrites: 0 javascript_blocks_minified: 0 javascript_minification_failures: 0 javascript_total_bytes_saved: 0 converted_meta_tags: 0 resource_url_domain_acceptances: 0 resource_url_domain_rejections: 0 resource_404_count: 0 slurp_404_count: 0 total_page_load_ms: 0 page_load_count: 0 num_cache_control_rewritable_resources: 0 num_cache_control_not_rewritable_resources: 0 num_flushes: 1 num_fallback_responses_served: 0 num_rewrites_executed: 0 num_rewrites_dropped: 0 serf_fetch_request_count: 0 serf_fetch_bytes_count: 0 serf_fetch_failure_count: 0 memcached_hits: 0 memcached_misses: 0
The PageSpeed console displays graphs for these metrics:
- Resources not loaded because of fetch failures: Images, CSS or JavaScript could not be loaded because backend HTTP fetch failed. Remedy: You may have to reconfigure your server to allow outgoing connections or to have access to DNS.
- Resources not rewritten because domain wasn't authorized: Resources could not be rewritten because they were on a different domain than the HTML and that domain was not explicitly authorized. Remedy: Add Domain authorizations for all domains you control.
-
Resources not rewritten because of restrictive Cache-Control headers:
Resources could not be rewritten because they had restrictive Cache-Control
headers explicitly set (for example:
Cache-Control: private,Cache-Control: max-age=0orCache-Control: no-transform). Remedy: Reconfigure your server to serve these resources with public caching headers (or no Cache-Control headers at all). For example,Header set Cache-Control "max-age=600"in Apache config. - Cache misses: Resources were not rewritten because they were not found in cache. This is expected while your cache warms up, soon after you install, however if it continues to be high, your cache is probably too small. Remedy: Increase the FileCacheSizeKb to be about 5 times as large as your website content (to store all of the original resources and various versions of rewritten resources).
-
Cache lookups that were expired:
Although these resources were found in cache, they were not rewritten
because they were older than their max-age.
Remedy: (A) Enable
LoadFromFile to tell
the server to load the files straight from disk rather than through
HTTP. (B) Reconfigure your server to serve resources with longer TTL.
For example,
Header set Cache-Control "max-age=3600"to set one hour TTL in Apache config. - CSS files not rewritten because of parse errors: CSS files could not be rewritten because our parser found syntax errors that it could not recover from. Remedy: Fix CSS files to use proper syntax. You can use the stand-alone css_minify_main to find what part of the CSS file has parse errors. We have had some problems in the past with valid CSS3 or proprietary CSS extensions causing CSS parsing errors. If you find that your valid CSS is failing to parse, let us know at mod-pagespeed-discuss so that we can fix the parser.
- JavaScript minification failures: JavaScript files could not be minified because our parser found some syntax problem that it could not recover from. This is very uncommon. Remedy: As with CSS, fix the JavaScript files that had problems.
-
Image rewrite failures:
Image files were not rewritten for various reasons:
image_norewrites_high_resolution: Image was too large. Currently we only rewrite images below 8 Megapixels. Remedy: Resize original images to a reasonable size.image_rewrites_dropped_decode_failure: Image could not be parsed by our code. Remedy: Fix malformed images.image_rewrites_dropped_due_to_load: Image was not rewritten because your system was already busy rewriting other images. This generally can happen when you first install PageSpeed while the cache warms up. Remedy: If image rewrites continue to be dropped after a day or so, you may need to increase your cache size or increase the ImageMaxRewritesAtOnce.image_rewrites_dropped_mime_type_unknown: Image could not be rewritten because we do not recognize its MIME-type. For example, we do not parseimage/x-iconorimage/svg+xml. Remedy: Convert your images to a type that we understand (gif, png, jpg, webp) or perhaps just fix a broken server config that is serving images with a bogusContent-Typeheader.image_rewrites_dropped_server_write_fail: Unexpected server error while rewriting images. If you get a significant number of these email us at mod-pagespeed-discuss so that we can investigate.
(source: https://developers.google.com/speed/pagespeed/module/console)
Example PageSpeed Console statistics. Click on image for full statistics screenshots.
Centmin Mod v1.2.3-eva2000.03 menu
--------------------------------------------------------
Centmin Mod 1.2.3-eva2000.03 - http://centminmod.com
--------------------------------------------------------
Centmin Mod Menu
--------------------------------------------------------
1). Centmin Install
2). Add Nginx vhost domain
3). NSD setup domain name DNS
4). Nginx Upgrade / Downgrade
5). PHP Upgrade / Downgrade
6). XCache Re-install
7). APC Cache Re-install
8). XCache Install
9). APC Cache Install
10). Memcached Server Re-install
11). MariaDB 5.2.x Branch Upgrade Only
12). MariaDB 5.2.x to MariaDB 5.5 YUM upgrade
13). Install ioping.sh vbtechsupport.com/1239/
14). SELinux disable
15). Install/Re-install imagick PHP Extension
16). Change SSHD Port Number
17). Multi-thread compression: pigz,pbzip2,lbzip2,p7zip etc
18). Suhosin PHP Extension install
19). Install FFMPEG and FFMPEG PHP Extension
20). NSD Re-install
21). Exit
--------------------------------------------------------
Enter option [ 1 - 21 ]
--------------------------------------------------------
The current updated Centmin Mod Nginx configure options used to add ngx_pagespeed module support. Full list of Centmin Mod Nginx included modules configured at compile time is listed here.
nginx version: nginx/1.4.1 built by gcc 4.4.7 20120313 (Red Hat 4.4.7-3) (GCC) TLS SNI support enabled configure arguments: --sbin-path=/usr/local/sbin --conf-path=/usr/local/nginx/conf/nginx.conf --add-module=../ngx_pagespeed-release-1.6.29.5-beta --with-http_spdy_module --with-http_ssl_module --with-http_gzip_static_module --with-http_stub_status_module --with-http_sub_module --with-http_addition_module --with-http_secure_link_module --with-http_flv_module --with-http_realip_module --with-libatomic --with-pcre=../pcre-8.32 --with-pcre-jit --with-http_dav_module --add-module=../nginx-dav-ext-module-0.0.2 --add-module=../ngx-fancyindex-ngx-fancyindex --add-module=../ngx_cache_purge-2.1 --add-module=../nginx-accesskey-2.0.3 --with-openssl=../openssl-1.0.1e --add-module=../nginx-http-concat-master --with-openssl-opt=enable-tlsext
