[TYPO3-performance] Cache priming

Jonas Eberle jonas.eberle at d-mind.de
Thu Dec 18 10:52:28 CET 2014


Hi list,

Just if someone is interested in 'priming' the page cache of a site, 
here is a script I use.

I put it up here as open for discussion. I think a goal should always be 
to not let the frontend user regenerate the cache. With this script 
there is at least some reasonable possiblity that this regenerates the 
page cache instead of the user having to wait.

Regards,
Jonas


#! /bin/bash
# @author jonas.eberle at d-mind.de
# Crawl a site to 'prime' the frontend cache. It will log the total time 
taken for one run.
# It will not spawn multiple times (locking mechanism). Thus it is save 
calling it in an often-running cronjob. Keep in mind that it might
# affect your stats counter if it does not ignore the wget user-agent.
#
# @param URL $1
# @param User $2 (HTTP-Auth)
# @param Password $3 (HTTP-Auth)

URL="$1"
if [ -z "$URL" ]; then
         echo 'ERROR: no URL given'
         exit 1
fi

THIS=$(readlink -f $0)
THISDIR=$(dirname $THIS)
THISDIRBASE=$THISDIR/$(basename $0)
URL_sanitized=$(echo $URL | sed -r s/[:/]/_/g)

LOG=${THISDIRBASE}_${URL_sanitized}.log
TMP_LOG=${LOG}.temp

LOCKFILE=/var/lock/$(basename $0)_${URL_sanitized}
{
         if ! flock -n 9; then
                 echo "Unable to lock $LOCKFILE, exiting" 2>&1
                 exit 1
         fi

         # remove lockfile
         trap "rm $LOCKFILE 2> /dev/null" EXIT INT KILL TERM

         echo STARTED $(date -Is -u) >> "$LOG"
         rm $TMP_LOG 2> /dev/null
         { time wget -o ${TMP_LOG} --no-check-certificate -nv -e 
robots=off -R.css,.js,.jpg,.png,.pdf -nd -r --delete-after -l 3 
--user=$2 --password=$3 $URL ; } 2>> "$LOG"
         printf "files: %d, thereof %d HTML\n" "$(wc -l $TMP_LOG | cut 
-f1 -d' ')" "$(grep -Ec '[-]>.*.htm[l]?[\"]' $TMP_LOG)" >> "$LOG"

         rm $TMP_LOG 2> /dev/null
} 9>"$LOCKFILE"



-- 
Neu: Portal Region Stuttgart - www.region-stuttgart.de
Neu: emobil-in-bw.de
Neukunde: Statistisches Landesamt BaWü
--------------------------------
d-mind bei Facebook: http://www.facebook.com/werbeagentur.internetagentur.stuttgart
--------------------------------
d-mind
Fuchs/Weiß/Strobel GbR
Mörikestraße 69
70199 Stuttgart
Tel.: +49 711 280481-1 (Durchwahl: -18)
Fax: +49 711 2804813
Inhaber: Jens Fuchs, Michael Weiß, Jens Strobel
www.d-mind.de



More information about the TYPO3-performance mailing list