[TYPO3-performance] Cache priming
Jonas Eberle
jonas.eberle at d-mind.de
Thu Dec 18 10:52:28 CET 2014
Hi list,
Just if someone is interested in 'priming' the page cache of a site,
here is a script I use.
I put it up here as open for discussion. I think a goal should always be
to not let the frontend user regenerate the cache. With this script
there is at least some reasonable possiblity that this regenerates the
page cache instead of the user having to wait.
Regards,
Jonas
#! /bin/bash
# @author jonas.eberle at d-mind.de
# Crawl a site to 'prime' the frontend cache. It will log the total time
taken for one run.
# It will not spawn multiple times (locking mechanism). Thus it is save
calling it in an often-running cronjob. Keep in mind that it might
# affect your stats counter if it does not ignore the wget user-agent.
#
# @param URL $1
# @param User $2 (HTTP-Auth)
# @param Password $3 (HTTP-Auth)
URL="$1"
if [ -z "$URL" ]; then
echo 'ERROR: no URL given'
exit 1
fi
THIS=$(readlink -f $0)
THISDIR=$(dirname $THIS)
THISDIRBASE=$THISDIR/$(basename $0)
URL_sanitized=$(echo $URL | sed -r s/[:/]/_/g)
LOG=${THISDIRBASE}_${URL_sanitized}.log
TMP_LOG=${LOG}.temp
LOCKFILE=/var/lock/$(basename $0)_${URL_sanitized}
{
if ! flock -n 9; then
echo "Unable to lock $LOCKFILE, exiting" 2>&1
exit 1
fi
# remove lockfile
trap "rm $LOCKFILE 2> /dev/null" EXIT INT KILL TERM
echo STARTED $(date -Is -u) >> "$LOG"
rm $TMP_LOG 2> /dev/null
{ time wget -o ${TMP_LOG} --no-check-certificate -nv -e
robots=off -R.css,.js,.jpg,.png,.pdf -nd -r --delete-after -l 3
--user=$2 --password=$3 $URL ; } 2>> "$LOG"
printf "files: %d, thereof %d HTML\n" "$(wc -l $TMP_LOG | cut
-f1 -d' ')" "$(grep -Ec '[-]>.*.htm[l]?[\"]' $TMP_LOG)" >> "$LOG"
rm $TMP_LOG 2> /dev/null
} 9>"$LOCKFILE"
--
Neu: Portal Region Stuttgart - www.region-stuttgart.de
Neu: emobil-in-bw.de
Neukunde: Statistisches Landesamt BaWü
--------------------------------
d-mind bei Facebook: http://www.facebook.com/werbeagentur.internetagentur.stuttgart
--------------------------------
d-mind
Fuchs/Weiß/Strobel GbR
Mörikestraße 69
70199 Stuttgart
Tel.: +49 711 280481-1 (Durchwahl: -18)
Fax: +49 711 2804813
Inhaber: Jens Fuchs, Michael Weiß, Jens Strobel
www.d-mind.de
More information about the TYPO3-performance
mailing list