[TYPO3-core] RFC #12232: Bug: md5_file() to check if a file has been changed is very expensive [performance]

[Steffen Gebert]

> This would call 2 "filesytem-query" + md5-hash for each file/image even
> if one (filemtime) is enough.
> I'm not an expert on filesystem behavior, but in the worst case this
> would mean something like this:
>
>    PHP: filemtime('filename')
>     FS: look up the inode for `filename`
>     FS: update atime for `filename`
>    PHP: filesize('filename')
>     FS: look up the inode for `filename`
>     FS: update atime for `filename`

That's why PHP has a file status cache (which can be cleared using 
clearstatcache()).
All mentioned functions are supported by this cache and I expect (*) PHP 
to fill the whole cache entry (which can be retrieved using stat()) and 
not only the value for a single property (like mtime or size). Thus file 
system querying is only queried once per file, independent of the number 
of called file_____() functions.

I'm +/-0 for creating new fields or reusing the md5hash field. While new 
fields would be a cleaner solution, it would leave the superfluous 
md5hash field there for an undetermined amount of time/versions.
Reusing it would cause the overhead of hashing, but the upgrade path 
would be cleaner (no legacy code for md5 fallback, only an upgrade wizard).

What also made me think is a user-comment in PHP manual [1], which says 
that Windows returns timestamps daylight-saving-time dependent, thus the 
filemtime changes every 6 months, resulting in recreation of all images 
at once.
I think, we should encapsulate the filemtime/filesize check in 
t3lib_utility_File and use it in this function. For the moment, we can 
go with the current approach (as it's IMHO too late for 4.5).

Any opinions in using the md5field for saving md5(file_mtime() . 
filesize()) or creating dedicated fields?

Thanks for your patience, André!

Kind regards
Steffen

-- 
(*) and my empirical experiments prove this assumption
[1] http://de3.php.net/manual/en/function.stat.php#58404

Steffen Gebert
TYPO3 Core Team Member

Use a newsreader! Check out
http://typo3.org/community/mailing-lists/use-a-news-reader/