[TYPO3-dam] "Indexing date" issues

[Dan Osipov]

Steffen,

Yes, it is a problem. Here are some thoughts:

 > So I propose one of the following fixes:
 > * Modify TCE to set crdate only if this field of $incomingFieldArray 
isn't
 > set, yet (think bad idea)

Yea, probably won't pass...

 > * Modify the selection of indexing date: Don't only use the exact crdate
 > timestamp, but get the X next files with crdate >= indexing-crdate. X is
 > already saved in item_count (maybe a hotfix, but problematic if two 
runs at
 > the same time).

And what's the cut off time? 10 seconds? 1 minute? And yes, several runs 
are a problem

 > * Improvement of the above: Only select X files with same cruser_id 
as the
 > index run

Same issue as above

 > * Add a field (e.g.) indexingPid to tx_dam and reference the uid of the
 > indexing run.

Seems like the best option - although it does add an extra field. It's 
the only way to be accurate. Also the information about the indexing run 
can be stored separately for historical purposes (indexing reruns & whatnot)

Dan Osipov
Calkins Media
http://danosipov.com/blog/

Steffen Gebert wrote:
> Hi DAM fans,
> 
> selecting the time under "Indexing date" in Element Browser is a nice (fast) 
> feature to easily find the just uploaded files.
> 
> Unfortunately my client complains that very often no or not all uploaded 
> images are listed after selecting "Indexing date" as filter.
> 
> After digging into the problem I found out the following:
> * Selecting an indexing date selects all files which have tx_dam.crdate = 
> tx_dam_log_index.crdate
> 
> During indexing:
> * File is indexed and crdate for the entry in tx_dam is set to the start 
> time of the indexing run (so all files of the run should have the same 
> crdate): tx_dam_indexing::indexFile() gets the crdate as parameter
> 
> * The file information ($meta['fields']) is passed to the TCE in 
> tx_dam_db::insertUpdateData()).
> 
> * TCE overwrites the crdate with time()
> 
> As this is very annoying when the start of the indexing run is not in the 
> same second as indexing of all just uploaded files (which seems to be the 
> case for me very often), I'd absolutely try to fix this issue (bug report not 
> created, yet).
> 
> So I propose one of the following fixes:
> * Modify TCE to set crdate only if this field of $incomingFieldArray isn't 
> set, yet (think bad idea)
> * Modify the selection of indexing date: Don't only use the exact crdate 
> timestamp, but get the X next files with crdate >= indexing-crdate. X is 
> already saved in item_count (maybe a hotfix, but problematic if two runs at 
> the same time).
> * Improvement of the above: Only select X files with same cruser_id as the 
> index run
> * Add a field (e.g.) indexingPid to tx_dam and reference the uid of the 
> indexing run.
> 
> Personally I'd prefer the second last solution. What's your opinion?
> 
> Regards
> Steffen
>