[TYPO3-mvc] QAW: Simple question!? Filter List with localization...

Mon Jan 20 13:31:00 CET 2014

Hi Stephan,

very nice summary.
It shows the complexity of the involved methods.
One way to solve the overlay stuff could be to do it in stored procedures on the DB. Hence you can apply filtering etc on the result of the procedure.

One thing I'd like to add: We do not always do overlaying in language processing, it strongly depends on the TS settings. So never assume the user of an extension actually does use overlaying. ;-)

Kind regards
Markus

------------------------------------------------------------
Markus Klein
TYPO3 CMS Active Contributors Team Member

> -----Original Message-----
> From: typo3-project-typo3v4mvc-bounces at lists.typo3.org [mailto:typo3-
> project-typo3v4mvc-bounces at lists.typo3.org] On Behalf Of Stephan Schuler
> Sent: Monday, January 20, 2014 11:40 AM
> To: 'Alex Kellner'; 'TYPO3 v4 MVC project'
> Subject: [TYPO3-mvc] QAW: Simple question!? Filter List with localization...
> 
> Hey there.
> 
> 
> I faced a problem which might be related to this. Form a very superficial
> perspective it this could be the case for what you experience.
> 
> The thing is: The Typo3DbBackend limits queries to "sys_language_uid IN (0, -
> 1)", which is "default language and 'all'", but no translation value and no
> "language record which isn't a translation".
> 
> I do have a change for this, but since it's a very central behavior of the
> current Extbase and changing it would be very very breaking, I think we
> cannot call it a bug even if it might be one.
> 
> My solution for now goes by "xclass" the Typo3DbBackend. Xclassing stuff
> always is pretty ugly, but I didn't find a better solution until now.
> 
> That's the TypoScript:
> >
> config.tx_extbase.objects.Tx_Extbase_Persistence_Storage_BackendInterf
> > ace.className =
> Tx_Yourextension_Persistence_Storage_Typo3DbBackend
> 
> And here's the PHP stuff:
> > http://pastebin.com/GiXscvx0
> 
> 
> This doesn't really *solve* your problem because you have a translation
> (sys_language_uid=1, l10n_parent=0) instead of a "not-translated foreign
> language record" (sys_language_uid=1, l10n_parent=$parentRecord).
> But maybe you find a solution based on my explanation.
> 
> 
> Think about this situation:
> 
> Record 1: uid=100, sys_language_uid=0, l10n_parent=0, title="a" - default
> language, so no translation.
> Record 2: uid=101, sys_language_uid=1, l10n_parent=100, title="a" -
> translation of the first record Record 3: uid=102, sys_language_uid=1,
> l10n_parent=0, title="a" - not a translation, but originally created as foreign-
> language record
> 
> Search for "a" in sys_language_uid=1.
> 
> The default behavior of would resoult only uid=100 because it matches, and
> do the language overlay to uid=101 after that.
> My change directly finds uid=100 and uid=102. The uid=100 will be overlayed
> by uid=101 just like normal.
> 
> From this example you can see: Both "SELECT" statements do not directly
> search the uid=101-Record. The 101 is only returned as an overlay of 100, but
> not covered by the raw SQL.
> If you extend the "addSysLanguageStatement" to cover the uid=101-record
> as well, you will find all three records -- which is plain wrong, too.
> 
> So a generic solution isn't that simple.
> 
> The main problem (imho) is, that there is partial overlay caused by TCA
> (fallback fields). This always requires PHP side post processing. If your SELECT
> stuff (no matter if it is high level like QOM) targets values that are only
> calculated (fallback fields that are taken from sys_language_uid=0 because
> they are either not translated at all or "mergeIfNotBlank") and hence this
> requires PHP processing, that thingy completely destroys lazy Query
> execution and query processing which is a central feature of e.g. pagination.
> I really hope I could explain the problem properly. It's a very annoying show
> stopper because this means calculating SELECT properly and performance
> boosts like pagination will never occur at the same time :(.
> 
> A really generic solution would create TCA based view on the database and
> throw a SELECT statement. But I doubt anybody will transform TCA to SQL
> just to make SELECT against it. There are ~6500 lines of code in the TCEmain /
> DataHandler, and I guess ~2000 of them would be required to be migrated to
> a huge "TcaSqlFactory". That's way out of scope, imho. This doesn't stop with
> language. If you have versioning/workspaces, there's another layer of
> complexity which goes by the very same principles  -- overlaying that needs
> to be done in the database to have generically processed scope to apply
> filters to.
> 
> 
> Regards,
> 
> 
> 
> Stephan Schuler
> Web-Entwickler
> 
> Telefon: +49 (911) 539909 - 0
> E-Mail: Stephan.Schuler at netlogix.de
> Website: media.netlogix.de
> 
> 
> 
> --
> netlogix GmbH & Co. KG
> IT-Services | IT-Training | Media
> Neuwieder Straße 10 | 90411 Nürnberg
> Telefon: +49 (911) 539909 - 0 | Fax: +49 (911) 539909 - 99
> E-Mail: info at netlogix.de | Internet: http://www.netlogix.de
> 
> netlogix GmbH & Co. KG ist eingetragen am Amtsgericht Nürnberg (HRA
> 13338) Persönlich haftende Gesellschafterin: netlogix Verwaltungs GmbH
> (HRB 20634)
> Umsatzsteuer-Identifikationsnummer: DE 233472254
> Geschäftsführer: Stefan Buchta, Matthias Schmidt
> 
> 
> 
> -----Ursprüngliche Nachricht-----
> Von: typo3-project-typo3v4mvc-bounces at lists.typo3.org [mailto:typo3-
> project-typo3v4mvc-bounces at lists.typo3.org] Im Auftrag von Alex Kellner
> Gesendet: Montag, 20. Januar 2014 10:59
> An: typo3-project-typo3v4mvc at lists.typo3.org
> Betreff: [TYPO3-mvc] Simple question!? Filter List with localization...
> 
> I ran into a strange problem with extbase and maybe someone knows the
> answer.
> 
> *Configuration:*
> * Let's create a simple Object with just one property (e.g. title)
> * Configure TYPO3 with &L=0 for german and &L=1 for english
> * In addition add one record in the backend list view and localize it (see
> attached image) - I wrote "deutsch" and "english" in both titles.
> 
> *Extension:*
> 1) Add a FE Plugin with a controller and make a findAll() in the controller. You
> will see the germany record in L=0 and the english record in L=1 - so far so
> good
> 2) Now we want to add a filter field over the list. Let's create an own
> repository method:
> <pre>
>         public function findByTitleTest() {
>                 $query = $this->createQuery();
> 
>                 $query->matching(
>                         $query->like('title', 'eng%')
>                 );
> 
>                 return $query->execute();
>         }
> </pre>
> You will not find a record with "eng%" in &L=1 but you will find a record with
> "deu%".
> 
> Tested in TYPO3 4.5 and 4.7
> 
> Is this a bug or do we need to do the localization by our own with
> setRespectSysLanguage(FALSE);?
> 
> My Test extension as t3x file for download:
> www [dot] in2code [dot] de/fileadmin/user_upload/T3X_tt_news-3_4_0-z-
> 201401201049.t3x
> begin 644 loc.png
> MB5!.1PT*&@H````-24A$4@```DD```!W"`8````7;]3``````7-21T(`KLX<
> MZ0````1G04U!``"QCPO\804````)<$A9<P``#L,```[#`<=OJ&0``!%+241!
> M5'A>[9V+;U37G<?G3TD(HA6*5(0L^@@1*ZJL4Q6"*:$@P&`Z3LHBW`!-(6D*
> M)-"D!C^@@J1KQ&Z#@<2D+HX11G;!#]HJ:DI=A(PQ=M:ED$VR-
> DMX!Q(2?KWG
> MON;,G3-WQN-
> A/&8^'^D3W_L[CWN5N=+YYMRQ$[ESYXX@(B(B8KR$)$1$1$2#
> MA"1$1$1$@TE#4OT[K8B(B(@%:VA(VK1I$R(B(F)!2DA"1$1$-!@:DC9NW(B(
> MB(A8D(:&I`T;-B`B(B(6I*$AZ:677D)$1$0L2$-#THLOOHAYY);:-T?DTPN7
> M&^=!1$3$U(:&I!=>>`'S2!5\[MV[E[8J))GF041$Q-2&AJ1UZ];A*/U=G]CT
> M_<[<[AOK:&ZW]$+2E2LW4WJF[[P=DDSS("(B8FI#0]+SSS\_/MUQ0BXYD4/Z
> M#CJU'2=4Y9*<V!'H>S\]Z`0?[QY\O?OK.QA?3];?-9.09)H'$1$14QL:DM:N
> M79O4EU]^66[?OFTOZD%47;6;QN7$[;&0)')6&JS:=B\D;3?TOR]N%^>2)V1[
> ML.9QMD'KGVQ,S&0AZ;$9Q0DU+R0%YT!$1,3T#`U):]:L";6AH<%9[`.HNJE_
> MSJSMDF%U(\/#]L_AKEJI[;*/I*O6[=-P5O6(80465??*9QMB\]C'L0:K7ZW8
> MTVG8??1[6-- at Q3.[(5"WC)LKOLUILH)=H*[,)"0%YT!$1,3T#`U)SSWW7$K[
> M^_OM]=Y#G9OZY=3J3B<D];XE;_6J at V'I[75"4F>UU>X49;BSVNY?W>DDGMZ
> W
> M8FUJK%?7CU4?[]@;;]2]!V,?[1K!-F=N]SX#;<&0I,)1T&!("LZ!B(B(Z1D:
> MDBHJ*E*J_B*EZJ]0/]6YJ5].K>IP0](!Z_R`N)'$8E at ZJBJDJL,).4&&.ZIB
> M_8<[I,,ZZ.VU_N$>6V=RP+Z&/J=#[P'S/3AS!MH.N*/M^XMO<^[-N<]@6R8[
> M2<$Y$!$1,3U#0]*J5:O2\NVWW[;7?/73U)YSM[DAZ<Q^^WR;'XJL\+'-:M]_
> MQCEUVX-ZS4[__>*?FOH'KA73&3?<L2U0MPRYOG.O9V1_H*[,)"0%YT!$1,3T
> M#`U)*U>N3-NVMC9C?4RL;)<A.X3L<VO[W*`S).V5;I]]?O3Q.;,OV'9&]JVL
> ME'9[,FMT>Z7=7ND5?%0_=ZRO.VZH72J#M3BT>S*.B9DL))GT0E)P#D1$1$S/
> MT)"T8L4*'(WU/78,ZJDWM)E,T3^3D&2:!Q$1$5,;&I*>??99'*5[G=PC/7O-
> M[;ZQCN9V2Q62SE_\V`Y`Z:A"DFD>1$1$3&UH2'KFF6<PCU0A206?D6B:!Q$1
> M$5,;&I*BT2 at B(B)B01H:DI8O7XZ(B(A8D(:&I+*R,D1$1,2"-#0DE9:6(B(B
> M(A:DH2%IR9(EB(B(B`5I:$@"````*%0(20`````&"$D`````!@A)``````8(
> M20`````&"$D`````!@A)````^4!+5"*1B&61U`RXM1S1$HU(M,4]R3M:)&K_
> M>[$LJI%<_JLA)`$``.0!B4%E0&J*<A&85`B)6O^T&*B1(N\X([2Y%*.>3T.%
> MR!PGN1&')/5_HA^IZO]&#P```,DP!"(5,'*Q<Y+-Z]S'>QZ+W:Z,0M*]>_=&
> M)"$)```@!%.X"-DY48'!>367^`HJKLW;Q?%?Y07J%@,U1?YU8L<JM.G]M3%)
> MY])>BUFJ:?2Y%<GN6_4KBD:ER!\?W'W*U:Y:/!F')!5\TI&0!```D`)#(%+!
> M(5"R<,)+D9\6G'.OGPHAL39U'@P;#OJN3++C&/'7"!(V/G:>^KYCH<EPO5SM
> MJ at 7(."1YK])263 at AJ5U^-G&B/#QQCM0-NB4``(`T2`PG*B at 8=DY4F`J$!7L7
> M1G5,&B2<T.'OX-AZ<^O7,5U3[0[IM73G4FCG8?<=-L[#$")S`:_;+`9WS['"
> MC0HXB<[>K1)/,`"9`A$A"0``,L$0"NQPDK at +%`L6'FJL$[`2VQQ4`(NO6W-[
> M at 44%*^\ZP9"EM[FDG"LXWCT/N^^$<6K.P'75^#'(2)F'I!/OG;*/4_T<#R$I
> MAA=TULEQMV*&D`0``-G"$(B2[9P$=F14>/#/@[LUUGFT10LC+O:K+:^@7R=X
> M'!=<%&G,%;A^LK:$^XZ?-/[<OFXP1.:&K.\D_<>!D at 3'?TC2`]"@U,U1QYKK
> MV at -]G%&Q6DQG9PH``,`E(12X(2+NE59L!\<.)EX],"ZNS0M>:GZ_9LUC!0YO
> M+GV'QS^V=Y#T>92IY]+'J=O2YU8DN^]@O^"Y-=*XJY8+LKJ3].-]3\EK+6MD
> M;N5W?-6YJH\/T at E)IG-%DCYS=HO3)1:N?J8R%0``@$5B*(`X#"$R5V1M)ZG\
> MS>_;@>B%0S^2>573[=J\;=/M<U4O?W.6.T,^D\60U+[.#D0FV4T"```'YQ66
> ML\,R-KLE>8NV,S56(3)K.TD_VO,]^>'V&9;_)@LL59\%UKDZ5C^7[WG2G2&?
> MN0\AR=])`@``@/%$UG:2EO[GO\OB7=\UMJFZ:L]_T at U)WJNSD)#DG\>_7C
> N^
> MCM=M````XX&L[20MVOE=*:LKEF7*W<5V'_O8==&NF>X,^4RZ(2GP9P.2?G';
> M\"5O2T(2``!`_I.UG23UJDU]]^C`7W\CT?_^OEU3/]7YJT?7V*_<`````,8+
> M&84D]2O]1JNGRX;F%?9OL_VXWG'CX17V%[D!````QA,C#DFIT'_]?^Y6Y;?=
> M%@```(#Q0]9#$@```,"#`"$)````P``A"0```,``(0D```#``"$)````P``A
> M"0```,``(0D```#`0&A(ZN[N1D1$1"Q(V4D"````,$!(`@```#!`2`(````P
> M0$@"````,$!(`@```#!`2`(````P0$@"````,)"5D/1ZUTVI^L-5V71X6%8W
> M7'2K`````..7K(2DJK:K\OZ9J_+'[D^E;,\YMPH```#CFH$:*8I$)!(IDIH!
> MMS9B6B0:B5K_'']D)21M>'=(NDY>EM8_#<O<FK^[U7PB'SZ@\?N0``#`>&1`
> M:HI4P-$=2=A1XT<3CCRT]4^%KJ(::^;Q059"4L7^"[*LKD]*JKMEYL8_N]5\
> M(@L!9=0?+"$)``!RB2GDJ+4H(D5I)9]LK5L%'I+R'T(2```4&LEV at M1ZI-7]
> M5VJ.7H!JB6H[4/;ZE[@S%?47M>`:IY][QX'QL<%Y2]9"TM&C1Z6YN=D]RP-:
> MHG$?9$3_\)(\$,D_9/,'&_<`I7S at G+FBVICTDCP``$`F)`M)SOKE+&7!=4\?
> M$VP+8*]U7GNR]3-P/.H-A]R244CJZ.B0X\>/2UM;FQPY<D0:&QNEOKY>ZNKJ
> MI+:V5C9OWBSKUZ^7BHH**2\O=T?E$O6!Z`]&D at _+)NR!"/E at 50C34[!U'GO
> @
> M`M>V&U1=2]UQ#Q<``$"V22,D)6PH.,;6L\`ZE70#(MA7/]>."R$DM;:FO\M4
> M6EKJ'N608(#1/Z`1/1`A'ZP=<O0'Q"7AVA[!N9,_O````*,GV3H36!.-:Y8B
> ML&[9ZZ>^CNGMP34N25LAA*2\)U5(2O>!T,^3?K"JCQ:T"$D``)`7F-899\V*
> M_YJ)]I;#8J"FR+QQ$%S?XD*3ZJM=*Z%-6TOCUL+\9D0A:?6)J%1TELG*]J6R
> MXMABMRJRK.5I6=@\6TK>*98G]\UPJR)3MC\JDRN_)I.V/"*37IG at 5G-`\$-(
> M^+#"'H at D'W)@SM at 8!W5N/W1V/_VAM.:T.VH/B0TA"0``[B=JG8E_:V+\$P#
> V
> MNJ7WT==+?=U2RZ+6+QJ-:U?KH+DM?AY_#GT1S5-&%))6=2R3YH%&:>H_*-
> '6
> M!6Y59,&[L^2-O]5*]5^VR(P]WW*K(I-_-4D6-<V3A8?FRL1-#[O5W)#\P[)(
> M^D"$CXO[8(-S&%_%.>I?W(X]$H0D``"`?&9$(6G%L25RZ%R#-/8=L'>//)YJ
> M>$(JW]LDO^A:*]-V37&K(I,V3Y`%OR^1^8USY)$-#[E5````@/QG1"&IO'6A
> M+#\Z7Y8>F2>+#Y>X59$G]CXNC^^>)M-V3I%O5$]VJV+O'DW<^)`=D`A)````
> M,)YX,+^X#0```#!*"$D`````!@A)``````8(20`````&"$D`````!@A)````
> M``8(20`````&0D/2[=NW$1$1$0M20A(B(B*B04(2(B(BHD%"$B(B(J)!0A(B
> M(B*B04(2(B(BHL&\"DFO=]V4JC]<E4V'AV5UPT5C'T1$1,1<F%<AJ:KMJKQ_
> MYJK\L?M3*=MSSM@'$1$1,1?F54C:\.Z0=)V\+*U_&I:Y-7\W]AD_-DE9I$R:
> M$HZ3F4X?1$1\T+QV[5J"5ZY<D\L??23'(A%ILQSZX`,9&AJ23S[Y1#ZRZA]^
> M^*%<N'!!SI\_;YP3LV->A:2*_1=D65V?E%1WR\R-?S;V&3^.-/00DA`1"U$5
> MBCSNW;OG^Z6U%E]<NU;^^9.?R-
> U;M^3NW;ORQ1=?R.>??VZOTVKLX.!@PGR8
> M/?GB]GV3D(2(B*GU0I(>D+[ZZBNY^]EG,O"#'\BYV;/E\^O7X\+19U:;^MG?
> MWQ\W5X(]6V5J)"(1UZE;>]PV9\TI*S.UH6?>A:2C1X]*<W.SL>V^FNF#%!@7
> M\8..'GKB`U"3-E<D,E6V]L3Z\,`B(A:6*B0%`Y+RBUNW[(#45_RDW+EQPUZ;
> M57\5D&Y9;>IG7U]?PGPQX]>>V[=[9.M4?<V)2%F3VV:O9?R'>M`Q#4D='1UR
> M_/AQ:6MKDR-'CDAC8Z/4U]=+75V=U-;6RN;-FV7]^O5245$AY>7EQCFR8Z8/
> MDFKS^GGG>IOAN*E,(F5-
> ;G_GW)D[[#J(B/B@>GK^?+EBK8'_OW^_#._=*T._
> M_:T,[=DCG[S^NIPM+I;>F3/EPZHJN6CYS\I*^<>KK\K@*Z_(A9T[Y6UKW3#-
> M::O6&ZL]:&S-T=<8?=U#SS$-2:VMZ5^CM+346,^*F3Y(P<"3+!CIQ_[.DSYG
> ML+^2!Q81L1#LM-:$B^O7R_\L6B0?S/^A#,R=*^=FS?(#4L_TZ7+ZF]^44]::
> MT/WHHW+RZU^7OTZ8(*>MMAIKK&E.VX0U2I<U)QWY3I(RTP<IDY`4US?-
> ,!8W
> M#A$1'R3?GS%#+A\\*/_WQAOR\:Z=\O&.'?*_-35R\;77I.>QQ^3TM&GRCY__
> M7`8M/[#"5/]/?RKG5J^6P5_^4OXK+"2YZXS_AL*R9^M4UIP1."8A:?6)J%1T
> MELG*]J6RXMABO[ZLY6E9V#Q;2MXIEB?WS?#K4[8_*I,KOR:3MCPBDUZ9X->S
> M9X8/4O"56,*KN,3CV+R.ZMSY[A$/+")B(7KY\F7Y\LLO[=]>TW^#[?:U:W9`
> M.C5EBMS\]%.Y>?.FW+AQ0ZY?OVZKCD^=.F6<T]=_>^%I6J.4K#DFQR0DK>I8
> M)LT#C=+4?U"BK0O\^H)W9\D;?ZN5ZK]LD1E[ON77)_]JDBQJFB<+#\V5B9L>
> M]NM9-<,'284<?XQ53Q62$JXS=:OT)/17\L`B(A:"ERY=LD.2"D?ZK_A_9H4D
> M%9"Z)T^6ZU8?+R!Y?TM)'9\\>=(X)V;',0E)*XXMD4/G&J2Q[X"]>^35GVIX
> M0BK?VR2_Z%HKTW9-\>N3-D^0!;\OD?F-<^21#0_Y];Q3O7[S0P\B(F)JU1^)
> MU'>0_%_UMX)0[ZQ9TE-
> <+#<N7]9VDF*[282D^^N8A*3RUH6R_.A\67IDGBP^
> M7.+7G]C[N#R^>YI,VSE%OE$]V:^KW:.)&Q^R`U)^A22UVZ/M"L7M!"$B(J;6
> M]%>TU1^)[+,"T&^LM>77EJ<[.^7TZ=/VZ[53W=UV./(TS8G9D2]N(R(B(AHD
> M)"$B(B(:)"0A(B(B&B0D(2(B(AHD)"$B(B(:)"0A(B(B&B0D(2(B(AH,#4F(
> MB(B(A6K2D(2(B(A8R!*2$!$1$0T2DA`1$1$-$I(0$1$1#1*2$!$1$0T2DA`1
> B$1$-$I(0$1$1$[PC_P*;]=],ZY8-_0````!)14Y$KD)@@@``
> `
> end
> 
> _______________________________________________
> TYPO3-project-typo3v4mvc mailing list
> TYPO3-project-typo3v4mvc at lists.typo3.org
> http://lists.typo3.org/cgi-bin/mailman/listinfo/typo3-project-typo3v4mvc
> _______________________________________________
> TYPO3-project-typo3v4mvc mailing list
> TYPO3-project-typo3v4mvc at lists.typo3.org
> http://lists.typo3.org/cgi-bin/mailman/listinfo/typo3-project-typo3v4mvc