[TYPO3-dev] EM, TER2, XML and other animals

[Ries van Twisk]

JoH wrote:

>>>>TER transforms extension list into XML
>>>>TER sends XML (as zipped file)
>>>>Client parses XML into array (after unzipping)
>>>>
>>>>Future:
>>>>
>>>>TER transforms extsion list into array
>>>>TER sends serialized array (as zipped file)
>>>>Client unserializes array (after unzipping)
>>>>
>>>>No need to handle, parse or do anything client-wise. Just forget
>>>>about the XML. It's unnecessary overhead.
>>>>        
>>>>
>>>You don't understand the problem. The array is too big to
>>>unserialize at once.
>>>      
>>>
>>As far as I undertsood it's too big to parse with the XML parser. But
>>if it's so large that even unserialzing fails the more I'd think of
>>another mechanism to distribute the data.
>>
>>But if nobody comes up with another idea, I'd use your version (with
>>a few comments why all of this splitting, etc happens)
>>
>>The only request is that the code should use uinique temp. names.
>>Neither extensions.xml.gz nor extensions.bin are safe.
>>    
>>
>
>Well - I think it could be better to drop both: XML _and_ serialized Arrays.
>
>As far as I can see there is a simple table containing one line for each
>extension.
>So why don't you send it as simple CSV values without the formatting
>overhead of XML or serialized Arrays?
>To split up the whole process into smaller chunks it could be a solution to
>use different CSV files for each category.
>
>So the first CSV file would setup the categories as an Array but only on the
>client side.
>While processing this array the client would load and process the necessary
>CSV files for each category separately.
>
>Just an idea
>
>Joey
>
>  
>
Actually I don't understand why we need to fetch the complete XML at all???
What is wrong with sending a query to TER and fetch the result in XML.

We just talk about 2000 records in a database (even if there where 
20.000 records), that should be doable in a snap.
I search for news, and I get a XML feed with all extensions with news in 
it (limited to let's say 200 extensions).

That XML can be parsed quickly with no big memery requirements and 
displayed.
I know they have been doing this to get the load of typo3.org, but 
really this is the wrong method.
Any database can handle and send at least 20-40 queries/sec, if 
bandwidth is a problem then zip it up and send (like most webservers do 
anyways).

Compare this : When I want to see a page on typo3.org, my browser 
doesn't fetch a recursive version of the
complete typo3.org, doesn't it? it just fetches the part I am interested it.

just my 2 cents..

Ries




-- 
Ries van Twisk
Freelance Typo3 Developer
=== Private:
email: ries at vantwisk.nl
web:   http://www.rvantwisk.nl/freelance-typo3.html
skype: callto://r.vantwisk
=== Work:
email: ries at livetravelguides.com
web:   http://www.livetravelguides.com