[Typo3-typo3org] Hosting the TYPO3 sites

[Ingmar Schlecht]

Hi Robert,

Robert Lemke wrote:
> As all of you know, we currently have a variety of difficulties with
> TYPO3.org. Basically I'd boil these down to the following:

First of all, I'd like to say that I found it always amazing to see the
immensive amount of pageviews on typo3.org, considering that TYPO3 is
#12 project on sourceforge, as far as pageviews are concerned. So even
if it's been unstable quite some times, even that has been an enormous
thing.

>   a) the technical design for the documentation section and the TER causes
>      too much load, a redesign is neccessary

Definitely, I think that the technical design of the current extrep is
far too resource intensive.

>   b) the current hosting solution can't cope with this load and doesn't 
>      scale well, at peak times TYPO3.org is partly unusable and the TER is
>      not reachable. We don't have detailed information about the reasons
>      when load is too high or the site fails generally.

While you might not have detailed informations about the reasons of the
failures, I think the problem is rather obvious: The bad design of the
current documentatoin and TER functionality.

>   c) debugging TYPO3.org is quite hard for me as I can do it only while I 
>      have contact to someone of the netfielders staff. I currently have:
> 
>        - a shell account to the TYPO3.com server which has the filebase from
>          TYPO3.org mounted into a directory
> 
>      what I definately need for debugging is:
> 
>        - control on restarting the webservers and flushing the accelerator
>          cache
>        - an (S)FTP account for the TYPO3.org filebase
>        - access on the PHP error log and the Apache logfiles

...and Jan-Hendrik is not going to give that to you?

> The solutions we planned for these task are currently
> 
>    a) - short term:
>           - refactor the current TER to be file based

I guess that will take most of the load away from typo3.org, which means
most of the problems will probably already be solved by this.

>           - disable other features on TYPO3.org which cause much load
>           - upgrade to 3.8.0 and use caching headers / reverse proxy
>       long term: 
>           - create a TER 2.0

What is the difference between TER 2.0 and "refactor the current TER to
be file based"?

>           - create a different solution for the documentation section
>           - split up TYPO3.org into teams.typo3.org, documentation.typo3.org
>             etc.

> Another very interesting offer comes from Jürgen Egeling from punkt.de: He
> does not only offer to invest quite some money and time into setting up a
> professional and reliable hosting solution but also invites one or two
> developers to come to Karlsruhe for planning and developing the new
> extension repository. 

OK, but in which way is the "professional and reliable hosting solution"
different to the current hosting? Wouldn't a switch to another server
cause even more short-term problems?

> He would pay half of the daily rate for these
> developers and provide working time from hist PHP developers and sysadmins.

Does punkt.de get anything in return or is it just because of their
commitment to TYPO3 (like Jan-Hendrik's current offer)?

>   - keep the mailing lists at netfielders if they really put effort into
>     switching to Sympa or making them controllable via SOAP in a certain
>     way.

Didn't mailman work great so far? Is Sympa as reliable?

>   - Distribute hosting of the different sites instead of having one provider 
>     for all. In particular I'd like to see the following distribution if all
>     parties agree:
> 
>     netfielders: www.typo3.com, edu.typo3.com, gov.typo3.com, 
>                  news.typo3.org, typo3.netfielders.de

Those are the "easy ones" as I understand it. Wouldn't they be the sites
that are more easy to move away from netfielders to somewhere else?

>     ELIOS:       bugs.typo3.org, association.typo3.org, wiki.typo3.org

OK, they are already at Elios, so I think it's reasonable to keep them
there.

>     punkt.de:    all remaining *.typo3.org, demo.typo3.com, 
>                  lists.association.typo3.org

>   - Do you agree with this decision?

See above. I don't see in which way hosting at punkt.de is more
reliable. As far as I see, the problem of the current solution is not
the Netfielders hosting farm but the Extrep software. If that is solved,
I think the other problems will vanish automatically.

>   - Should anyone else be asked before taking this decision? We need some
>     really trustworthy people and a quick and future-proof solution at the
>     same time.

I think Monday is a bit too early to make this desicion, what do you think?

cheers,
Ingmar