[Typo3-typo3org] Hosting the TYPO3 sites
Jan-Hendrik Heuing [NF]
jh at netfielders.de
Tue Apr 12 01:43:04 CEST 2005
As Robert already wrote, we had a chat together on Saturday morning. Here is
a summary for all the others who did not participate ;)
Some details about history:
Roughly a little more than a year ago, netfielders did a lot of experiments
with clustering MySQL. Actualy we already fixed a deal with EMIC Networks
about licenses for a MySQL-Cluster.
We had successfully set up a MySQL cluster which in the end did not work
because of some strange incapabilities of that product in relation with some
Although there could have been a chance to fix this on our side by modifying
TYPO3, our requests have not been taken that serious that time.
--> Analysing .org sept. 04
After the forecast when doing those cluster-experiments, we finally figured
out that the only way would be to change typo3.org. As there was still no
changes in pipeline from Kaspers' side, we had a look at typo3.org in
detail. We figured out a couple of problems including that TER saves it's
extensions in a database table etc.
At this point we reported this to DH, Rob and Kasper. Unfortunately we got a
few comments feedback but no active help which helped us to solve those
problems. Maybe it was because of a lack of time, but anyway - we really
tried to solve the situation but have been slowed down by missing help from
the core devs.
Actualy we offered one of our employees for some limited time (a couple of
weeks if I remember right) fulltime to fix a few things, even extracting
ext-files from the table, doing that in the filesystem would help'ed a lot!
Well, after asking a few times, there was no feedback on this!!
--> Setting up Proxy
Last December, after nothing really happened, we started to enable a proxy
server. We had it running perfectly, but it didn't cache any pages, just
images. Why ? Because headers are not send, which are needed for caching. So
we figured out that it does not only cache in the proxy, but also not at the
client-level. That means, that EVERY request a user sends to typo3.org is
seriously send, instead of caching that page.
After some discussion with us and some other people, Kasper finally became
aware of the cache-control headers which are now implemented in TYPO3 3.8.0.
Result of this was that since December we want to update. After asking every
once in a while, implementation has been finished. Asking Kasper for a
timetable (while already discussing 3.8.0), I got the only feedback "no
This time, Rob already was in charge of typo3.org, but because of missing
communication, we didn't know about that and kept updating other people
until Rob complained....
--> SSH access (see below, one of Rob's concerns) + some infrastructure
Great, that there was no communication between DH, Kasper and Rob, as I did
tell DH and Kasper a few times about the status what we wanted to do:
Right now typo3.org is hosted on webservers, db-server, file-server,
loadbalancer. There is no access to the webserver at the moment, because
than there would also be access to the fileserver straight away. There was
no plan to open up the webserver via ssh for anyone beside us, only this
strategy gave as the security we wanted. To access the webservers, you'd go
through the loadbalancer, which you then would have full access to as well.
So, the idea was to also move those webservers to the virtuozzo platform,
which we were already running on other servers. Virtuozzo separates between
the real hardware node and virtual servers. So you can use i.e. redhat or
debian within the virtual server, while you can not break the system on the
node level (and other virtual servers). Virtuozzo is something similiar to
the Linux vServer project. It separates the hardware node from infividual
implementations within virtual servers.
This way Rob could have full root-access to a virtual server which makes use
of the whole server performance, as there would be no other virtual server
running. (btw: www.sw-soft.com for details). In case of people making things
not work (apache, whatever), we could always just shut down that virtual
server, start a new one (which could be pre configured) and here we go.
Files and DB would still be external. You do not need to access those via
SSH, you can always access those via remote (nfs and mysql).
--> new Platform beside typo3.org:
Beside what I just wrote, here some details about the rest of it: typo3.com
and subsites like gov are already running on virtual servers like that on
separate hardware. This is after splitting up typo3.org and .com soon ago.
So Rob has got access to a control center to add new virtual servers,
applications etc. with a few clicks, beside having access to the virtual
server in any way he wants (including root-ssh). Lists.netfielders.de are
also running on of those, but using debian as virtual server. BTW: Thanks
Mr. Stucki for the huge amount of time you spend on the listserver though
the last years. The list- and news-server is also running on a virtual
server which is debian based. Through the last year, Michael had full root
ssh access all the time!
Statistics: We added those sites to a urchin-log-analyzer. We will add those
individual sites like edu.typo3.com as well. Beside that we'd install UTM an
all those pages, so that we get a detailed analysis of screensizes, better
understanding of clickpath etc.
--> Cooperation in general, and hosting of .org:
After all those things we did, after more than a year without cooperation in
anyway. It's like setting up the stats and sending access and links to it 4
or 5 times, what is the reason for this? Of course people were busy, people
set priorities to other things, but still.
If there is no cooperation like it has been for a quite long time, then I am
not interested to get on with hosting typo3.org. However it seems as if Rob
would take the chance to change this to something good. In this case I'd do
anything to keep this going on our servers. The debugging session already
was very productive and efficient. Thanks Rob.
BTW: We never advertised with it or anything, it has always been dedication
It's not a matter of the hosting platform, but rather depends on
architecture, which you all know now, is a huge problem. Just some figures:
typo3.org produces roughly 3-4 times the traffic than .com. While typo3.org
would need like 4 webservers, 3-4 database servers (as cluster) to run
alright also in peak times, typo3.com is hosted on a single server which is
not loaded at all.
next steps as internaly communicate (=not on the lists), this is what we
(Rob and I) will do:
- finding cache BUG in 3.8.0
- update to 3.8.0 after finding the bug
- enable proxy, at the next possible weekend after updating
- running for about 2-3 weeks to make sure proxy is stable, time to tweak
- some time after running the proxy, after we make sure there are no
problems, we will reinstall the webservers to make use of the virtuozzo
platform, so that Rob has got real access for the next updates or debugging,
which will hopefully not be needed very soon.
- Beside proxy-teaking, installation of webservers, we will set up new
monitoring on a separate server (US) on a separate virtual server, parallel
to one in the same infrastructure. We should think about the status page in
this case. Right now we use Mesh-monitoring, which does not check for
correct results, just for http.
Details of this will have to be fixed, one thing depends on the other in
some cases, but the next steps are already fixed.
This is a _rough_ overview, not going into details, and not writing about
all those details in between...
Thanks for listening,
PS: If people feel that I do not like them, this is mostly not the case, it's
more that I do not like things which has happened through the last month and
More information about the TYPO3-team-typo3org