[Flow] TYPO3 Surf questions
Mark Kuiphuis
mark at capesso.com.au
Mon Nov 10 00:45:37 CET 2014
Hi Lienhart and Christian,
Yes, I believe either Christian or Beat has already mentioned that the
composer.lock has to be under Version Control. After I sent the last
message to the newsgroup I sort of figured that out myself as well.
This project we did was a bit weird as it started out as a "simple" Join
Online form. But then we found out that the information from this Join
Online form was of better quality (normalised, etc. etc.) than the data
which was in their old system (and we initially had to migrate this data
to). So, we kept adding functionality to turn it into a full blown
membership system. The customer already started using the system last
April, as our system (even under heavy development then) was already
better suited, according to them, than the system they had been using
for years prior. But with systems under heavy development, things will
go wrong and will break. That's why the first couple of months, daily
updates to the live server were very common. Luckily not anymore, but my
mindset had not been adjusted yet to the lower update frequency. :-)
Thanks everyone for their help. I wouldn't have been able to figure it
out myself....
Cheers, Mark Kuiphuis
P.S. To me the thread was anything but confusing. It helped me a lot :D
On 7/11/2014 7:32 pm, Lienhart Woitok wrote:
> Hi Mark,
>
> this thread is now a bit long and confusing and I'm not sure if the
> following has already been mentioned or not.
>
> Surf does a ´composer install´ internally. ´composer install´ looks for
> a file composer.lock and if that exists checks out the exact version or
> revision that is recorded in that file. So if you update one of your
> packages and push a new revision to the repository you also have to
> update the composer.lock file (I think someone already stated that this
> file has to be under version control). To update the composer.lock file
> you can simply run ´composer update your/package´. If you do that and
> specify your dependency as "dev-master" (or whatever your branch is
> called) there should be no need to set minimum-stability to dev.
>
> Hope this helps.
>
> Regards
>
>
> Lienhart Woitok
> Web-Entwickler | netlogix Media
>
> Telefon: +49 (911) 539909 - 0
> E-Mail: Lienhart.Woitok at netlogix.de
> Web: media.netlogix.de
>
>
>
>
> netlogix GmbH & Co. KG
> IT-Services | IT-Training | Media
> Neuwieder Straße 10 | 90411 Nürnberg
> Telefon: +49 (911) 539909 - 0 | Fax: +49 (911) 539909 - 99
> E-Mail: info at netlogix.de | Web: http://www.netlogix.de
>
> netlogix GmbH & Co. KG ist eingetragen am Amtsgericht Nürnberg (HRA 13338)
> Persönlich haftende Gesellschafterin: netlogix Verwaltungs GmbH (HRB 20634)
> Umsatzsteuer-Identifikationsnummer: DE 233472254
> Geschäftsführer: Stefan Buchta, Matthias Schmidt
>
>
>
> -----Ursprüngliche Nachricht-----
> Von: flow-bounces at lists.typo3.org [mailto:flow-bounces at lists.typo3.org] Im Auftrag von Christian Loock
> Gesendet: Freitag, 7. November 2014 08:00
> An: TYPO3 Flow mailing list
> Betreff: Re: [Flow] TYPO3 Surf questions
>
>
> Well I guess it is either that or you create own branches for your
> stable versions. That is the purpose of minimum-stability, to prevent
> you from accidently installing unstable code.
>
> Am 07.11.2014 um 01:45 schrieb Mark Kuiphuis:
>> Not really no, in answer to your remark about "if you want to use @dev
>> stability". :-)
>>
>> I would like to get it working properly first and I can then always
>> finetune and optimise.
>>
>> I tried the minimum-stability option, but this is then also applicable
>> to all TYPO3.Flow packages and its dependencies (doctrine, migrations,
>> etc. etc.) Not a good idea for a production environment...
>>
>> I am now going to look into creating tags in our own packages...
>> Instead of using @dev or dev-master I suppose then have to use "0.1.*"
>> (for example).
>>
>> But does every small change I am then making to the package, be
>> followed up with adding a tag, so my package is going to be updated
>> when doing a new deployment?
>>
>> Cheers, Mark
>>
>> On 6/11/2014 4:54 pm, Christian Loock wrote:
>>> If you want to use @dev stability, you probably need to set the
>>> minimum-stability of your main composer.json to "dev". Then, in your
>>> local installation, run a composer.phar update, so that it updates the
>>> composer.lock (just needed when you have your composer.lock under
>>> version control)
>>>
>>> Minimum-stability defaults to stable, which let's composer ignore
>>> anything below that.
>>>
>>> Cheers,
>>>
>>> Christian
>>>
>>>
>>> Am 06.11.2014 um 05:45 schrieb Mark Kuiphuis:
>>>> Hi all,
>>>>
>>>> Well, I have made good progress over the last couple of days in
>>>> regards to TYPO3.Surf (I think), but I don't think I am there yet.
>>>>
>>>> I have been able to start using migrations, with the suggestions from
>>>> this thread. No complaints anymore with an existing database that
>>>> tables can't be modified anymore etc. etc. :-)
>>>>
>>>> I did a successful deployment to a local folder where some packages
>>>> are downloaded from Packagist and some from our own GIT server.
>>>>
>>>> Then I made a (small) change to one of our own packages and pushed
>>>> that to the GIT server. Upon running the deployment again (./flow
>>>> surf:deploy <deploymentFile>) I would have expected that our own
>>>> modified package was going to be downloaded again including this
>>>> change, but it looked like that wasn't the case.
>>>>
>>>> With your (Christian) suggestion to store the composer.lock into the
>>>> git repo, I was able to deploy the site for the first time, but upon
>>>> the 2nd deployment, no changes were made. It looks like composer
>>>> thinks that it is already using the stable version (without my change).
>>>>
>>>> When I changed the stability of my two packages in the root
>>>> composer.json from dev-master to @dev, they disappeared from the
>>>> composer.lock file. Running the deployment again causes the 2 packages
>>>> not to be downloaded at all...
>>>>
>>>> The deployment also in my opinion still takes quite a while, so I am
>>>> unsure whether the cached versions of TYPO3.Flow, TYPO3.Fluid, etc.
>>>> etc. are being used or downloaded again.
>>>>
>>>> Shouldn't the "cache" folder contain a cached version of TYPO3.Flow,
>>>> TYPO3.Fluid, and all other packages (or are the composer cached
>>>> versions stored elsewhere?)
>>>>
>>>> My root composer.json currently looks like this. (formatting can be
>>>> corrupt due to Thunderbird having 80 chars per line):
>>>>
>>>> {
>>>> "name": "typo3/flow-base-distribution",
>>>> "description": "TYPO3 Flow Base Distribution",
>>>> "license": "LGPL-3.0+",
>>>> "config": {
>>>> "vendor-dir": "Packages/Libraries",
>>>> "bin-dir": "bin"
>>>> },
>>>> "require": {
>>>> "typo3/flow": "2.2.*",
>>>> "doctrine/migrations": "@dev",
>>>> "vendor/package": "@dev",
>>>> "vendor/package-client": "@dev",
>>>> "typo3/swiftmailer": "dev-master",
>>>> "famelo/pdf": "dev-master"
>>>> },
>>>> "repositories": [
>>>> {
>>>> "type": "vcs",
>>>> "url":
>>>> "ssh://user@server.tld:22/home/git/Vendor.Package.git"
>>>> },
>>>> {
>>>> "type": "vcs",
>>>> "url":
>>>> "ssh://user@server.tld:22/home/git/Vendor.Package.Client.git"
>>>> }
>>>> ],
>>>> "suggest": {
>>>> "ext-pdo_sqlite": "For running functional tests out-of-the-box
>>>> this is required"
>>>> },
>>>> "scripts": {
>>>> "post-update-cmd":
>>>> "TYPO3\\Flow\\Composer\\InstallerScripts::postUpdateAndInstall",
>>>> "post-install-cmd":
>>>> "TYPO3\\Flow\\Composer\\InstallerScripts::postUpdateAndInstall",
>>>> "post-package-update":
>>>> "TYPO3\\Flow\\Composer\\InstallerScripts::postPackageUpdateAndInstall",
>>>> "post-package-install":
>>>> "TYPO3\\Flow\\Composer\\InstallerScripts::postPackageUpdateAndInstall"
>>>> }
>>>> }
>>>>
>>>> This "package" also contains the "Configuration" folder including the
>>>> Settings.yaml files as well as the Routes.yaml.
>>>>
>>>> And last (but not least) the composer.lock which I believe is causing
>>>> me grief to either not download the updated version of my package
>>>> (with 'dev-master' stability flag or not downloading it at all (with
>>>> '@dev' stability flag))
>>>>
>>>> Sorry for the long post.....
>>>>
>>>> Cheers, Mark Kuiphuis
>>>>
>>>> On 30/10/2014 5:02 pm, Christian Loock wrote:
>>>>> Hey,
>>>>>
>>>>> Am 29.10.2014 um 22:11 schrieb Mark Kuiphuis:
>>>>>> Thanks guys....
>>>>>>
>>>>>> Might need to some work on this then :-) But I wouldn't be where I am
>>>>>> currently at without both your help, so really appreciated :-)
>>>>>>
>>>>> I know surf can be somewhat confusing, since the documentation
>>>>> isn't all
>>>>> that good. I took me quite a while to get into it by experimenting
>>>>> around quite a bit. But once you get it running, the deployment works
>>>>> really good. It also not too hard to use surf to deploy other
>>>>> applications. We is it to deploy our Oxid Shop aswell, which shows how
>>>>> flexible surf is. I really love it and am glad I could help :)
>>>>>
>>>>>> So if I understand you correctly, on every deploy you are downloading
>>>>>> TYPO3.Flow, TYPO3.Fluid, Libraries like Doctrine, Yaml, etc. etc.?
>>>>>>
>>>>> As mentioned before, composer caches libraries, so once they are
>>>>> downloaded, composer will prefer the cached version, unless the
>>>>> requested version changed.
>>>>>
>>>>>
>>>>>> Cheers, Mark
>>>>>>
>>>>>> P.S. Might need to work on doctrine:migrate a bit more because up
>>>>>> until now we have only used doctrine:update :( Thanks for this
>>>>>> tip....
>>>>>>
>>>>>> On 29/10/2014 6:20 pm, Christian Loock wrote:
>>>>>>> Just to add to this:
>>>>>>>
>>>>>>> Be aware that using doctrine:update, even if it might seem very
>>>>>>> convient, might screw up your possibilites to generate migrations
>>>>>>> with
>>>>>>> doctrine:migrationgenerate. We used to use doctrine:update on our
>>>>>>> loca
>>>>>>> working copies aswell, untill we tried to deploy our application the
>>>>>>> first time to our production environment, and realized that our
>>>>>>> migrations were broken, since they were missing stuff. Since
>>>>>>> then, we
>>>>>>> made it a convention to always use doctrine:migrate and never
>>>>>>> update the
>>>>>>> database directly with doctrine:update. This adds a little step, but
>>>>>>> makes your migrations safe and tight.
>>>>>>>
>>>>>>> Also note, that composer has a cache, that will cache
>>>>>>> repositories it
>>>>>>> checks out, for later use. So as long as you have you composer.lock
>>>>>>> versioned, it should only download the repos at the first time you
>>>>>>> deploy, and then take them straight from the cache, which is very
>>>>>>> quick.
>>>>>>>
>>>>>>> Cheers
>>>>>>>
>>>>>>>
>>>>>>> Am 29.10.2014 um 09:09 schrieb Beat Guggisberg:
>>>>>>>> Hey Mark
>>>>>>>>
>>>>>>>> If you copied the Database with all Data in it, then it should have
>>>>>>>> also copied the Migrations Table. So without a codechange there
>>>>>>>> should
>>>>>>>> not have been any migration nessessary.
>>>>>>>>
>>>>>>>> On a Development system you should have no problem just to just
>>>>>>>> clear
>>>>>>>> your database and make a migration to come to the correct Databasae
>>>>>>>> structure. To test your new things that change the database you can
>>>>>>>> update manually with doctrine:update. Never do that on a production
>>>>>>>> server, because it destroys future migrations.
>>>>>>>>
>>>>>>>> To create the migrations you need on the production server,
>>>>>>>> clear the
>>>>>>>> DB on the testserver, do a doctrine:migrate and then a
>>>>>>>> doctrine:migrationgenerate. Now you have a file with all the DB
>>>>>>>> changes in it. You can split up that file for your different
>>>>>>>> packages.
>>>>>>>>
>>>>>>>> For your problem with the downloads, i have created local copies of
>>>>>>>> all nessessary Git repositorys. You can do that with "git clone
>>>>>>>> --mirror", to update them use "git remote update".
>>>>>>>>
>>>>>>>> Now you have to add them as repository in your json.
>>>>>>>>
>>>>>>>> Kind Regards
>>>>>>>> Beat
>>>>>>>>
>>>>>>>>
>>>>>>>> Am Mittwoch, 29. Oktober 2014 01:54 CET, Mark Kuiphuis
>>>>>>>> <mark at capesso.com.au> schrieb:
>>>>>>>> Thanks Christian and Beat :-)
>>>>>>>>
>>>>>>>> Since this is all still very new to me, I wanted to test things
>>>>>>>> thoroughly on a local server before putting it to use on a Live
>>>>>>>> Production Server.
>>>>>>>>
>>>>>>>> I therefore made a copy of my development database instead of
>>>>>>>> creating
>>>>>>>> an empty database (which I did refer to by the way in the
>>>>>>>> Settings.yaml
>>>>>>>> in both Production and Development Contexts).
>>>>>>>>
>>>>>>>> It turned out that, because there was already a database (with
>>>>>>>> tables of
>>>>>>>> my packages and the TYPO3.Party tables) it could not rename the
>>>>>>>> party_abstractparty to typo3_party_domain_model_abstractparty,
>>>>>>>> because
>>>>>>>> that table already existed.....
>>>>>>>>
>>>>>>>> I now need to find a way to move a lot of folders from the releases
>>>>>>>> folder to the "shared" folder and symlink them, because I don't
>>>>>>>> want to
>>>>>>>> download TYPO3.Flow, TYPO3.Fluid, and all libraries when I made a
>>>>>>>> change
>>>>>>>> to one of my own packages and want to deploy these changes.
>>>>>>>>
>>>>>>>> And while my own packages have been downloaded succesfully from
>>>>>>>> our own
>>>>>>>> git server, it did not create the necessary tables upon first
>>>>>>>> run....but
>>>>>>>> that could maybe be due to the lack of a migration script....
>>>>>>>> (not something that I want to do upon each deploy anyway as it
>>>>>>>> should
>>>>>>>> reuse existing tables and modify them instead of creating (for
>>>>>>>> example
>>>>>>>> party_abstractparty which then should be renamed again to
>>>>>>>> typo3_party_domain_model_abstractparty)...
>>>>>>>>
>>>>>>>> Cheers, Mark
>>>>>>>>
>>>>>>>> On 27/10/2014 4:47 pm, Christian Loock wrote:
>>>>>>>>> Hey,
>>>>>>>>>
>>>>>>>>> you are probably just missing the Config for the Production
>>>>>>>>> Context
>>>>>>>>> in you
>>>>>>>>> base repo. Surf will automatically call everything in Production
>>>>>>>>> context,
>>>>>>>>> so you need your Configuration/Production/Settings.yaml to
>>>>>>>>> reflect to
>>>>>>>>> your
>>>>>>>>> actual production database.
>>>>>>>>>
>>>>>>>>> Also you are right, Surf will automatically create the whole
>>>>>>>>> folder
>>>>>>>>> structur for you. Make sure your vhost on your production node
>>>>>>>>> points to
>>>>>>>>> %projectdir%/releases/current/Web where %projectdir%= the
>>>>>>>>> directory
>>>>>>>>> you
>>>>>>>>> deploy to.
More information about the Flow
mailing list