[Flow] TYPO3 Surf questions

[Mark Kuiphuis]

Not really no, in answer to your remark about "if you want to use @dev 
stability". :-)

I would like to get it working properly first and I can then always 
finetune and optimise.

I tried the minimum-stability option, but this is then also applicable 
to all TYPO3.Flow packages and its dependencies (doctrine, migrations, 
etc. etc.) Not a good idea for a production environment...

I am now going to look into creating tags in our own packages...
Instead of using @dev or dev-master I suppose then have to use "0.1.*" 
(for example).

But does every small change I am then making to the package, be followed 
up with adding a tag, so my package is going to be updated when doing a 
new deployment?

Cheers, Mark

On 6/11/2014 4:54 pm, Christian Loock wrote:
> If you want to use @dev stability, you probably need to set the
> minimum-stability of your main composer.json to "dev". Then, in your
> local installation, run a composer.phar update, so that it updates the
> composer.lock (just needed when you have your composer.lock under
> version control)
>
> Minimum-stability defaults to stable, which let's composer ignore
> anything below that.
>
> Cheers,
>
> Christian
>
>
> Am 06.11.2014 um 05:45 schrieb Mark Kuiphuis:
>> Hi all,
>>
>> Well, I have made good progress over the last couple of days in
>> regards to TYPO3.Surf (I think), but I don't think I am there yet.
>>
>> I have been able to start using migrations, with the suggestions from
>> this thread. No complaints anymore with an existing database that
>> tables can't be modified anymore etc. etc. :-)
>>
>> I did a successful deployment to a local folder where some packages
>> are downloaded from Packagist and some from our own GIT server.
>>
>> Then I made a (small) change to one of our own packages and pushed
>> that to the GIT server. Upon running the deployment again (./flow
>> surf:deploy <deploymentFile>) I would have expected that our own
>> modified package was going to be downloaded again including this
>> change, but it looked like that wasn't the case.
>>
>> With your (Christian) suggestion to store the composer.lock into the
>> git repo, I was able to deploy the site for the first time, but upon
>> the 2nd deployment, no changes were made. It looks like composer
>> thinks that it is already using the stable version (without my change).
>>
>> When I changed the stability of my two packages in the root
>> composer.json from dev-master to @dev, they disappeared from the
>> composer.lock file. Running the deployment again causes the 2 packages
>> not to be downloaded at all...
>>
>> The deployment also in my opinion still takes quite a while, so I am
>> unsure whether the cached versions of TYPO3.Flow, TYPO3.Fluid, etc.
>> etc. are being used or downloaded again.
>>
>> Shouldn't the "cache" folder contain a cached version of TYPO3.Flow,
>> TYPO3.Fluid, and all other packages (or are the composer cached
>> versions stored elsewhere?)
>>
>> My root composer.json currently looks like this. (formatting can be
>> corrupt due to Thunderbird having 80 chars per line):
>>
>> {
>>     "name": "typo3/flow-base-distribution",
>>     "description": "TYPO3 Flow Base Distribution",
>>     "license": "LGPL-3.0+",
>>     "config": {
>>         "vendor-dir": "Packages/Libraries",
>>         "bin-dir": "bin"
>>     },
>>     "require": {
>>         "typo3/flow": "2.2.*",
>>         "doctrine/migrations": "@dev",
>>         "vendor/package": "@dev",
>>         "vendor/package-client": "@dev",
>>         "typo3/swiftmailer": "dev-master",
>>         "famelo/pdf": "dev-master"
>>     },
>>     "repositories": [
>>         {
>>             "type": "vcs",
>>             "url": "ssh://user@server.tld:22/home/git/Vendor.Package.git"
>>         },
>>         {
>>             "type": "vcs",
>>             "url":
>> "ssh://user@server.tld:22/home/git/Vendor.Package.Client.git"
>>         }
>>     ],
>>     "suggest": {
>>         "ext-pdo_sqlite": "For running functional tests out-of-the-box
>> this is required"
>>     },
>>     "scripts": {
>>         "post-update-cmd":
>> "TYPO3\\Flow\\Composer\\InstallerScripts::postUpdateAndInstall",
>>         "post-install-cmd":
>> "TYPO3\\Flow\\Composer\\InstallerScripts::postUpdateAndInstall",
>>         "post-package-update":
>> "TYPO3\\Flow\\Composer\\InstallerScripts::postPackageUpdateAndInstall",
>>         "post-package-install":
>> "TYPO3\\Flow\\Composer\\InstallerScripts::postPackageUpdateAndInstall"
>>     }
>> }
>>
>> This "package" also contains the "Configuration" folder including the
>> Settings.yaml files as well as the Routes.yaml.
>>
>> And last (but not least) the composer.lock which I believe is causing
>> me grief to either not download the updated version of my package
>> (with 'dev-master' stability flag or not downloading it at all (with
>> '@dev' stability flag))
>>
>> Sorry for the long post.....
>>
>> Cheers, Mark Kuiphuis
>>
>> On 30/10/2014 5:02 pm, Christian Loock wrote:
>>> Hey,
>>>
>>> Am 29.10.2014 um 22:11 schrieb Mark Kuiphuis:
>>>> Thanks guys....
>>>>
>>>> Might need to some work on this then :-) But I wouldn't be where I am
>>>> currently at without both your help, so really appreciated :-)
>>>>
>>> I know surf can be somewhat confusing, since the documentation isn't all
>>> that good. I took me quite a while to get into it by experimenting
>>> around quite a bit. But once you get it running, the deployment works
>>> really good. It also not too hard to use surf to deploy other
>>> applications. We is it to deploy our Oxid Shop aswell, which shows how
>>> flexible surf is. I really love it and am glad I could help :)
>>>
>>>> So if I understand you correctly, on every deploy you are downloading
>>>> TYPO3.Flow, TYPO3.Fluid, Libraries like Doctrine, Yaml, etc. etc.?
>>>>
>>> As mentioned before, composer caches libraries, so once they are
>>> downloaded, composer will prefer the cached version, unless the
>>> requested version changed.
>>>
>>>
>>>> Cheers, Mark
>>>>
>>>> P.S. Might need to work on doctrine:migrate a bit more because up
>>>> until now we have only used doctrine:update :( Thanks for this tip....
>>>>
>>>> On 29/10/2014 6:20 pm, Christian Loock wrote:
>>>>> Just to add to this:
>>>>>
>>>>> Be aware that using doctrine:update, even if it might seem very
>>>>> convient, might screw up your possibilites to generate migrations with
>>>>> doctrine:migrationgenerate. We used to use doctrine:update on our loca
>>>>> working copies aswell, untill we tried to deploy our application the
>>>>> first time to our production environment, and realized that our
>>>>> migrations were broken, since they were missing stuff. Since then, we
>>>>> made it a convention to always use doctrine:migrate and never
>>>>> update the
>>>>> database directly with doctrine:update. This adds a little step, but
>>>>> makes your migrations safe and tight.
>>>>>
>>>>> Also note, that composer has a cache, that will cache repositories it
>>>>> checks out, for later use. So as long as you have you composer.lock
>>>>> versioned, it should only download the repos at the first time you
>>>>> deploy, and then take them straight from the cache, which is very
>>>>> quick.
>>>>>
>>>>> Cheers
>>>>>
>>>>>
>>>>> Am 29.10.2014 um 09:09 schrieb Beat Guggisberg:
>>>>>> Hey Mark
>>>>>>
>>>>>> If you copied the Database with all Data in it, then it should have
>>>>>> also copied the Migrations Table. So without a codechange there
>>>>>> should
>>>>>> not have been any migration nessessary.
>>>>>>
>>>>>> On a Development system you should have no problem just to just clear
>>>>>> your database and make a migration to come to the correct Databasae
>>>>>> structure. To test your new things that change the database you can
>>>>>> update manually with doctrine:update. Never do that on a production
>>>>>> server, because it destroys future migrations.
>>>>>>
>>>>>> To create the migrations you need on the production server, clear the
>>>>>> DB on the testserver, do a doctrine:migrate and then a
>>>>>> doctrine:migrationgenerate. Now you have a file with all the DB
>>>>>> changes in it. You can split up that file for your different
>>>>>> packages.
>>>>>>
>>>>>> For your problem with the downloads, i have created local copies of
>>>>>> all nessessary Git repositorys. You can do that with "git clone
>>>>>> --mirror", to update them use "git remote update".
>>>>>>
>>>>>> Now you have to add them as repository in your json.
>>>>>>
>>>>>> Kind Regards
>>>>>> Beat
>>>>>>
>>>>>>
>>>>>> Am Mittwoch, 29. Oktober 2014 01:54 CET, Mark Kuiphuis
>>>>>> <mark at capesso.com.au> schrieb:
>>>>>>   Thanks Christian and Beat :-)
>>>>>>
>>>>>> Since this is all still very new to me, I wanted to test things
>>>>>> thoroughly on a local server before putting it to use on a Live
>>>>>> Production Server.
>>>>>>
>>>>>> I therefore made a copy of my development database instead of
>>>>>> creating
>>>>>> an empty database (which I did refer to by the way in the
>>>>>> Settings.yaml
>>>>>> in both Production and Development Contexts).
>>>>>>
>>>>>> It turned out that, because there was already a database (with
>>>>>> tables of
>>>>>> my packages and the TYPO3.Party tables) it could not rename the
>>>>>> party_abstractparty to typo3_party_domain_model_abstractparty,
>>>>>> because
>>>>>> that table already existed.....
>>>>>>
>>>>>> I now need to find a way to move a lot of folders from the releases
>>>>>> folder to the "shared" folder and symlink them, because I don't
>>>>>> want to
>>>>>> download TYPO3.Flow, TYPO3.Fluid, and all libraries when I made a
>>>>>> change
>>>>>> to one of my own packages and want to deploy these changes.
>>>>>>
>>>>>> And while my own packages have been downloaded succesfully from
>>>>>> our own
>>>>>> git server, it did not create the necessary tables upon first
>>>>>> run....but
>>>>>> that could maybe be due to the lack of a migration script....
>>>>>> (not something that I want to do upon each deploy anyway as it should
>>>>>> reuse existing tables and modify them instead of creating (for
>>>>>> example
>>>>>> party_abstractparty which then should be renamed again to
>>>>>> typo3_party_domain_model_abstractparty)...
>>>>>>
>>>>>> Cheers, Mark
>>>>>>
>>>>>> On 27/10/2014 4:47 pm, Christian Loock wrote:
>>>>>>> Hey,
>>>>>>>
>>>>>>> you are probably just missing the Config for the Production Context
>>>>>>> in you
>>>>>>> base repo. Surf will automatically call everything in Production
>>>>>>> context,
>>>>>>> so you need your Configuration/Production/Settings.yaml to
>>>>>>> reflect to
>>>>>>> your
>>>>>>> actual production database.
>>>>>>>
>>>>>>> Also you are right, Surf will automatically create the whole folder
>>>>>>> structur for you. Make sure your vhost on your production node
>>>>>>> points to
>>>>>>> %projectdir%/releases/current/Web where %projectdir%= the directory
>>>>>>> you
>>>>>>> deploy to.