[Flow] TYPO3 Surf questions

[Mark Kuiphuis]

Hi all,

Finally found the time to start digging again in regards to TYPO3.Surf

I am experiencing the following problem and although I do see the error 
message "No datrabase selected", I have no idea how to solve it...

staging (Vendor.Package) typo3.surf:typo3:flow:symlinkdata
staging (Vendor.Package) typo3.surf:typo3:flow:symlinkconfiguration
staging (Vendor.Package) typo3.surf:typo3:flow:copyconfiguration
Stage migrate
staging (Vendor.Package) typo3.surf:typo3:flow:migrate
Uncaught Exception
   An exception occurred while executing 'SHOW FULL TABLES WHERE Table_type
   = 'BASE TABLE'':

   SQLSTATE[3D000]: Invalid catalog name: 1046 No database selected

More Information
   Exception code      #0
   File 
/private/var/www/staging/app/releases/20141027121635/Packages/Libraries/doctrine/dbal/lib/Doctrine/DBAL/DBALException.php 
line 47
Got exception "Command returned non-zero return code: 1" rolling back.
Rolling back TYPO3\Surf\Task\TYPO3\Flow\CopyConfigurationTask
Rolling back TYPO3\Surf\Task\TYPO3\Flow\SymlinkConfigurationTask
Rolling back TYPO3\Surf\Task\TYPO3\Flow\SymlinkDataTask
Rolling back TYPO3\Surf\Task\Composer\InstallTask
Rolling back TYPO3\Surf\Task\Generic\CreateSymlinksTask
Rolling back TYPO3\Surf\Task\GitCheckoutTask
Rolling back TYPO3\Surf\Task\TYPO3\Flow\CreateDirectoriesTask
Rolling back TYPO3\Surf\Task\Generic\CreateDirectoriesTask
Rolling back TYPO3\Surf\Task\CreateDirectoriesTask

My understanding of TYPO3.Surf was that you can create an empty folder 
(/private/var/www/staging/app) on a server (I am currently using my own 
development machine - OSX) and let that folder be the root folder for 
the deployment.

Inside this folder, the TYPO3.Surf specific folders are generated 
(cache, releases and shared including some of their subfolders). 
Therefore I believe, but please correct me if I am wrong, that I should 
only start with an empty folder and not a folder which already has the 
"default" TYPO3 Flow folder structure like:

/private/var/www/app/bin
/private/var/www/app/Build
/private/var/www/app/Configuration
/private/var/www/app/Data
/private/var/www/app/Packages
/private/var/www/app/Web
(and some files like composer.json, Readme.txt, Upgrading.txt etc. etc.)

When running the deploy script, it creates the folder structure like:

/private/var/www/app/cache
/private/var/www/app/cache/transfer
/private/var/www/app/releases
/private/var/www/app/shared
/private/var/www/app/shared/Configuration
/private/var/www/app/shared/Configuration/Production
/private/var/www/app/shared/Data
/private/var/www/app/shared/Data/Logs
/private/var/www/app/shared/Data/Persistent
/private/var/www/app/shared/Data/Resources

it downloads the framework, additional libraries, all my custom packages 
until it hits the
typo3.surf:typo3:flow:migrate command (see above)...

and then does a rollback and removes all of the automatically downloaded 
packages again..... :(

(I currently don't have a "Migrations" folder in one of my packages)

Any idea where my thinking goes haywire?

Regards,

Mark Kuiphuis
Capesso Pty Ltd

On 13/10/2014 5:59 pm, Mark Kuiphuis wrote:
> Thanks Christian,
>
> I will look into it a bit more later on.
>
> Cheers, Mark
>
> On 13/10/2014 5:36 pm, Christian Loock wrote:
>> Almost.
>>
>> If you have a look at the composer package typo3/flow-base-distribution,
>> you will see, that it only contains a very bare bone skelton for your
>> project structure, and the composer.json. Packages will be installed
>> with composer, based on this config. You do not have to version flow
>> etc. by yourself, since composer will just clone it from the typo3 git
>> servers. Surf uses composer to install your appluication. Composer will
>> look, which packages you reference in your composer.json and install
>> them accordingly. Each package resides in its own git repository. You
>> just need to tell composer, where it can find those packages. This can
>> be done by either adding them to the global packagist at packagist.org,
>> hosting your own composer repository service (like hosting your own
>> packagist or satis) or just configuring your repositories in your
>> composer.json¹
>>
>> You might also want to have a look at the composer docs for more detail
>> on that. Also check out this² link for more details on how flow manages
>> packages
>>
>> For your clients, I would just create a simple project based on the flow
>> base distribution and alter the composer.json, so that it contains the
>> packages you want to be installed. Each of these projects then goes into
>> its own repository. Also note, that each base distribution is also a
>> composer package of its own, so that you can easily install it manually
>> using composer's create-project command, as long as composer is aware of
>> where your projects are hosted.
>>
>> Hope this clearifies it a bit more.
>>
>> [1] https://getcomposer.org/doc/05-repositories.md#vcs
>> [2]
>> http://docs.typo3.org/flow/TYPO3FlowDocumentation/TheDefinitiveGuide/PartIII/PackageManagement.html
>>
>>
>>
>> Just
>> Am 13.10.2014 um 09:05 schrieb Mark Kuiphuis:
>>> Hi Christian,
>>>
>>> Thank you for your quick reply.
>>>
>>> To answer your questions. The "modules" are all unique packages, but
>>> we named them this way for consistency reasons. So, in short:
>>>
>>> Vendor.Package         --> Vendor.Package.git
>>> Vendor.Package.Client    --> Vendor.Package.Client.git
>>> etc.
>>>
>>> Right now "Package" is the name of our membership system, but in the
>>> future we might create Vendor.OtherPackage.git as our core for a
>>> totally different application, with Vendor.OtherPackage.Client.git as
>>> the specific client package tied in to Vendor.OtherPackage.
>>>
>>> (The reason we have to create Vendor.Package.Client is that Custom
>>> Invoice templates are being used as well as client specific images and
>>> not to forget their own implementation of their rules to determine,
>>> for example, a member's financial status).
>>>
>>> Currently we git clone the individual packages from our own GIT Server
>>> and then setup the Routes.yaml to let them work.
>>>
>>> If I understand you correctly, you are saying that I can create a base
>>> distribution for each client (does that include all sources of
>>> TYPO3.Flow, TYPO3.Eel, TYPO3.Fluid, etc. etc.) which I don't think is
>>> a good idea... but maybe I am totally wrong here.
>>>
>>> Could I create a bare folder structure like:
>>>
>>> -- client
>>>  -- bin
>>>  -- Build
>>>  -- Configuration
>>>  -- Data
>>>  -- Packages
>>>   -- Application
>>>  -- Web
>>>  -- composer.json
>>>
>>> And then define in the composer.json all packages I need for this
>>> specific client? Or am I missing the mark here?
>>>
>>> Kind regards,
>>>
>>> Mark
>>>
>>>
>>> On 13/10/2014 4:09 pm, Christian Loock wrote:
>>>> The structure you are using here, seems to be a bit problematic to
>>>> handle. If I understand correct, your Modules are Subpackages of
>>>> Vendor.Package? If i see that right, you wont really be able to deploy
>>>> them sperately, since they all reside inside the same composer package.
>>>> You would have to split them up into separated (composer) packages.
>>>> Then
>>>> in order to deploy those diffenrent configurations, you would need a
>>>> base distribution repo, that contains a composer.json that references
>>>> all the packages you want to install. You can use the same application
>>>> configuration class in surf for this, if the deployment process remains
>>>> the same. Just exchange the git repository that is used.
>>>>
>>>> Cheer,
>>>>
>>>> Christian
>>>>
>>>> Am 13.10.2014 um 08:03 schrieb Mark Kuiphuis:
>>>>> Hi all,
>>>>>
>>>>> How can I use TYPO3.Surf to deploy multiple packages to a staging
>>>>> server?
>>>>>
>>>>> Packages/Application/Vendor.Package
>>>>> Packages/Application/Vendor.Package.Client
>>>>> Packages/Application/Vendor.Package.Module1
>>>>> ..
>>>>> ..
>>>>> Packages/Application/Vendor.Package.Module4
>>>>> etc. etc.
>>>>>
>>>>> Another installation can have:
>>>>>
>>>>> Packages/Application/Vendor.Package
>>>>> Packages/Application/Vendor.Package.OtherClient
>>>>> Packages/Application/Vendor.Package.Module3
>>>>> Packages/Application/Vendor.Package.Module4
>>>>>
>>>>> Vendor.Package is the package with the Core Functionality.
>>>>> Vendor.Package.ModuleX allow for additional functionality potentially
>>>>> for 1 or more clients.
>>>>>
>>>>> All the examples only talk about one $application send to a node, but
>>>>> for every client with their own staging version, I need to have at
>>>>> least 2 (Core and Client) or more packages being updated.
>>>>>
>>>>> Because all the clients use the same Core Package, I believe I should
>>>>> only have 1 repo on my git server for the Core package, instead of
>>>>> saving the full set of modules for each client in a repo....
>>>>>
>>>>> But I fail to see how to deploy this with TYPO3.Surf
>>>>>
>>>>> Thanks in advance,
>>>>>
>>>>> --
>>>>> Mark Kuiphuis
>>>>> Capesso Pty Ltd