The challenge: You have a big Tiki project and you want to use the traditional development, testing, acceptance and production setup (DTAP)
How do you deal with:
- Generic code that will be upstreamed
- Custom code that won't be upstreamed (but should be versioned locally)
- Custom theme/design (which should be versioned locally)
- Content (in database)
- Content via Wiki Plugins is routinely used to do stuff we used to do in PHP
- Configuration, typically in database, but can be in text files via System Configuration
- Translations that will be upstreamed
- Translations that won't be upstreamed (but should be versioned locally) (could be in DB or lang/xx/custom.php)
- Maintainability of advanced configurations of features
On a related note, here is a Messaging systems evaluation
Examples
- Content evolve in prod (user generated content)
- A new feature is in development, so code needs to be merged from dev to testing. But often preferences and modules need to be configured.
- If you refresh the testing data from the production server (like as Pre-Dogfood Server, you will loose your preferences & module configurations.
Wishes
- A way to easily check on which server I am (which doesn't get overwritten with rsync)
- This is to avoid mistakes
- Quick way to compare the configurations between the 2 to 4 sites and have a way to merge data / propagate some data
- Make it possible to identify a site as staging or development and have different settings, such as
- <meta NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
- Don't send out notification emails
- Site closed
Dev, Test, Prod
- Dev: 10.x
- Test: 9.x latest
- Prod: 9.x older
Before doing svn up on prod, we do svn up on test, and we do a diff of:
- svn diff test
- svn diff prod
Idea
If both sites have shell and use System Configuration, maybe use something like http://www.fossil-scm.org/ to compare/merge...
Who
- Marc Laporte
- you? (Add your name to join the working group)
Related links
Options on Ohloh.net
Overview
T-DOSE 2012 Toshaan Bharvani, Puppet, Chef, CFEngine, Ansible, Salt, different approaches
http://www.coloandcloud.com/editorial/func-mcollective-salt-and-rundeck/
Puppet
Salt
Chef
Free hosted version up to 5 nodes: http://www.opscode.com/chef/
Just scheduled for Jul 31: Thomas S. Hatch - SaltStack update (http://saltstack.com/community.html). We had him on back in Nov 2011, but a lot has happened in the Salt world, including bare metal provisioning, and a lot of other cloudy things.
CFEngine
Bcfg2
cdist
Quattor
Juju
Crowbar
Nuri
Managing TikiWiki CMS with Nuri
Ansible
Alexandre Bourget demoes Ansible:
Fredistrano
OCS Inventory NG
Radmind
Rudder
Spacewalk
Software Test Automation Framework (STAF)
Ganeti
"Ganeti is a cluster virtual server management software tool built on top of existing virtualization technologies such as Xen or KVM and other Open Source software."
https://code.google.com/p/ganeti/
Serf
Serf is a decentralized solution for cluster membership, failure detection, and orchestration. Lightweight and highly available.
Phing
PHing Is Not GNU make; it's a PHP project build system or build tool based on Apache Ant.
Archipel
"Archipel is a solution to manage and supervise virtual machines. No matter if you have a few locally on your computer or thousands through data centers, Archipel is a central solution to manage them all. You can perform all basic virtualization commands and many other things like live migration, VMCasts, packages, etc."
Capistrano
http://capistranorb.com/
Roboconf
Related: Vagrant
alias
!! See also: ((Configuration Management for Tiki Projects))
The challenge: You have a big Tiki project and you want to use the traditional development, testing, acceptance and production setup (DTAP)
How do you deal with:
* Generic code that will be upstreamed
* Custom code that won't be upstreamed (but should be versioned locally)
* Custom theme/design (which should be versioned locally)
* Content (in database)
** Content via Wiki Plugins is routinely used to do stuff we used to do in PHP
* Configuration, typically in database, but can be in text files via ((doc:System Configuration))
* Translations that will be upstreamed
* Translations that won't be upstreamed (but should be versioned locally) (could be in DB or lang/xx/custom.php)
* ((Maintainability of advanced configurations of features))
On a related note, here is a [http://community.aegirproject.org/node/284|Messaging systems evaluation]
! Examples
* Content evolve in prod (user generated content)
* A new feature is in development, so code needs to be merged from dev to testing. But often preferences and modules need to be configured.
** If you refresh the testing data from the production server (like as ((tw:Pre-Dogfood Server)), you will loose your preferences & module configurations.
! Wishes
* A way to easily check on which server I am (which doesn't get overwritten with rsync)
** This is to avoid mistakes
* Quick way to compare the configurations between the 2 to 4 sites and have a way to merge data / propagate some data
* Make it possible to identify a site as staging or development and have different settings, such as
** ~np~<me<x>ta NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">~/np~
** Don't send out notification emails
** Site closed
! Dev, Test, Prod
* Dev: 10.x
* Test: 9.x latest
* Prod: 9.x older
Before doing svn up on prod, we do svn up on test, and we do a diff of:
* svn diff test
* svn diff prod
! Idea
If both sites have shell and use ((doc:System Configuration)), maybe use something like http://www.fossil-scm.org/ to compare/merge...
! Who
* Marc Laporte
* you? (Add your name to join the working group)
! Related links
* {wish id=4061}
* {wish id=2965}
* ((tw:show.tiki.org))
* ((WikiFarm))
* ((TRIM))
* ((Backup))
* ((Infrastructure Revamp))
* ((Monitoring)) and ((Email alerts of hard errors))
* ((Recovery))
* ((System Configuration))
* ((Database Schema Upgrade))
* ((Server tools))
* ((Server Check)) and ((Compatibility))
* ((URL Rewriting Revamp))
* ((Logs))
* ((htaccess))
* ((Operating System independence))
* ((Performance))
* ((Reverse proxy))
* ((Permission Check))
* ((tw:Pre-Dogfood Server))
* ((Maintainability of advanced configurations of features))
* ((Custom installation version))
* ((Continuous Integration))
* ((Distributed revision control))
* ((Distributed data))
* ((doc:Tracker Synchronization))
* ((Command-line tool))
* [http://en.wikipedia.org/wiki/Development,_testing,_acceptance_and_production] (DTAP)
* http://en.wikipedia.org/wiki/Development_environment_(software_development_process)
* The Drupal community is working on this:
** http://groups.drupal.org/node/155559
** http://london2011.drupal.org/sites/default/files/Content_Staging_and_Deployments_in_Drupal_0.pdf
** http://groups.drupal.org/build-systems-change-management/cmi
* https://en.wikipedia.org/wiki/Orchestration_(computing)
* http://heyrocker.com/cmi-feature-freeze
* http://symfony.com/blog/new-in-symfony-2-3-interactive-management-of-the-parameters-yml-file
! Options on Ohloh.net
{HTML()}<script type='text/javascript' src='https://www.ohloh.net/stacks/107830/widgets/stack_normal?icon_width=24&icon_height=24&title=Configuration%20Mngmt%20Stack&width=250&projects_shown=16&noclear=true'></script>{HTML}
!! Overview
!!! T-DOSE 2012 Toshaan Bharvani, Puppet, Chef, CFEngine, Ansible, Salt, different approaches
{youtube movie="https://www.youtube.com/watch?v=T56uN6N_hs0"}
http://www.coloandcloud.com/editorial/func-mcollective-salt-and-rundeck/
!! Puppet
{HTML()}<script type="text/javascript" src="https://www.ohloh.net/p/5695/widgets/project_factoids_stats.js"></script>{HTML}
!! Salt
{HTML()}<script type="text/javascript" src="https://www.ohloh.net/p/488977/widgets/project_factoids_stats.js"></script>{HTML}
* [http://www.blackducksoftware.com/news/releases/2012-01-18|Black Duck Software Names 2011 Open Source Rookies of the Year]
* http://opencredo.com/blog/a-dive-into-salt-stack
* [http://docs.saltstack.org/en/latest/topics/tutorials/starting_states.html#default-data-yaml|uses YAML]
* http://www.coloandcloud.com/editorial/an-interview-with-salt-author-thomas-hatch/
* http://www.theregister.co.uk/2013/01/16/open_and_shut/
* http://opensourcedelivers.com/2012/04/24/what-makes-open-source-projects-take-off-the-salt-case-study/
* https://salt-cloud.readthedocs.org/en/latest/
* [https://groups.google.com/d/msg/salt-users/JKhQ-FtT5LM/XMWgRSflL5QJ|Salt business model]
* http://www.linuxjournal.com/content/using-salt-stack-and-vagrant-drupal-development?page=0,2
* http://missingm.co/2013/06/ansible-and-salt-a-detailed-comparison/
{youtube movie="TK-dKCJFyVo"}
!! Chef
{HTML()}<script type="text/javascript" src="https://www.ohloh.net/p/256946/widgets/project_factoids_stats.js"></script>{HTML}
Free hosted version up to 5 nodes: http://www.opscode.com/chef/
{youtube movie="MTHgfrm2CUQ"}
[https://plus.google.com/105487854388646525021/posts/ZFvEFEMmyWw|Just scheduled for Jul 31: Thomas S. Hatch - SaltStack update (http://saltstack.com/community.html). We had him on back in Nov 2011, but a lot has happened in the Salt world, including bare metal provisioning, and a lot of other cloudy things.]
!! CFEngine
{HTML()}<script type="text/javascript" src="https://www.ohloh.net/p/9219/widgets/project_factoids_stats.js"></script>{HTML}
!! Bcfg2
{HTML()}<script type="text/javascript" src="https://www.ohloh.net/p/7286/widgets/project_factoids_stats.js"></script>{HTML}
* [http://en.wikipedia.org/wiki/Bcfg2]
* [http://docs.bcfg2.org/]
* To make a bcfg2 bundle for Tiki, we first should make ((DEB Packaging))
!! cdist
{HTML()}<script type="text/javascript" src="https://www.ohloh.net/p/616453/widgets/project_factoids_stats.js"></script>{HTML}
!! Quattor
{HTML()}<script type="text/javascript" src="https://www.ohloh.net/p/347136/widgets/project_factoids_stats.js"></script>{HTML}
!! Juju
{HTML()}<script type="text/javascript" src="https://www.ohloh.net/p/597063/widgets/project_factoids_stats.js"></script>{HTML}
!! Crowbar
{HTML()}<script type="text/javascript" src="https://www.ohloh.net/p/596606/widgets/project_factoids_stats.js"></script>{HTML}
!! Nuri
{HTML()}<script type="text/javascript" src="https://www.ohloh.net/p/637525/widgets/project_factoids_stats.js"></script>{HTML}
!!! Managing TikiWiki CMS with Nuri
{youtube movie="wMGchBSOJoA"}
!! Ansible
{HTML()}<script type="text/javascript" src="https://www.ohloh.net/p/596190/widgets/project_factoids_stats.js"></script>
{HTML}
{youtube movie="_9E7yzbKnBY"}
* http://www.blackducksoftware.com/news/releases/black-duck-announces-2012-open-source-rookies-year-winners
* http://highscalability.com/blog/2012/4/18/ansible-a-simple-model-driven-configuration-management-and-c.html
* http://server.dzone.com/articles/ansible-cm-deployment-and-ad
* [http://ansible.cc/docs/YAMLSyntax.html|Uses YAML]
* http://ansible.cc/faq.html
* http://www.arrfab.net/blog/?p=361
* http://fourkitchens.com/blog/2012/06/29/ansible-simple-configuration-deployment
* https://en.wikipedia.org/wiki/Ansible_(software)
* http://www.coloandcloud.com/editorial/an-interview-with-ansible-author-michael-dehaan/
* http://ansible.cc/docs/modules.html
** SVN, Git, hg, mysql_db, mysql_user, nagios, apt/yum
* http://missingm.co/2013/06/ansible-and-salt-a-detailed-comparison/
Alexandre Bourget demoes Ansible:
{youtube movie="https://www.youtube.com/watch?v=up3ofvQNm8c"}
{youtube movie="https://www.youtube.com/watch?v=BTAgQ9-LD5o"}
!! Fredistrano
{HTML()}<script type="text/javascript" src="https://www.ohloh.net/p/17056/widgets/project_factoids_stats.js"></script>{HTML}
!! OCS Inventory NG
{HTML()}<script type="text/javascript" src="https://www.ohloh.net/p/28369/widgets/project_factoids_stats.js"></script>{HTML}
!! Radmind
{HTML()}<script type="text/javascript" src="https://www.ohloh.net/p/11759/widgets/project_factoids_stats.js"></script>{HTML}
!! Rudder
{HTML()}<script type="text/javascript" src="https://www.ohloh.net/p/587272/widgets/project_factoids_stats.js"></script>{HTML}
!! Spacewalk
{HTML()}<script type="text/javascript" src="https://www.ohloh.net/p/15042/widgets/project_factoids_stats.js"></script>{HTML}
!! Software Test Automation Framework (STAF)
{HTML()}<script type="text/javascript" src="https://www.ohloh.net/p/8737/widgets/project_factoids_stats.js"></script>{HTML}
!! Ganeti
{HTML()}<script type="text/javascript" src="https://www.ohloh.net/p/15347/widgets/project_factoids_stats.js"></script>{HTML}
"Ganeti is a cluster virtual server management software tool built on top of existing virtualization technologies such as Xen or KVM and other Open Source software."
https://code.google.com/p/ganeti/
!! Serf
Serf is a decentralized solution for cluster membership, failure detection, and orchestration. Lightweight and highly available.
{HTML()}<script type="text/javascript" src="https://www.openhub.net/p/719111/widgets/project_factoids_stats.js"></script>{HTML}
!! Phing
[http://www.phing.info|PHing Is Not GNU make; it's a PHP project build system or build tool based on Apache Ant. ]
!! Archipel
{HTML()}<script type="text/javascript" src="http://www.ohloh.net/p/485631/widgets/project_factoids_stats.js"></script>{HTML}
"Archipel is a solution to manage and supervise virtual machines. No matter if you have a few locally on your computer or thousands through data centers, Archipel is a central solution to manage them all. You can perform all basic virtualization commands and many other things like live migration, VMCasts, packages, etc."
!! Capistrano
[http://capistranorb.com/]
!! Roboconf
{HTML()}<script type='text/javascript' src='https://www.openhub.net/p/roboconf/widgets/project_factoids_stats?format=js'></script>{HTML}
Related: ((Vagrant))
-=alias=-
* (alias(DTAP))
* (alias(Configuration Management))
* (alias(Systems Orchestration))
* (alias(Bcfg2))
* (alias(Salt))
* (alias(Chef))
* (alias(CFEngine))
* (alias(cdist))
* (alias(Quattor))
* (alias(Juju))
* (alias(Crowbar))
* (alias(Ansible))
* (alias(Fredistrano))
* (alias(OCS Inventory NG))
* (alias(Radmind))
* (alias(Rudder))
* (alias(Spacewalk))
* (alias(STAF))
* (alias(Software Test Automation Framework))
* (alias(Ganeti))
* (alias(Roboconf))