[sepia] Pets and Cattle

Sage Weil sage at newdream.net
Fri Sep 4 05:58:49 PDT 2015


On Fri, 4 Sep 2015, Loic Dachary wrote:
> Hi,
> 
> Now that we have the OpenStack backend for teuthology (thanks Zack for 
> your patience with that)[1] and an OpenStack tenant with large quotas to 
> add to the sepia lab, we're going to start using it in various ways. To 
> start on the right foot, I propose we separate cattle from pets so the 
> pets are not trampled by the cattle.
> 
> When the OpenStack backend launches hundreds of virtual machines and 
> creates hundreds of disk volumes, it also deletes all that eventually. 
> And when something bad happens, a script collects the left over and 
> delete them. There are safeguards : multiple teuthology clusters can 
> co-exist within the same OpenStack tenant and will not destroy each 
> other VM. In the typical use case, when the teuthology suite completes, 
> its results are uploaded to public location [2] and there is nothing 
> left in the OpenStack tenant.
> 
> The OpenStack cluster can also be used for hosting gitbuilder machines, 
> bots, secondary DNS servers and what not. The cheapest instance costs 3 
> euros per month with 2GB RAM, 1 vCPU, 10 GB disk: there is plenty of 
> uses cases for that. These are long lived machines which are not 
> expected to be destroyed.
> 
> I would be extremely nervous to know that teuthology operates in the 
> same OpenStack tenant as long lived instances. It's not a matter of 
> knowing if a pet will be slaughtered by teuthology, it's a matter of 
> when. Bugs are bound to happen and transform teuthology into an 
> instrument of mass destruction.
> 
> To avoid that unfortunate scenario, I propose that long lived machines 
> are hosted in a separate tenant. The OpenStack provider allows for more 
> than one tenant to be created under the same account. I created this 
> tenant (called Pets), renamed the current tenant (Cattle) and asked the 
> provider that the quotas are set to limits higher than the defaults and 
> (hopefully) suitable to the use case of long lived machines. That is:

Sounds good, except I think 'Pet's and 'Cattle' are swapped?  The 
gitbuilders are pets in this case... but that tenant is named cattle 
currently.

sage


> 
>   100 instances
>   400 vCPUS
>   1TB RAM
>   100 volumes
>   10TB disk
> 
> It's not set in stone: the provider can increase the quotas with one 
> week delay. And if this turns out to be a bad idea, the tenant can be 
> left unused and won't cost anything.
> 
> What do you think ?
> 
> Cheers
> 
> [1] https://github.com/ceph/teuthology/#openstack-backend
> [2] http://teuthology-logs.public.ceph.com/
> 
> 
> -- 
> Loïc Dachary, Artisan Logiciel Libre
> 
> 


More information about the Sepia mailing list