How to make vagrant create a new virtual disk for virtualbox AND handle the edge cases?

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

How to make vagrant create a new virtual disk for virtualbox AND handle the edge cases?

valts
Backstory: I'm making a vagrant box for development. All the software and services are running inside the vagrant box (Oracle Virtualbox, Centos 7), but two things reside outside: the source code and the MySQL DB. The reasoning is that these two things should survive a "vagrant destroy"/"vagrant up" cycle. But if the developer wants to start a completely clean slate, he should be able to simply delete the appropriate folders, and Vagrant will create them anew.

The source code is simply mounted as a shared folder and symlinked where it need to be inside the system. That works well. I tried to do the same thing with the DB - just pointing MySQL at the shared folder - but the world falls apart when the machine is suspended/resumed. So I'm trying another approach - add a separate virtual disk for MySQL data.

Unfortunately, virtualbox is being a b*tch about it. When you create a virtual disk (VDI), it also assigns it a new UUID and then registers both the path and the UUID in its internal storage registry. If a developer subsequently deletes the file, trying to recreate the file with "vboxmanage createhd" will fail because VirtualBox already has a different file registered for that path. Simply placing a new file in that location will also fail because the UUIDs won't match. To do anything, I need to issue a "vboxmanage closemedium" first which removes it from the registry. Unfortunately I cannot just execute that command blindly either, because it will fail (and in turn trip up vagrant) if the file does not exist in the registry. And as far as I'm aware, there's no way I can query the registry from within a vagrant script...

One solution would be to use a new random filename each time, but that would seriously pollute the Virtualbox storage registry. Also, virtualbox likes to complain about invalid disks in its registry.

Any ideas?

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/vagrant/issues
IRC: #vagrant on Freenode
---
You received this message because you are subscribed to the Google Groups "Vagrant" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/vagrant-up/00ea994e-a6c0-4d65-be3c-2dc0bb3a3ba6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: How to make vagrant create a new virtual disk for virtualbox AND handle the edge cases?

Alvaro Miranda Aguilera
Hello

I am not sure I understand the solution very well.

when Vagrant create a vm, lets say:

Template -> Guest1

and you add a new VDI disk to Guest1

it will be exactly the same as putting the mysql db in /var/something/something

because if you do vagrant destroy, vagrant will delete the VM and the VM will cascade the deletion to include the disk that is attached.

So I am missing something in your persistent story?


In the other hand, if you use vagrant triggers, before destroy you could detach the disk and in this way won't be deleted with a vagrant destroy.

Now.

whats the real issue with having developers to delete a local development db ?

There are tools that can track schema and data change that can be incorportated into the code and then other tools can be used to deploy schema/data changes into a new DB if needed.

Ie, FLYWAY.

If you could explain a bit more, I am sure someone here could help better.

Thanks
Alvaro.

On Mon, Sep 18, 2017 at 1:54 PM, <[hidden email]> wrote:
Backstory: I'm making a vagrant box for development. All the software and services are running inside the vagrant box (Oracle Virtualbox, Centos 7), but two things reside outside: the source code and the MySQL DB. The reasoning is that these two things should survive a "vagrant destroy"/"vagrant up" cycle. But if the developer wants to start a completely clean slate, he should be able to simply delete the appropriate folders, and Vagrant will create them anew.

The source code is simply mounted as a shared folder and symlinked where it need to be inside the system. That works well. I tried to do the same thing with the DB - just pointing MySQL at the shared folder - but the world falls apart when the machine is suspended/resumed. So I'm trying another approach - add a separate virtual disk for MySQL data.

Unfortunately, virtualbox is being a b*tch about it. When you create a virtual disk (VDI), it also assigns it a new UUID and then registers both the path and the UUID in its internal storage registry. If a developer subsequently deletes the file, trying to recreate the file with "vboxmanage createhd" will fail because VirtualBox already has a different file registered for that path. Simply placing a new file in that location will also fail because the UUIDs won't match. To do anything, I need to issue a "vboxmanage closemedium" first which removes it from the registry. Unfortunately I cannot just execute that command blindly either, because it will fail (and in turn trip up vagrant) if the file does not exist in the registry. And as far as I'm aware, there's no way I can query the registry from within a vagrant script...

One solution would be to use a new random filename each time, but that would seriously pollute the Virtualbox storage registry. Also, virtualbox likes to complain about invalid disks in its registry.

Any ideas?

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/vagrant/issues
IRC: #vagrant on Freenode
---
You received this message because you are subscribed to the Google Groups "Vagrant" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/vagrant-up/00ea994e-a6c0-4d65-be3c-2dc0bb3a3ba6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
Alvaro

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/vagrant/issues
IRC: #vagrant on Freenode
---
You received this message because you are subscribed to the Google Groups "Vagrant" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/vagrant-up/CAHqq0eyO9KPOA7gaRh_HThF5AiAav6dUkM1WNf%2BbHwJ4kk2AfQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: How to make vagrant create a new virtual disk for virtualbox AND handle the edge cases?

valts
Hello!

Ahh, good point about the "vagrant destroy" also destroying the disks. I hadn't thought about that.

Anyways, what I wanted to do was, well... as I said above - keep the source code and the DB contents outside of the vagrant machine, so that you could update it or recreate it or whatever without it affecting your work. On the other hand, if you DO want to start with a clean slate, you can just manually delete the DB and source folders, and the next "vagrant up" will just bring it all back with defaults.

Do you think this is an overkill? Maybe I shouldn't bother with it and just keep the DB inside the box?

I suppose a "vagrant destroy" won't be a very often executed command. I use it a lot while putting the machine together and testing out things, but after that... Well, I don't know. I think the machine would get updated now and then so people would need to do a "vagrant destroy"/"vagrant up" to get the latest version... but that's just a guess.

Valts.


On Monday, September 18, 2017 at 3:18:20 PM UTC+3, Alvaro Miranda Aguilera wrote:
Hello

I am not sure I understand the solution very well.

when Vagrant create a vm, lets say:

Template -> Guest1

and you add a new VDI disk to Guest1

it will be exactly the same as putting the mysql db in /var/something/something

because if you do vagrant destroy, vagrant will delete the VM and the VM will cascade the deletion to include the disk that is attached.

So I am missing something in your persistent story?


In the other hand, if you use vagrant triggers, before destroy you could detach the disk and in this way won't be deleted with a vagrant destroy.

Now.

whats the real issue with having developers to delete a local development db ?

There are tools that can track schema and data change that can be incorportated into the code and then other tools can be used to deploy schema/data changes into a new DB if needed.

Ie, FLYWAY.

If you could explain a bit more, I am sure someone here could help better.

Thanks
Alvaro.

On Mon, Sep 18, 2017 at 1:54 PM, <<a href="javascript:" target="_blank" gdf-obfuscated-mailto="_WHXxA4rBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">va...@...> wrote:
Backstory: I'm making a vagrant box for development. All the software and services are running inside the vagrant box (Oracle Virtualbox, Centos 7), but two things reside outside: the source code and the MySQL DB. The reasoning is that these two things should survive a "vagrant destroy"/"vagrant up" cycle. But if the developer wants to start a completely clean slate, he should be able to simply delete the appropriate folders, and Vagrant will create them anew.

The source code is simply mounted as a shared folder and symlinked where it need to be inside the system. That works well. I tried to do the same thing with the DB - just pointing MySQL at the shared folder - but <a href="https://dba.stackexchange.com/questions/185711/running-mysql-in-virtualbox-with-data-dir-on-shared-folder-sometimes-results-in" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fdba.stackexchange.com%2Fquestions%2F185711%2Frunning-mysql-in-virtualbox-with-data-dir-on-shared-folder-sometimes-results-in\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFibrrsrgtjK5LFKxzm37Rz86uL3g&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fdba.stackexchange.com%2Fquestions%2F185711%2Frunning-mysql-in-virtualbox-with-data-dir-on-shared-folder-sometimes-results-in\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNFibrrsrgtjK5LFKxzm37Rz86uL3g&#39;;return true;">the world falls apart when the machine is suspended/resumed. So I'm trying another approach - add a separate virtual disk for MySQL data.

Unfortunately, virtualbox is being a b*tch about it. When you create a virtual disk (VDI), it also assigns it a new UUID and then registers both the path and the UUID in its internal storage registry. If a developer subsequently deletes the file, trying to recreate the file with "vboxmanage createhd" will fail because VirtualBox already has a different file registered for that path. Simply placing a new file in that location will also fail because the UUIDs won't match. To do anything, I need to issue a "vboxmanage closemedium" first which removes it from the registry. Unfortunately I cannot just execute that command blindly either, because it will fail (and in turn trip up vagrant) if the file does not exist in the registry. And as far as I'm aware, there's no way I can query the registry from within a vagrant script...

One solution would be to use a new random filename each time, but that would seriously pollute the Virtualbox storage registry. Also, virtualbox likes to complain about invalid disks in its registry.

Any ideas?

--
This mailing list is governed under the HashiCorp Community Guidelines - <a href="https://www.hashicorp.com/community-guidelines.html" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwww.hashicorp.com%2Fcommunity-guidelines.html\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGHa_CaKfZaJiwbkUndUQbVGlQCDw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fwww.hashicorp.com%2Fcommunity-guidelines.html\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNGHa_CaKfZaJiwbkUndUQbVGlQCDw&#39;;return true;">https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: <a href="https://github.com/mitchellh/vagrant/issues" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fmitchellh%2Fvagrant%2Fissues\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHxcA5DuHlQlPInMi_k1Pz4pMNqNw&#39;;return true;" onclick="this.href=&#39;https://www.google.com/url?q\x3dhttps%3A%2F%2Fgithub.com%2Fmitchellh%2Fvagrant%2Fissues\x26sa\x3dD\x26sntz\x3d1\x26usg\x3dAFQjCNHxcA5DuHlQlPInMi_k1Pz4pMNqNw&#39;;return true;">https://github.com/mitchellh/vagrant/issues
IRC: #vagrant on Freenode
---
You received this message because you are subscribed to the Google Groups "Vagrant" group.
To unsubscribe from this group and stop receiving emails from it, send an email to <a href="javascript:" target="_blank" gdf-obfuscated-mailto="_WHXxA4rBAAJ" rel="nofollow" onmousedown="this.href=&#39;javascript:&#39;;return true;" onclick="this.href=&#39;javascript:&#39;;return true;">vagrant-up+...@googlegroups.com.
To view this discussion on the web visit <a href="https://groups.google.com/d/msgid/vagrant-up/00ea994e-a6c0-4d65-be3c-2dc0bb3a3ba6%40googlegroups.com?utm_medium=email&amp;utm_source=footer" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/msgid/vagrant-up/00ea994e-a6c0-4d65-be3c-2dc0bb3a3ba6%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/msgid/vagrant-up/00ea994e-a6c0-4d65-be3c-2dc0bb3a3ba6%40googlegroups.com?utm_medium\x3demail\x26utm_source\x3dfooter&#39;;return true;">https://groups.google.com/d/msgid/vagrant-up/00ea994e-a6c0-4d65-be3c-2dc0bb3a3ba6%40googlegroups.com.
For more options, visit <a href="https://groups.google.com/d/optout" target="_blank" rel="nofollow" onmousedown="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;" onclick="this.href=&#39;https://groups.google.com/d/optout&#39;;return true;">https://groups.google.com/d/optout.



--
Alvaro

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/vagrant/issues
IRC: #vagrant on Freenode
---
You received this message because you are subscribed to the Google Groups "Vagrant" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/vagrant-up/5e87422b-28cc-4316-a461-dc1342cf0509%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: How to make vagrant create a new virtual disk for virtualbox AND handle the edge cases?

Alvaro Miranda Aguilera
I think that something in vagrant space will be overkill, better implement a CI/CD for the mysql seed database changes.


Thats just one tool, but if you check around CI/CD Mysql you will get plenty of tools.

so I think will be better implement a workflow that will be more natural in the middle/long run.

Alvaro.

On Mon, Sep 18, 2017 at 3:22 PM, <[hidden email]> wrote:
Hello!

Ahh, good point about the "vagrant destroy" also destroying the disks. I hadn't thought about that.

Anyways, what I wanted to do was, well... as I said above - keep the source code and the DB contents outside of the vagrant machine, so that you could update it or recreate it or whatever without it affecting your work. On the other hand, if you DO want to start with a clean slate, you can just manually delete the DB and source folders, and the next "vagrant up" will just bring it all back with defaults.

Do you think this is an overkill? Maybe I shouldn't bother with it and just keep the DB inside the box?

I suppose a "vagrant destroy" won't be a very often executed command. I use it a lot while putting the machine together and testing out things, but after that... Well, I don't know. I think the machine would get updated now and then so people would need to do a "vagrant destroy"/"vagrant up" to get the latest version... but that's just a guess.

Valts.


On Monday, September 18, 2017 at 3:18:20 PM UTC+3, Alvaro Miranda Aguilera wrote:
Hello

I am not sure I understand the solution very well.

when Vagrant create a vm, lets say:

Template -> Guest1

and you add a new VDI disk to Guest1

it will be exactly the same as putting the mysql db in /var/something/something

because if you do vagrant destroy, vagrant will delete the VM and the VM will cascade the deletion to include the disk that is attached.

So I am missing something in your persistent story?


In the other hand, if you use vagrant triggers, before destroy you could detach the disk and in this way won't be deleted with a vagrant destroy.

Now.

whats the real issue with having developers to delete a local development db ?

There are tools that can track schema and data change that can be incorportated into the code and then other tools can be used to deploy schema/data changes into a new DB if needed.

Ie, FLYWAY.

If you could explain a bit more, I am sure someone here could help better.

Thanks
Alvaro.

On Mon, Sep 18, 2017 at 1:54 PM, <[hidden email]> wrote:
Backstory: I'm making a vagrant box for development. All the software and services are running inside the vagrant box (Oracle Virtualbox, Centos 7), but two things reside outside: the source code and the MySQL DB. The reasoning is that these two things should survive a "vagrant destroy"/"vagrant up" cycle. But if the developer wants to start a completely clean slate, he should be able to simply delete the appropriate folders, and Vagrant will create them anew.

The source code is simply mounted as a shared folder and symlinked where it need to be inside the system. That works well. I tried to do the same thing with the DB - just pointing MySQL at the shared folder - but the world falls apart when the machine is suspended/resumed. So I'm trying another approach - add a separate virtual disk for MySQL data.

Unfortunately, virtualbox is being a b*tch about it. When you create a virtual disk (VDI), it also assigns it a new UUID and then registers both the path and the UUID in its internal storage registry. If a developer subsequently deletes the file, trying to recreate the file with "vboxmanage createhd" will fail because VirtualBox already has a different file registered for that path. Simply placing a new file in that location will also fail because the UUIDs won't match. To do anything, I need to issue a "vboxmanage closemedium" first which removes it from the registry. Unfortunately I cannot just execute that command blindly either, because it will fail (and in turn trip up vagrant) if the file does not exist in the registry. And as far as I'm aware, there's no way I can query the registry from within a vagrant script...

One solution would be to use a new random filename each time, but that would seriously pollute the Virtualbox storage registry. Also, virtualbox likes to complain about invalid disks in its registry.

Any ideas?

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/vagrant/issues
IRC: #vagrant on Freenode
---
You received this message because you are subscribed to the Google Groups "Vagrant" group.
To unsubscribe from this group and stop receiving emails from it, send an email to vagrant-up+...@googlegroups.com.
To view this discussion on the web visit https://groups.google.com/d/msgid/vagrant-up/00ea994e-a6c0-4d65-be3c-2dc0bb3a3ba6%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
Alvaro

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/vagrant/issues
IRC: #vagrant on Freenode
---
You received this message because you are subscribed to the Google Groups "Vagrant" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/vagrant-up/5e87422b-28cc-4316-a461-dc1342cf0509%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.



--
Alvaro

--
This mailing list is governed under the HashiCorp Community Guidelines - https://www.hashicorp.com/community-guidelines.html. Behavior in violation of those guidelines may result in your removal from this mailing list.
 
GitHub Issues: https://github.com/mitchellh/vagrant/issues
IRC: #vagrant on Freenode
---
You received this message because you are subscribed to the Google Groups "Vagrant" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
To view this discussion on the web visit https://groups.google.com/d/msgid/vagrant-up/CAHqq0ey4vV8X1yzFYde9CSA05d7XL_6eVyXhBDABLnYEPOdDxQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.