Episode Details

This week, Allan is out of town, but since when has that ever stopped us from bringing you a new episode of BSDNow? We have news,

This episode was brought to you by

Headlines

Unix's file durability problem

• Another article by Chris Siebenmann from the University of Toronto
• This time, the issue was a lost comment on his Python based blog which uses files on disk rather than a database
• After an unexpected restart of the system, a recently posted comment no longer existed
• The post goes on to investigate what the ‘right way’ to ensure file durability is
• The answer, as you might expect, is “it depends…”
• Normally, fsync() should work, but it seems with ext4 and some other file systems, you must also fsync() the directory where the file was created, or it might not be possible to find the file after a crash
• Do you need to fsync() the parent of that directory too? Then what is fdatasync() for? What about just calling sync()?
• “One issue is that unlike many other Unix API issues, it's impossible to test to see if you got it all correct and complete. If your steps are incomplete, you don't get any errors; your data is just silently sometimes at risk. Even with a test setup to create system crashes or abrupt power loss (which VMs make much easier), you need uncommon instrumentation to know things like if your OS actually issued disk flushes or just did normal buffered writes. And straightforward testing can't tell you if what you're doing will work all the time, because what is required varies by Unix, kernel version, and the specific filesystem involved.”
• Second post by author: How I'm trying to do durable disk writes
• Additional Discussion on Hacker News
• The discussion on HN also gets into AIO and other more complicated facilities, but even those seem to be vague about when your data is actually safe
• At least ZFS ensures you never get half of your new data, and half of your old data. ***

Build a FreeBSD 10.3-release Openstack Image with bsd-cloudinit

• Are you using FreeBSD and OpenStack or would you like to be? We next have a great tutorial which explains the ins-and-outs of doing exactly that.
  
• Remy van Elst brings us a great walkthrough on his site on how to get started, and hint it involves just a few ‘pip’ commands.
• After getting the initial Python tools bootstrapped, next he shows us how to save our OpenStack settings in a sourceable shell command, which comes in handy before doing admin on a instance.
• Next the ‘glance’ and ‘cinder’ tools are used to upload the target OS ISO file and then create a volume for it to install onto.
• Next the VM is started and some specific steps are outlined on getting FreeBSD 10.3 installed into the instance. It includes some helpful hints as how to fix a mountroot error, if you installed to ada0, but need to mount via vtdb0 instead now.
• After the installation is finished, the prep for ‘cloudinit’ is done, and the resulting image is compressed and made ready for deployme