All VMs are currently affected by the “Dirty Cow” kernel bug. The upcoming release 2016_034 contains a kernel update which upgrades Linux to the unaffected version 4.4.28. As usual, the kernel update requires to reboot all VMs.
- Tue 15 through Thu 17 November 2016: reboot staging VMs
- Thu 17 through Thu 24 November 2016: reboot productive VMs.
VM reboots will be scheduled along the agreed maintenance windows. We will piggy-back a Qemu binary environment update which would require a separate reboot otherwise.
Back in May I introduced you to the development of vulnix, a tool which initially was done to find out whether a system (might) be affected by a security vulnerability. It does this by matching the derivations name with the product and version specified in the cpe language of the so-called CVEs (Common Vulnerabilities and Exposures). In the meantime we introduced the tool to the community at the Berlin NixOS Meetup and got some wonderful input in which directions we might extend the features. We sprinted the next two days to improve the code quality and broaden the feature set.
Continue reading Vulnix v1.0 release
We are planning to implement some cool stuff for the Flying Circus hosting platform and its underlying infrastructure during the second half of this year. In this post, I will give a preview to technical improvements you can happily look forward to.
All of these improvements are included in the platform subscription (this is what the platform subscription is actually for!) so you don’t have to pay extra for any of them.
Continue reading Sneak Preview: Upcoming FC Platform and Infrastructure Features
During our daily work with deployments it is kind of routine to customize a package downloaded from the internet. In most cases this means downloading a tar file, extracting it and finally to modify some of its content to fit our needs.
This blog post will show you how this can be done during the deployment using batou and without fuzzing around with command line options of tar.
Continue reading Downloading and extracting files with batou
Migrating PostgreSQL from one version to another has become pretty easy. Using pg_upgrade it takes a few seconds to upgrade even a 100GiB database. It becomes interesting when you switch platforms, say from 32bit to 64bit, as we are facing it with our switch from Gentoo to NixOS. Our NixOS-based platform is stable enough for customers to use it. Some larger databases benefit especially from larger RAM sizes.
So the question is: How to migrate from 32bit to 64bit with as little downtime as possible?
Continue reading PostgreSQL version migration
We have been using Ceph since 0.7x back in 2013 already, starting when we were fed up with the open source iSCSI implementations, longing to provide our customers with a more elastic, manageable, and scalable solution. Ceph has generally fulfilled its promises from the perspective of functionality. However, if you have been following this blog or searched for Ceph troubles on Google you will likely have seen our previous posts.
Aside from early software stability issues we had to invest a good amount of manpower (and nerves) into learning how to make Ceph perform acceptably and how all the pieces of hard drives, SSDs, raid controllers, 1- and 10Gbit network, CPU and RAM consumption, Ceph configuration, Qemu drivers, … fit together.
Today, I’d like to present our learnings both from a technical and methodical view. Specifically the methodical aspects should be seen in the retrospective of running a production cluster for a comparatively long time by now, going through version upgrades, hardware changes, and so on. Even if you won’t be bitten by the specific issues of the 0.7x series in the future, the methods may prove useful in the future to avoid navigating into troublesome waters. No promises, though. 🙂
Continue reading Ceph performance learnings (long read)
Our customers at times require Oracle Java for their applications. Our new platform is based on NixOS. As with most Linux distributions, Oracle Java cannot be installed just like that. Oracle’s license prevents redistribution or direct downloading from their servers. NixOS is no exception there.
While manual installation is pretty straightforward on NixOS, ultimately an automated process is what makes operators happy. We use Batou for this.
Continue reading Automatic installation of Oracle Java