レナート   Wunschkonzert, Ponyhof und Abenteuerspielplatz   ﻟﻴﻨﺎﺭﺕ

Wed, 20 Apr 2011

systemd for Administrators, Part VIII

Another episode of my ongoing series on systemd for Administrators:

The New Configuration Files

One of the formidable new features of systemd is that it comes with a complete set of modular early-boot services that are written in simple, fast, parallelizable and robust C, replacing the shell "novels" the various distributions featured before. Our little Project Zero Shell[1] has been a full success. We currently cover pretty much everything most desktop and embedded distributions should need, plus a big part of the server needs:

On a standard Fedora 15 install, only a few legacy and storage services still require shell scripts during early boot. If you don't need those, you can easily disable them end enjoy your shell-free boot (like I do every day). The shell-less boot systemd offers you is a unique feature on Linux.

Many of these small components are configured via configuration files in /etc. Some of these are fairly standardized among distributions and hence supporting them in the C implementations was easy and obvious. Examples include: /etc/fstab, /etc/crypttab or /etc/sysctl.conf. However, for others no standardized file or directory existed which forced us to add #ifdef orgies to our sources to deal with the different places the distributions we want to support store these things. All these configuration files have in common that they are dead-simple and there is simply no good reason for distributions to distuingish themselves with them: they all do the very same thing, just a bit differently.

To improve the situation and benefit from the unifying force that systemd is we thus decided to read the per-distribution configuration files only as fallbacks -- and to introduce new configuration files as primary source of configuration wherever applicable. Of course, where possible these standardized configuration files should not be new inventions but rather just standardizations of the best distribution-specific configuration files previously used. Here's a little overview over these new common configuration files systemd supports on all distributions:

It is our definite intention to convince you to use these new configuration files in your configuration tools: if your configuration frontend writes these files instead of the old ones, it automatically becomes more portable between Linux distributions, and you are helping standardizing Linux. This makes things simpler to understand and more obvious for users and administrators. Of course, right now, only systemd-based distributions read these files, but that already covers all important distributions in one way or another, except for one. And it's a bit of a chicken-and-egg problem: a standard becomes a standard by being used. In order to gently push everybody to standardize on these files we also want to make clear that sooner or later we plan to drop the fallback support for the old configuration files from systemd. That means adoption of this new scheme can happen slowly and piece by piece. But the final goal of only having one set of configuration files must be clear.

Many of these configuration files are relevant not only for configuration tools but also (and sometimes even primarily) in upstream projects. For example, we invite projects like Mono, Java, or WINE to install a drop-in file in /etc/binfmt.d/ from their upstream build systems. Per-distribution downstream support for binary formats would then no longer be necessary and your platform would work the same on all distributions. Something similar applies to all software which need creation/cleaning of certain runtime files and directories at boot, for example beneath the /run hierarchy (i.e. /var/run as it used to be known). These projects should just drop in configuration files in /etc/tmpfiles.d, also from the upstream build systems. This also helps speeding up the boot process, as separate per-project SysV shell scripts which implement trivial things like registering a binary format or removing/creating temporary/volatile files at boot are no longer necessary. Or another example, where upstream support would be fantastic: projects like X11 could probably benefit from reading the default keyboard mapping for its displays from /etc/vconsole.conf.

Of course, I have no doubt that not everybody is happy with our choice of names (and formats) for these configuration files. In the end we had to pick something, and from all the choices these appeared to be the most convincing. The file formats are as simple as they can be, and usually easily written and read even from shell scripts. That said, /etc/bikeshed.conf could of course also have been a fantastic configuration file name!

So, help us standardizing Linux! Use the new configuration files! Adopt them upstream, adopt them downstream, adopt them all across the distributions!

Oh, and in case you are wondering: yes, all of these files were discussed in one way or another with various folks from the various distributions. And there has even been some push towards supporting some of these files even outside of systemd systems.

Footnotes

[1] Our slogan: "The only shell that should get started during boot is gnome-shell!" -- Yes, the slogan needs a bit of work, but you get the idea.

posted at: 22:57 | path: /projects | permanent link to this entry | 43 comments


Posted by Don Marti at Thu Apr 21 01:02:14 2011
So, when you boot, log in, start a terminal, and type "echo $$" what do you get?

Posted by Michael at Thu Apr 21 01:57:56 2011
That look great. I am looking forward to be able to support this in Mageia ( but for now, we focus on the upcoming release without systemd, but I will try to push for having theses files supplied for next stable in 6 months ).

Posted by Anonymous at Thu Apr 21 02:37:57 2011
What about /etc/hosts?  Have you considered providing a libnss-systemd that integrates with systemd-hostname, so /etc/hosts never needs to change again?

How about /etc/motd and /etc/motd.tail, the mechanism which adds the kernel release to the default motd?  That could use a shell-ectomy as well.

Do you handle loading and saving the ALSA mixer state?

Regarding module loading, do FUSE, KVM, and cpufreq, all support loading modules automatically without an /etc/modules-load.d/*.conf file?

Posted by Anonymous at Thu Apr 21 02:40:29 2011
Why does machine-id live in /etc if it doesn't represent configuration?  Shouldn't it go somewhere in /lib?

Posted by Anonymous at Thu Apr 21 02:43:53 2011
The documentation for /etc/locale.conf mentions that it cannot set LC_ALL, but it doesn't say why.  Why can't /etc/locale.conf set LC_ALL?

Posted by Lennart at Thu Apr 21 02:46:20 2011
Anonymous, there is nss-myhostname a smart person has written years ago. With that installed editing /etc/hosts should not be necessary anymore. I can only recommend distributions to ship that installed by default.

/etc/motd thing is a Debianism afaics. (And a pretty questionnable one if you ask me.) If this is really useful then it is trivial to patch pam_motd to optionally prefix its output with the kernel release data. I am pretty sure the PAM maintainers would not have a problem with merging such a patch. This just needs someone to sit down and do it (don't count on me, since we don't have that on Fedora).

ALSA nowadays comes with simple udev and systemd integration out-of-the-box.

FUSE is autoloaded via devnode. KVM and cpufreq are still left in the cold, but pretty soon CPU-model and CPU-feature based module autoloading should be available in the kernel. As soon as that is done I execpect those modules to be loaded via that mechanism instead of static listings.

Posted by Anonymous at Thu Apr 21 02:48:24 2011
Does "Project Zero Shell" include zero shell in initramfs?

Posted by Anonymous at Thu Apr 21 02:53:46 2011
I knew about nss-myhostname, and it rocks.  I just wondered if the more sophisticated hostname handling in systemd-hostnamed required a corresponding nss module, or if nss-myhostname would still Just Work.

Regarding motd, I personally agree that it seems questionable; I just know that my current set of boot-time shell-scripts implemented it, so I wondered what would replace it. :)  /etc/issue actually seems quite a bit more sane, since it has escape sequences that insert the current hostname and kernel version.

Thanks for the quick and thorough answers!

Posted by Lennart at Thu Apr 21 02:53:51 2011
/etc/machine-id is read-only configuration data which is different for each installation. Stuff in /lib is identical for all installations OTOH.

LC_ALL is mostly a debug tool which overrides all other LC_xxx variables. On a normal system LANG= should be set as default, and the the individual LC_xxx= variables are useful to override specific categories. But using LC_ALL in addition to it would make these settings redundant, since it overrides them all. Hence: use LC_ALL on the shell, for debugging, for hacks. But don't use it in system configuration.

Posted by Lennart at Thu Apr 21 02:57:19 2011
Anonymous: No plans regarding initrd.

No, hostnamed does not require a special NSS module. But I actually considered merging nss-myhostname into systemd proper, and call it nss-systemd or so, in order to make clear that that's what people should really use, and so that people stop editing /etc/hosts all the time.

Posted by Anonymous at Thu Apr 21 03:06:45 2011
Awesome; that answers all the loose ends I had about the migration to these new configuration files.  I look forward to seeing this in place across various distributions.  Thanks!

Merging nss-myhostname into systemd seems like an excellent idea.

Posted by Michele at Thu Apr 21 07:56:37 2011
This truly rocks. I'd love to see some of the useless current distro differences cleaned up.

Kepp up the good work. This is groundbreaking :)

Posted by Ralph Aichinger at Thu Apr 21 08:59:15 2011
I really love how after one or two decades of stasis and immobility finally somebody steps up and renews basic system functionality.

One wish: Would it be possible to combine these blog postings into a single document and include them in the systemd documentation?

Sometimes I think it is this type of documentation, that explains design decisions and sets them into context of actual use, that is most lacking in Linux.

Please keep up this great work!

/ralph

Posted by Matt Walton at Thu Apr 21 09:13:50 2011
I've been reading these articles with great interest, and you're really selling systemd to me. I hope you're also selling it to distro teams, and that I won't have to wait too long for OpenSUSE to start using it.

Posted by hefti at Thu Apr 21 14:20:09 2011
Sorry, i have to bring it up, but your config file design doesnt pleases me. Some depend on location, on documentation (:DOSWin:M::MZ::/usr/bin/wine: wtf?) and basically look radomly hacked together. While the last one being very subjective and i can see a somewhat POSIX relation it's just true. /etc is a human/program_in_all_sorts_of_languages interface which is ourdays often even maintainend by programs.

It's not that you had to invent a format yourself, there are plenty available, yaml just to mention one.

Concise, easy to read for both humans and mashines and distinct. It might even help to get rid of the LSD inspired POSIX some time.

Protobuffers would also be nice possibility to take this hole config thingie one step ahead.

Missed opportunity. :(

Posted by Ben at Thu Apr 21 14:48:34 2011
@hefti:

Meh, just write a simple settings parser/handler program in python, which takes simple strings for properties and values (or quoted complex values), and writes them into human-readable configuration files. You could even allow for adding comments, if you wanted to make the interface read/write.

The parser could either be run as a daemon, or be called by programs or through dbus. If there aren't too many objections to Python as a dependency, it could even be made a standard (but I'm sure some distributions would prefer to avoid the dependency; I guess a simple C/C++ program could almost as easily be made, but a compatible file structure would be preferable).

Posted by Lennart at Thu Apr 21 15:09:08 2011
hefti: the plan was explicitly not to come up with new config file formats, but to reuse well-known and understood ones. The environment block like files are deeply entrenched in Linux, for example /etc/sysconfig/* and /etc/default/* all use these file formats. I have never seen YAML to be used by any system component. And it is completely unparsable in shell. However environment blocks can be trivially imported into shell scripts with "."

Posted by Anonymous at Thu Apr 21 16:51:48 2011
I was under impression X11 detects the keyboard layout on the fly nowadays, at least under some circumstances.

lshal | grep input.xkb.layout

Perhaps systemd could use that as a fallback if no default keyboard layout has been configured?

Posted by Lennart at Thu Apr 21 17:47:45 2011
Anonymous: HAL is dead and was never suitable as configuration database in the first place.

Posted by Peter Meier at Thu Apr 21 18:43:39 2011
> [1] Our slogan: "The only shell that should get started during boot is gnome-shell!" -- Yes, the slogan needs a bit of work, but you get the idea.

Ending a nice piece on a sour note with a non-inclusive statement like that just robbed me of all the enthusiasm you built up leading up to it. Couldn't resist, could you?

Posted by oiaohm at Thu Apr 21 18:57:58 2011
/etc/binfmt.d/  For wine really leaves a open issue.

A bug that goes back to the design of binfmt addons.  Wine prefered state is never running as root.  Strictly never.  I have questions about java and mono as well.  Basically I would not be pushing to be too far attached to this solution.  One day it might get altered to a cgroup feature so allowing secuirty.

Exactly what reason should java mono or wine ever need root.

Posted by John (J5) Palmieri at Thu Apr 21 19:45:11 2011
@Peter Meier

It is a joke because gnome-shell has the name shell in it. 

There is some substance to the statement though.  GNOME as a project believes in designing the OS from the Desktop down the stack all the way to the kernel.  It is no coincidence that a lot of the infrastructure work done these past five or more years were lead by people who are affiliated with GNOME in some way.

Those projects are however run in a way that is inclusive to other desktops and to the benefit of all involved.  Getting upset by something that clearly benefits Linux just because the person developing it mentions a project that you may not favor is silly and overly emotional.

Posted by Peter Meier at Thu Apr 21 20:56:07 2011
@John Palmieri

I get the wordplay, I get the fun factor in being a bit provocative, and I even get that the joke's on me for being the one who let himself be trolled quite that easily.

Yet I posted anyway, because I don't think we should be OK with a deterioration of community standards in conduct. One joke does not a crisis make, no. Many jokes spell mobbing, though.

Precisely because systemd is a tech lower in the stack it should aim to be inclusive and open to all above it. And it largely has been, on the technical side - and yes, actions speak louder than words. And yet a single harmless joke can very swiftly undo a lot of those actions and leave people alienated if one's not careful. I submit that it's in systemd's best interest for its spokespersons to be more level-headed and mature and restrain themselves rather than waving the flag for their personal upper-level preferences. Be better at wearing the right hat in the right situation.

I've said my piece, and I'll leave it at that, because certainly there's more productive technical discussion to be had in this comment thread. But frankly, the joke was clearly put there to provoke a reaction - Lennart isn't stupid enough not to be aware what he was doing - so it should come as no surprise that there was one. The sad part is that's probably a satisfying one, but hopefully some readers will think differently.

Posted by Adam Williamson at Fri Apr 22 01:39:10 2011
Peter Meier: um, I think you're over-interpreting rather a lot. It didn't read to me at all as if "the joke was clearly put there to provoke a reaction". That interpretation didn't occur to me in the slightest until I read your comment.

Lennart: this looks pretty nice, but the blog post could probably do with a definition of 'we'; who is this shadowy cabal? :) Always remember a post like this is likely to be spread all over the newsosphere, shorn of context.

Posted by Lennart at Fri Apr 22 02:40:08 2011
oiaohm: processes started via binfmt_misc are run under the same user credentials as the code invoking them. If a process running under a user id "lennart" invokes a Mono program, it will will invoke the mono interpreter also as user "lennart", root is not involved in any way. I fail to see what the problem here is you are pointing out? And I completely fail to see what binfmt_misc has to do with cgroups.

Peter: I am a GNOMEr, I will not lie about that. And I am all for tighter integration between the layers of the OS, and my focus there is clearly GNOME. But I fail to see how that would be excluding anybody else. Just because I will ensure the best integration for the GNOME stack this doesn't mean the KDE folks (or the XFCE, ...) couldn't reach similar integration with the same components, or I would block them or I wouldn't take their contributions. They just have to step up and do it.

Posted by Tobu at Fri Apr 22 12:44:42 2011
Here is what I have in /etc/default/keyboard:

cat /etc/default/keyboard
XKBMODEL="pc105"
XKBLAYOUT="fr"
XKBVARIANT="oss_nodeadkeys"
XKBOPTIONS="lv3:ralt_switch,compose:caps"
# KMAP=/etc/console-setup/defkeymap.kmap.gz

Which is picked up by X11 and the console. vconsole.conf's KEYMAP=fr doesn't cover it. If you want to replace existing configuration with newer sources and give the newer sources priority, they should be at least as expressive as the existing data models.


os-release isn't better than /etc/lsb-release. /etc/lsb-release is already authoritative, and is a good place for a distribution to write its configuration. lsb_release (a Python script where I am) merely parses it and uses the various distro files as a fallback, ensuring that its output is both reliable and portable. Since systemd doesn't need to be portable, it can require that /etc/lsb-release define everything and parse it directly.

Posted by Andreas at Fri Apr 22 13:02:43 2011
Good job everyone involved in this. Linux has too many pointless differences. Is there any work ongoing on standardizing static network interface configuration?

Posted by Anonymous at Fri Apr 22 13:47:00 2011
> > lshal | grep input.xkb.layout
>
> Anonymous: HAL is dead and was never suitable as
> configuration database in the first place.

The point was that the operating system is able to discover the keyboard layout and perhaps systemd could use that information as a fallback when no default is configured.

Whether this information is actually coming from HAL, udev, or any other such component is an implementation detail.

But, as Tobu clearly demonstrated, a simple LAYOUT=fr or so is not enough for X.

X and kbd have lots of differences in layouts and variants they offer, X being much more comprehensive and better maintained these days.

Posted by Lennart at Fri Apr 22 14:25:19 2011
Tobu: There's no point in trying to cover all X options with vconsole.conf. The idea is that X uses the vconsole.conf setting if it has nothing else configured. If the variants and other XKB specialties are configured the right place for that is with X configuration, or in gsettings or wherever.

Anonymous: Those properties originate from user configuration. Unfortunately there is no way to automatically determine the kbd mapping on modern PCs. The USB interface definition has a field for that but it is almost always unset or set to "en". DMI has a field for that too, but it is also unset usually.

Posted by ak at Fri Apr 22 15:10:32 2011
If the keyboard layouts in xkeyboard-config are better maintained than those for the console, wouldn't it make sense to use xkeyboard-config for the console, too, instead of having two different and diverging sources of keyboard layouts?

Posted by Tobu at Fri Apr 22 15:35:10 2011
Indeed. The /etc/default/keyboard mentioned above gets applied to the console (as well as X11 and gdm) through a tool called console-setup.

I appreciate what you're trying to do on standardisation, but some of these could become one more slightly-incompatible conffile.

Posted by Richard at Sat Apr 23 03:49:12 2011
What would also be really nice is a utility that lets one scriptably configure the system. When I set up a new machine, I have a shell script that applies about 100 personal preferences. The nice ones can be done with gconftool-2; the ugly ones have to be done with sed.

Posted by Anonymous at Sat Apr 23 04:53:34 2011
I hadn't looked closely at vconsole.conf yet, but if it really does just contain a layout with no variant flags, then it definitely can't replace the existing uses of console-setup, either for X or for the console.  I really do want a Compose key and similar features on the console, not just in X.

Posted by John Drinkwater at Sat Apr 23 15:56:25 2011
Is MIME dead? was mod_mime just an accident in your past ;) What's the rational in using ".conf" file names if the majority of configuration files you are keeping don't (though some sadly do) have file extensions
Does binfmt.d need to be more descriptive?
What makes tmpfiles plural and binfmt not?
Why on Earth is ".d" helpful to be added to dir names?
Doesn't /etc/tmpfiles.d/*.conf belong in the new /run ?

As ever, I like asking annoying questions,
Much love

Posted by John Drinkwater at Sat Apr 23 16:01:07 2011
Richard,
see the Augeas project. Hopefully this new configuration file clean sweep should reduce code size for that project, and in the future make it redundant - you would just need the standard conf file reading code.

Posted by oiaohm at Sat Apr 23 18:20:58 2011
Lennart "oiaohm: processes started via binfmt_misc are run under the same user credentials as the code invoking them. If a process running under a user id "lennart" invokes a Mono program, it will will invoke the mono interpreter also as user "lennart", root is not involved in any way. I fail to see what the problem here is you are pointing out? And I completely fail to see what binfmt_misc has to do with cgroups."

I was referring to a historic error in binfmt_misc.  It was created before cgroups existed and was created with the idea of allowing binaries that were fine to run everywhere.

Problem is user might have ID lennart then runs su or sudo to step up to root.  Now due to binfmt_misc global actions items that should not be run as root can be very simply.  With wine running as root id does cause some very major problems.  Like files now owning to root in prefix with sudo or in case of su a new prefix created as root user.

Its basically a design error in the way binfmt_misc and it has not been updated to the modern age of cgroups and user only binaries.

Really this is about making root user role clearer.  There are lots of things root user should never need todo.

Posted by Jehan at Sun Apr 24 11:20:41 2011
Hello,

what about the DNS resolving configuration files?

I think most distribution already use the same one: /etc/resolv.conf.
Do you intend on standardizing this file as well?

Posted by oiaohm at Sun Apr 24 14:30:03 2011
Lennart I also forgot to mention something else important.  Once you start using filesystem name spaces to bend the file system a user sees binfmt_misc no longer matches the users reality.

Remember wine can run windows viruses and those can destroy Linux.  Last user you want ever touching wine is root.  Mono also risks being in the same boat.

There is also a big unresolved problem for system administrators trying to secure Linux desktops.  How to do kisok mode.  KDE and Gnome have there own implementations that are incompatible.

Basic kisok mode done without requiring Windows manager or application playing ball would be a great step forward.

When I say basic I really do mean basic.  A simple lists of applications users are allowed to see the existence of or/and not allowed to see the existence of.  Can be done by groups of users.  And a option of an blanket lock from running executables from there own directories.

Ideally done without needing LSM's.

If you think about it there are a lot of applications users should never need to run as root.  Firefox irc clients.....  Also systemd as it replaces out the old sys v init system the number of programs required by root is also reduced.

Of course I don't expect this stuff in the first generations of systemd.  Getting rid of sys v init is a good start.

Posted by Lennart at Sun Apr 24 14:57:46 2011
John, we need to pick a suffix, for thefiles in .d/ directories, since otherwise we'll always end up reading edit and RPM backup files and such like. And we definitely don't want that. .conf is a well established generic suffix for config files, hence we use it. The kernel interface behind /etc/binfmt.d is know as binfmt_misc. The misc suffix is not relevant in this context hence we dropped it but kept the thing named after the kernel interface. And If you don't like the name "binfmt", then bring that up with the kernel folks of 1996 or so. Generally a suffix .d on Linux indidcates a drop-in file directory (even though there are exceptions of this rule). And no, tmpfiles stuff has no place in /run, since these config files are installed by the package manager usually and /run is volatile.

oiaohm: binfmt_misc is a systemd-wide setting. It is up to the wine, mono or java interpreters to refuse to be run as root, if that is desirable, much the same as with any other C or shell program. binfmt_misc enables you to register additional "shebang"-like rules. That's all. And this really has nothing to do at all with cgroups.

Jehan: resolv.conf is pretty much standard on Linux.

Posted by oiaohm at Mon Apr 25 11:28:19 2011
Lennart problem comes down to cases when you need 1 user needing wine and another not.

shebang rules don't avoid file system namespace of cgroups.  Last time I checked  binfmt_misc does avoid file system namespace.  So creating a nice magic hole out of a cgroup protection.  Its a simple oversite of something that should have been updated.  shebang if the interp is not accessible in the cgroup it fails.  The item you can see from the cgroup gets run.

Basically set up all this nice cgroup protection around services then enable binfmt_misc that is a hole maker if its still as defective as it was.

Wine is stuck in a ugly location.  Same with most interpreters.  Testing to make sure permissions are not somewhere wrong someone might want to run wine as root.  Of course this should not be on a multi server system.

Java is also in the bad boat as well.  If you have a service written in java there might be a reason.  The interpreter starting as root blocked will block other functionality.

Script wrapping is possible for wine mono java for items needing to run as root.  This is the problem.  The interpreters cannot be coded with no root lock incase something needs running there.

binfmt_misc Has no really knowledge of cgroups.

Best solution would be to make binfmt_misc a cgroup feature.  So it can be turned on for the users or applications that require it and off for everything that does not.  And while at it make it like shebang obeying cgroup restrictions.

Just making binfmt_misc support filesystem name spaces with it current design will lead to a major fail.  filesystem name space alters around the directories for a particular user so for most people /usr/bin/wine is wine location but for 1 particular user their wine is /system/bin/wine.  Now you are in trouble.  binfmt_misc does not support 2 different settings.  So now it does not work.  What about 3 ... and so on.  whereis and other items might not exist either.

Basically stop trying to pass the buck back to interps for something that is simply bad design.

Thinking of binfmt_misc in the same light as shebang is wrong.  shebang is controllable by cgroups.  shebang can work with filesystem namespace bending even if it requires duplicaiton of files containing a different shebang.  binfmt_misc fails cgroup incompatible need to be reworked. 

Lennart in the process of updating the init system have to one kernel part updated not really bad over all.

Posted by Musrum at Mon May 9 07:45:29 2011
"The only shell that should get started during boot is gnome-shell!" -- Yes, the slogan needs a bit of work, but you get the idea.

How about: "No shell but gnome-shell"

Posted by EdorFaus at Mon Jul 18 23:53:39 2011
> "The only shell that should get started during boot is gnome-shell!" -- Yes, the slogan needs a bit of work, but you get the idea.

Hmm, well, if you want to avoid getting certain non-GNOME people riled up, how about something like "The only shell that should get started during boot is the login shell!" (or: "No shell but the login shell" to merge Musrum's suggestion) ?

I'm not sure exactly how technically accurate that is (isn't the login shell typically started after login?), but it might seem less GNOME-centric at least... (Not that I care much personally, but some people apparently do.)

Posted by nim-nim at Wed Feb 15 10:01:21 2012
/etc/machine-id is nice for most software, but it's not sufficient for inventory people (they really need an hardware id stable across reinstalls and systems, and not linked to a specific inventory system implementation)

Leave a Comment:

Your Name:


Your E-mail (optional):


Comment:


As a protection against comment spam, please type the following number into the field on the right:
Secret Number Image

Please note that this is neither a support forum nor a bug tracker! Support questions or bug reports posted here will be ignored and not responded to!


It should be obvious but in case it isn't: the opinions reflected here are my own. They are not the views of my employer, or Ronald McDonald, or anyone else.

Please note that I take the liberty to delete any comments posted here that I deem inappropriate, off-topic, or insulting. And I excercise this liberty quite agressively. So yes, if you comment here, I might censor you. If you don't want to be censored your are welcome to comment on your own blog instead.


Lennart Poettering <mzoybt (at) 0pointer (dot) net>
Syndicated on Planet GNOME, Planet Fedora, planet.freedesktop.org, Planet Debian Upstream. feed RSS 0.91, RSS 2.0
Archives: 2005, 2006, 2007, 2008, 2009, 2010, 2011

Valid XHTML 1.0 Strict!   Valid CSS!