

Security
The RPM package format and tools have long supported SELinux, so that
policies are configured and files
get labeled correctly at installation time. But support for other
security solutions, Smack for example, is lacking in RPM. Elena Reshetova
presented some ideas for rectifying that in her presentation at the 2012
Linux Security Summit (LSS). By adding hooks into RPM
processing, more Linux Security Modules (LSMs) or other security components
could be supported.
Reshetova began with an overview of RPM. The format is used by multiple
distributions, beyond just the Red Hat distributions where
it began. SUSE/openSUSE, Mageia, Tizen, and others all use RPM.
RPM package installation has the notion of a "transaction", which
encompasses all of the packages to be installed or removed in a single
operation. Inside these
transactions are the individual packages or "transaction elements".
Dependency checking is done at the transaction level, so it is only done
once. Scripts to run before the transaction starts and after it ends can be configured in a package specification.
Installing each package entails a series of steps inside the transaction,
starting with the optional signature verification. If that passes (or is
turned off), then the "pre" script is run, the files are unpacked from the
archive and installed, and the "post" script is run.
As might be expected, there are a few other steps (e.g. initialization,
cleanup), but, as depicted in a flowchart (seen at right), pre-unpack-post
makes up
the bulk of the processing.
When SELinux handling was added to RPM, it was done to set up and install the
policies and label the files that get installed. That work was mostly done
in the sepolicy RPM plugin using the existing RPM hooks. But some SELinux
support is in the RPM core, including running the maintainer scripts (pre,
some labeling tasks. The maintainer scripts are run using
rpm_execcon() to
set a particular security context before their execution.
When Reshetova and others working on Tizen started looking into adding
Smack support for
RPM, they realized it needed a more generalized security
plugin interface. Smack requires setting up access control domains and
rules on a
per-package basis, but there are other security mechanisms that have needs as
well. The security policy
for a system or device might trust certain application repositories and
only allow packages from those source to access "sensitive services".
Integrity measurements may need to be bootstrapped, container configuration
established, or seccomp() restrictions enabled, all of which could
be handled by a generalized security plugin.
Currently, there are just a few hooks available in RPM: two before the
pre-transaction script is run, one before the pre script, one after the post
script, and a cleanup hook. Reshetova would like to work with the
LSM developers to create an expanded set of hooks that will serve all of
the LSMs (as well as the other uses). Making the hooks symmetrical, so
that there are hooks both before and after transactions and package
installation/removal, might be the starting point. Adding a hook to wrap script
execution for setting up the proper security context is another.
Currently, the verification step only allows specifying which keys to use
and what should happen if the package does not verify. Adding a hook for
verification would allow for additional checking, such as that the package
was signed by the right key (corresponding to the repository it came from,
as opposed to any installed key, as
RPM checks for currently), and to make security policy checks based on
the originating repository.
The other hooks that Reshetova proposed are associated with the individual
files in a package. Those would allow things like security labeling or
calculating hashes on the file contents (for integrity purposes). The last
hook she proposed is to handle conflicts. If a package wants to install a
file that another package has already installed, the hook could install a
conflict file recording the problem; later hooks could use that file to
make decisions
depending on the attributes of the two packages involved. If one package
is from a more trusted repository, its version could be chosen, for example.
In addition, some environments may have non-native applications that use
their own installer. Those have all of the same problems with handling
security contexts, labeling, and so on. It would be nice to have the
security plugin functionality available as standalone library that could be
used by non-native application installers, Reshetova said.
Once those hooks (or a similar set that is agreed upon) are available, the
SELinux-specific pieces of RPM could be moved out of the core. A unified
layer of security hooks would be beneficial for a wide variety of use
cases, she said. More information is available
on the Tizen wiki and a GitHub
repository contains the proposed changes for RPM.
Dan Walsh asked what the RPM maintainers thought about the changes;
Reshetova said they are interested in seeing a unified solution. They want
to make sure that there is agreement between the LSM developers, which is
one of the motivations for her presentation. The intent would be to cause
no disruption for the SELinux parts in RPM when moving that to the new
hooks, she said in answer to another question. Walsh said that there
really isn't anyone who is the "SELinux/RPM person", but he and others
wouldn't oppose a patch to move SELinux out of the core; "don't break
anything and I'm fine" with it, he said, though he did caution that
performance might be an issue.
Since the summit, Reshetova has started a wider discussion of
the hooks on the SELinux mailing list. It would seem likely that we will
have a more generalized solution for RPM in the not-too-distant future.
Brief items
This project strengthens the ludicrous idea in people's heads that photography is somehow a significant threat to safety or security. Photographic documentation is an extremely important part of modern democracy, and projects like these threaten the ability of people to take pictures.
New vulnerabilities).
A use-after-free flaw was found in the xacct_add_tsk() function in the
Linux kernel's taskstats subsystem. A local, unprivileged user could use
this flaw to cause an information leak or a denial of service.
From the Gentoo advisory:
An error has been found in the way mod_rpaf handles X-Forwarded-For
headers. Please review the CVE identifier referenced below for details.
A remote attacker could send a specially crafted HTTP header, possibly
resulting in a Denial of Service condition.
From the Red Hat Bugzilla entries [1, 2, 3]:
CVE-2012-4400:
A possibility to bypass file upload size constraint was found in the way the webservice script, called from the filepicker front end of Moodle, a course management system, performed sanitization of 'maxbytes' variable. A remote attacker could issue a specially-crafted request that, when processed could allow an attacker to upload a file even if it was larger than specified constraint.
CVE-2012-4402, CVE-2012-4403:
Users with permission to access multiple services were able to use a token
from one service to access another. An attacker could use this flaw,
in an unauthorized way, to access content of an external service.
CVE-2012-4408:
A security flaw was found in the way Moodle course management system performed permission check on course reset page (the course reset link was protected by a correct permission, but the reset page itself was being checked for a different permission). A remote attacker could use this flaw to in an unauthorized way to reset particular course.
From the Red Hat bugzilla entries [1, 2]:
CVE-2012-4401:
A security flaw was found in the way Moodle course management system performed user permissions validation by course topic management. A remote attackers, with course editing capabilities, but without ability to show / hide topics or set the current topic for a particular course could use this flaw to successfully complete these actions under certain circumstances.
CVE-2012-4407: A security flaw was found in the way file serving functionality of Moodle course management system enforced file access restrictions on blog post(s). A remote attacker could use this flaw to deliver files embedded as part of a blog without the publication state to be checked properly.
From the Gentoo advisory:
Multiple SQL injection vulnerabilities (CVE-2012-0811) and cross-site
scripting vulnerabilities (CVE-2012-0812) have been found in
Postfixadmin.
It was discovered that the apt-add-repository tool incorrectly validated
PPA GPG keys when importing from a keyserver. If a remote attacker were
able to perform a man-in-the-middle attack, this flaw could be exploited to
install altered package repository GPG keys.
The tor_timegm function in common/util.c in Tor before 0.2.2.39, and 0.2.3.x before 0.2.3.22-rc, does not properly validate time values, which allows remote attackers to cause a denial of service (assertion failure and daemon exit) via a malformed directory object, a different vulnerability than CVE-2012-4419.
Multiple vulnerabilities have been discovered in VMware Player, Server,
and Workstation.)..
Page editor: Jake Edge
Kernel development
Notable features in 3.6 include TCP small
queues, the client-side TCP fast open
implementation (server side has been merged for 3.7), IOMMU groups, the Btrfs send/receive feature, the VFIO device virtualization mechanism, and
more. See the KernelNewbies
3.6 page for details.
Stable updates: 3.5.5, 3.4.12 and 3.0.44 were released on October 2; each
contains a longer-than-usual list of important fixes.
$ git log --no-merges v3.5..v3.6 | \
egrep -i '(integer|counter|buffer|stack|fix) (over|under)flow' | \
wc -l
31
How many were security relevant? How many got CVEs?
This may include, but is not limited to, topics such as tooling to assist
in securing the Linux Kernel, verification and testing of critical
subsystems for vulnerabilities, security improvements for build tools, and
providing guidance for maintaining subsystem security.
The group intends to discuss a wide range of approaches including tool
development, static analysis, verification efforts, and even the
possibility of tightening the rules for patch signing. Interested people
are encouraged to join in.
Full Story (comments: none)
The plan is to start an OASIS working group which would help in the
development (and standardization) of version 1.0 of the virtio
specification. He is asking for comments on the idea, but few have been
posted as of this writing.
Now, a single report of trouble on the
linux-kernel list has developers questioning the change — or, at least,
whether it should be turned on by default. Linus fears that this report could be followed by
others:
Compatibility is just too important.
Other developers have argued for making the
change as soon as the 3.6.1 stable update. Needless to say, agreement on
this point is not universal; Kees Cook, the author of the change, argues that the benefits far outweigh the
pain. The kernel community is committed to not breaking things that used
to work, though; if this change appears to be causing problems more widely,
it will probably be reversed in the near future.
Kernel development news
Changes visible to kernel developers include:
The 3.7 merge window can be expected to stay open until approximately
October 14. That said, Linus has warned the community that he will be
traveling during this time; he, along with your editor, will be at the
Linux Foundation's Korea
Linux Forum. If the travel interferes with the merging process — which
hasn't been a problem in previous merge windows — this merge window may be
extended to compensate.
Anyone who follows Linux kernel security discussions has probably heard of
the "LSM stacking issue". It is a perennial topic on the mailing lists and
solutions have been proposed from time to time. The basic problem is that
only one Linux Security Module (LSM) can be active in a running kernel, and
that single slot is often occupied by one of the "monolithic" solutions
(e.g. SELinux or AppArmor) supplied by distributions. That leaves some of
the smaller or more special-purpose LSMs—or users who want to use
multiple approaches—out in the cold.
Back in February 2011, David Howells proposed
a stacking solution for LSMs. At the time, Casey Schaufler mentioned a
solution he had been working on that would be posted in a "day or two".
That
prediction turns out to have been overly optimistic, but his solution
has surfaced—more than a year-and-a-half later. He also discussed the patches in a lightning talk at the
recently held Linux Security Summit.
There are three types of LSMs available in the kernel today and there are
use cases for combining them in various ways. Administrators might want to
add some AppArmor restrictions on top of the distribution-supplied SELinux
configuration—or use SELinux-based sandboxes on a TOMOYO
system. The two "labeled" LSMs,
SELinux and Smack, require that files have extended attributes (xattrs)
containing labels that are used for access decisions. The two "path-based"
LSMs, AppArmor and TOMOYO, both base their access decisions on the paths
used to access
files in the system. The only other LSM currently available is Yama, which
is something of a container for discretionary access control (DAC)
enhancements.
Yama is the LSM that is perhaps most likely to be stacked. It adds some
restrictions to the
ptrace() attach operation that Ubuntu and ChromeOS use, and other
distributions are considering it as well. In fact, Yama developer Kees
Cook has proposed making the LSM
unconditionally stackable via the CONFIG_SECURITY_YAMA_STACKED
kernel build option (which was merged for 3.7). Over the years, though,
various other security ideas
have been proposed and pointed in the direction of the LSM API, so other
targeted LSMs may come about down the road. Making each separately
stackable is less than ideal, so a more general solution is desirable.
In addition, combining labeled and path-based solutions manually can't
really be sanely done.
When Howells posted his solution, he explicitly disallowed combining the
two labeled LSMs because of implementation difficulties (mainly with
respect to the LSM-specific secid which is used by SELinux and
Smack, but none of the others). There was also a
belief that mixing SELinux and Smack (or AppArmor and TOMOYO for that
matter) is not a particularly sought-after feature. But Schaufler thought
that was
an unnecessary restriction, one that he was trying to address in his
solution.
As it turns out, Schaufler ended up at the same place. His proposal also
defers stacking (or "composing") SELinux and Smack, noting that it
"has proven quite a
challenge". But he was able to get the other combinations
working—at least to the extent that the kernel would boot without
complaints in the logs. The Smack tests passed as well. Performance for
Smack with AppArmor, TOMOYO, and
Yama enabled is "within the noise", he said.
Schaufler's version ensures that the hooks for each enabled LSM are
called, which is different than Howells's approach that short-circuited
the other hooks if one denied the access. Instead, Schaufler patches call
each LSM's hooks, remembering the last non-zero return (denial or error of
some sort) as the return value for the hook. His argument is that an LSM
could reasonably expect to see—and possibly record information
about—each access decision, even if it has been denied by another LSM.
Much of the "guts" of the changes are described in the infrastructure
patch, which is the largest of the five patches. The others make
fairly modest (if pervasive) changes to SELinux, Smack, TOMOYO, and
AppArmor to support stacking. As it turns out, Yama "required no
change and gets in free". The changes to the individual LSMs are
optional, as they can still be used (in a non-stackable way) without them.
Stacking is governed by the CONFIG_SECURITY_COMPOSER option. If
that is not chosen, all of the existing LSMs function as they do today.
If stacking is built in, the security= boot parameter can then be
used to control which
LSMs are enabled. For example, security=selinux,apparmor will
enable those two. If nothing is specified on the boot command line,
all of the LSMs built into the kernel will be enabled. The
/proc/PID/attr/current interface has also been changed to report
information from any of the active LSMs (only SELinux, Smack, and AppArmor
actually use that interface today).
Existing kernels store pointers to the hooks implemented by an LSM in a
struct security_operations called
security_ops. Schaufler's patch replaces that with an array of
security_operations pointers called composer_ops. That
array is indexed based on
the order that is assigned to each LSM as it is registered. The
first entry (composer_ops[0]) is reserved for the Linux capabilities
hooks. Those have been manually "stacked" into the LSMs for some time, so
entries in composer_ops[0] get zeroed out if one of the other LSMs
implements the hook (as the capabilities checks will be done there). If
there is no entry in composer_ops[0], each of the hooks in the
other entries in that array are called, as described above.
The security "blobs" (private storage for each LSM) are still managed by
the LSMs, but because there are blob pointers sprinkled around various
kernel data structures (e.g. inodes, files, sockets, keys, etc.), a
"composer blob" is used. That blob contains pointers to each of the active
LSM blobs, and new calls are used to get and set the blob pointers
(e.g. lsm_get_inode() or lsm_set_sock()). Most of the
changes for the individual LSMs are converting to use this new interface.
So far, most of the comments have been about implementation details;
Schaufler addressed those in the second version of the patch set. Notably
missing, at least so far, were some of the concerns about strange
interactions between stacked LSMs leading to vulnerabilities that have come
up in earlier discussions. But, without
any major complaints, one would guess some more testing will be done,
including gathering some additional performance numbers, before the
linux-kernel gauntlet will be run. The rest of the kernel developers have
heard about the need for stacking LSMs enough times that it seems likely
that Schaufler's patches (or something derived from them) will eventually
pass muster.
Borislav Petkov was able to reproduce the problem; a dozen or so bisection
iterations later he narrowed down the problem to this
patch, which was duly reverted. There is just one little problem left: the
offending patch was, itself, meant to improve scheduler performance.
Reverting it fixed the PostgreSQL regression, but at the cost of losing an
optimization that improves things for many (arguably most) other
workloads. Naturally, that led to a search to figure out what the real
problem was so that the optimization could be restored without harmful
effects on PostgreSQL.
The kernel's scheduling domains mechanism
exists to optimize scheduling decisions by modeling the costs of moving
processes between CPUs. Migrating a process from one CPU to a
hyperthreaded sibling is nearly free; cache is shared at all levels, so the
moved process will not have to spend time repopulating cache with its
working set. Moving to another CPU within the same physical package will
cost more, but mid-level caches are still shared, so such a move is still
much less expensive than moving to another package entirely. The current
scheduling code thus tries to keep processes within the same package
whenever possible, but it also tries to spread runnable processes across the
package's CPUs to maximize throughput.
The problem that the offending patch (by Mike Galbraith) was trying to
solve comes from the fact that the number of CPUs built into a single
package has been growing over time. Not too long ago, examining every
processor within a package in search of an idle CPU for a runnable process
was a relatively quick affair. As the number of CPUs in a package
increases, the cost of that search increases as well, to the point that it
starts to look expensive. The current scheduler's behavior, Mike said at
the time, could also result in processes bouncing around the package
excessively. The result was less-than-optimal performance.
Mike's solution was to organize CPUs into pairs; each CPU gets one "buddy"
CPU. When one CPU wakes a process and needs to find a processor for that
process to run on, it examines only the buddy CPU. The process will be
placed on either the original CPU or the buddy; the search will go no
further than that even if there might be a more lightly loaded CPU
elsewhere in the package. The cost of iterating
over the entire package is eliminated, process bouncing is reduced, and
things run faster. Meanwhile, the scheduler's load balancing code can
still be relied upon to distribute the load across the available CPUs in
the longer term. Mike reported significant improvements in tbench
benchmark results with the patch, and it was quickly accepted for the 3.6
development cycle..
What is needed is a way to gain the benefits of Mike's patch without making
things worse for PostgreSQL-style loads. One possibility, suggested by Linus, is to try to reduce the
cost of searching for an idle CPU instead of eliminating the search
outright. It appears that there is some low-hanging fruit in this area,
but it is not at all clear that optimizing the search, by itself, will
solve the entire problem. Mike's patch eliminates that search cost, but it
also reduces movement of processes around the package; a fix that only
addresses the first part risks falling short in the end.
Another possibility is to simply increase the scheduling granularity,
essentially giving longer time slices to running processes. That will
reduce the number of preemptions, making it less likely that PostgreSQL
processes will step on each other's toes.
Increasing the
granularity does, indeed, make things better for the pgbench
load. There may be some benefit to be had from messing with the
granularity, but it is not without its risks. In particular, increasing
the granularity could have an adverse effect on desktop interactivity;
there is no shortage of Linux users who would consider that to be a bad
trade.
Yet another possibility is to somehow teach the scheduler to recognize
processes — like the PostgreSQL dispatcher — that should not be preempted
by related processes if it can be avoided. Ingo Molnar suggested investigating this idea:
The problem, of course, is the dragons. The O(1) scheduler, used by Linux
until the Completely Fair Scheduler (CFS) was merged for 2.6.23, had, over
time, accumulated no end of heuristics and hacks designed to provide the
"right" kind of scheduling for various types of workloads. All these
tweaks complicated the scheduler code considerably, making it fragile and
difficult to work with — and they didn't even work much of the time. This
complexity inspired Con Kolivas's "staircase deadline scheduler" as a much
simpler solution to the problem; that work led to the writing (and merging)
of CFS.
Naturally, CFS has lost a fair amount of its simplicity since it was merged;
contact with the real world tends to do that to scheduling algorithms. But
it is still relatively free of workload-specific heuristics. Opening the
door to more of them now risks driving the scheduler in a less
maintainable, more brittle direction where nothing can be done without
a significant chance of creating problems in unpredictable places. It
seems unlikely that the development community wants to go there.
A potentially simpler alternative is to let the application itself tell the
scheduler that one of its processes is special. PostgreSQL could request
that its dispatcher be allowed to run at the expense of one of its own
workers, even if the normal scheduling algorithm would dictate otherwise.
That approach reduces complexity, but it does so by pushing some of the
cost into applications. Getting application developers to accept that cost
can be a challenge, especially if they are interested in supporting
operating systems other than Linux. As a general rule, it is far better if
things just work without the need for manual intervention of this type.
So, in other words, nobody really knows how this problem will be solved at
this time. There are several interesting ideas to pursue, but none that
seem like an obvious solution. Further research is clearly called for.
One good point in all of this is that the problem was found before the
final 3.6 kernel shipped. Performance regressions have a way of hiding,
sometimes for years, before they emerge to bite some important workload.
Eventually, tools like Linsched may help to
find more of these problems early, but we will always be dependent on users
who will perform this kind of testing with workloads that matter to them.
Without Nikolay's 3.6-rc testing, PostgreSQL users might have had an
unpleasant surprise when this kernel was released.
Patches and updates
Kernel trees
Core kernel code
Development tools
Device drivers
Filesystems and block I/O
Memory management
Networking
Architecture-specific
Security-related
Virtualization and containers
Miscellaneous
Page editor: Jonathan Corbet
Distributions
LWN: First of all, what, in your mind, does the term "cloud" really
refer to?
Clearly, "Cloud" is a both marketing term and a hot business
buzzword, neither of which lend themselves to clarity. However, there are
some actual significant changes in computing represented by the word.
On the business side, there's a trend towards centralization of resources —
sometimes described as a big, constant pendulum swing, with "cloud
data center" simply standing in for "mainframe" this time around — but there
are actually interesting new developments which make cloud computing
compelling. It may be that I've been in a university setting too long, but I
like the NIST
definition [PDF] which describes the essential cloud
characteristics. Or, there's the "OSSM" definition
[video], which goes like this:
With this option, if you've got a new startup, putting together your own
data center is suddenly crazy. If you're an agile developer, on-demand
self-service is very appealing. And for larger enterprises, it's amazing to
have built-in scalability and measurability.
Small, nimble companies are already benefiting from cloud computing; bigger
companies are mostly dipping their toes in. There's a lot of interest in
on-premises private cloud, and especially in transparent hybrid cloud (where
local and public cloud infrastructure are mixed together). All of the
technology is still in flux and there are plenty of unresolved questions,
but it's certainly a matter of how-much-how-quickly, not of if-at-all.
On the user and client-platform side, the important trend is mobile and
tablet computing, and the movement away from general-purpose computing
devices to a locked-down "app store" model — even for desktop systems.
Previous attempts at making restricted computing devices always failed in
the market, because while most people only do a few things with their
computers, there's a long tail of different things each person wants,
and any narrow selection of most-common tools was never good enough.
Developer-friendly app marketplace and distribution channels which run right
on the device get around that — even when the platform is locked down,
consumer convenience is served by "there's an app for that". Consumers get
access to tools, developers get a market, and vendors get lock-in and
control. In proprietary operating systems, we're going to see more and more
of that.
[PULL QUOTE:
So, a key role for free and open source Linux distributions is to provide a
viable alternative to the locked-in "consumption device" dystopia. This
includes providing the convenience and functionality of cloud computing that
proprietary platforms offer.
END QUOTE]
There's one key thing the NIST cloud definition covers that isn't in the
OSSM one (to be pronounced "awesome", by the way). That's "broad network
access", and it means you can get to your data from anywhere, from any
client platform. It's the lure of cloud computing from an end-user point
of view — don't worry if you lose your phone, because it's all safe in the
cloud. In fact, don't worry if you're house burns down, because your family
photos are all safely floating out there as well.
At least, don't worry as long as there are sufficient protections in place!
A number of people are saying that open source on the desktop (or any client
device) doesn't even matter — that open web is the new front, and
that the battle is about making sure people have control over and access to
their own remotely-stored data.
I think that's important, and I'm glad people are fighting for it, but I
disagree that it's sufficient. It's not hard to imagine a future where the
normal tech platform most people buy isn't able to run arbitrary code — look
at the boot restrictions for Windows 8 ARM systems. Historically, the
mass-market platform has also been the development platform, and that's done
wonderful, magical things for the democratization of invention. But in a
restricted future, the flexible development platform is a special niche
product — more expensive and maybe even not available to everyone.
So, a key role for free and open source Linux distributions is to provide a
viable alternative to the locked-in "consumption device" dystopia. This
includes providing the convenience and functionality of cloud computing that
proprietary platforms offer, whether it's through open web and open cloud
initiatives, or through building in local cloud and peer-to-peer cloudlike
services.
LWN: How might Fedora fit into that picture?
We want these things in cloud computing as well, and over the next few
months I'll be working with other people in Fedora to identify more specific
outcomes to work toward in cloud computing, and from there we'll develop
programs and activities focused on those outcomes.
We have a strong base in the Fedora distribution, a great worldwide user and
developer community, and excellent infrastructure. And, we have a lot of
cloud-related work in progress.
LWN: Where are people using Fedora in cloudy settings now? How would you like
to see that change in the future?
We're also a huge center for cloud infrastructure software packaging and
development. OpenShift Origin and OpenStack Folsom are two of the big
features for the upcoming Fedora 18 release, along with Eucalyptus,
OwnCloud, and Heat (a cloud orchestration API for OpenStack). We've also got
work in progress on OpenNebula and CloudStack.
Now, I don't think anyone has delusions that a great number of people will
run production infrastructure on top of Fedora with these packages. The main
two use cases are: a) developers of cloud software (including, but
definitely not limited to, Red Hat) working to make sure the software is
ready for future use in enterprise Linux products, and b) cloud
early-adopters who are following that development. Those constituent groups
are going to remain crucial to Fedora in general and Fedora cloud in
specific, and I think there are some gentle adjustments we can make that
will help encourage those relationships even more.
Over the years we've lost a lot of our sysadmin and server-side Linux users,
as they've felt somewhat left behind by all the energy and development
around desktop. Sometimes it's seemed like the only voice available to that
group has been a negative one — "Hey, slow down!" — which is a frustrating
side of the conversation to be on, when really we all want to work together
to make better software and a better world (back to that vision statement).
When you're cast in the stop-energy role, it's hard to feel listened to, let
alone constructive.
So, the rise in cloud and all the exciting new development work gives that
group a positive voice. That's good for Red Hat Enterprise Linux as a
downstream project. It's good for developers of cloud software because we'll
make sure their code works in the distribution. And it's good for Fedora,
because this is a large group of technology professionals with years of
experience and wisdom.
I especially want to expand what we can do for the small, agile
organizations working at the leading edge of cloud technology. For this, the
Fedora focus isn't so much on the cloud infrastructure software as on the
cloud guest images.
As I mentioned, we've got an up-to-date EC2 image, and we're working on
making that even more lightweight and on offering more variants for
different use cases. We also support appliance creation with BoxGrinder
(a
Fedora 15 feature), and JEOS ["just enough operating system"] builds
with Oz. Or if you're looking for
something more complete — for example, to make a virtualized developer's
desktop — it's easy to make an image
from our Live CD ISO (although we need
to make that several steps easier).
Producing images isn't enough by itself, though. We also need to increase
our engagement with the DevOps community. At this point, that means mostly
listening and being present, more than direct evangelism. Fedora has always
aimed at users who might become contributors, and the DevOps world is a
natural fit, with good alignment with our values of "freedom, friends,
features, first".
On a technical level, we see the continuous struggle between having the most
up-to-date versions of specific software (Ruby gems, for example), while
having a base that you just don't have to worry about. That's a hard
problem, but in order to be relevant to actual users, we need to address it,
whether it's through Software
Collections, with something related to
OpenShift
cartridges, or in some other way.
We're also looking into how we can make better use of cloud technology for
Fedora developers. The Fedora Infrastructure Team has deployed and is
evaluating Eucalyptus and OpenStack for package mass rebuilds, for automated
(and non-automated) testing, and eventually for more.
The Fedora
features process has been great for the distribution as a whole
and is successful overall in helping us look at development in a
goal-oriented way. I want to make it easier for Fedora cloud contributors –
both new and already involved – to participate in this. As with any process
in software engineering, the "meta-work" involved is often very painful to
the actual implementers, sometimes simply because of the context switches
required, not because the actual paperwork is particularly onerous. That
wastes valuable developer time and produces less than ideal process results.
We need to have people – like, for example, me – involved in the various
features who can both keep up with the technical work and keep meta-work
from being a burden to developers, while still reaping the benefits of good
process.
For end-user and desktop cloud services, we haven't yet explored all of the
possibilities. We can make it easier for moderately-savvy users to set up
their own private infrastructure using SparkleShare or OwnCloud as
alternatives to proprietary hosted filesharing. Our OpenStack packages will
include Puppet modules making it trivial to get a fully-functional private
cloud out of the box, and we can extend that to other parts of the
distribution as well. We'll also continue to look at what more we can do to
enable users to connect themselves to open cloud offerings in ways that
align with the project mission. It'd be nice to provide a push-button
process where open source web applications can be deployed either locally or
pushed to a cloud provider — a tie in to Red Hat's OpenShift, say, or to any
open cloud provider.
LWN: Cloud providers tend to try to project an image of solidness and
stability. How well does that fit with the relatively bleeding-edge nature
of the Fedora distribution?
But, those cloud providers should use Fedora for their testbeds,
precisely because the field itself is on the bleeding edge and we can follow
it more quickly. For example, QEMU 1.2 was pulled into Fedora right after it
was released, and we track the upstream kernel closely, which means we get
the latest hardware and low-level virtualization support. Also, Fedora is
upstream for RHEL and is home to a lot of the exciting development activity
done by Red Hat. If you have an open source / free software cloud project
that you want to work in that ecosystem, Fedora is the natural path.
Meanwhile, those cloud providers should be interested in supporting Fedora
as a guest OS, because it provides value to their users. It's important to
stress that while Fedora is fast-moving, every actual release is
production-ready and stable. I won't claim perfection, but we have an
excellent quality engineering team that takes this very seriously. While we
develop on a six-month development cycle, we make sure it works before we
release.
LWN: Do you see Fedora's relatively short support cycle as being a problem for
cloud deployments? How might any such problems be mitigated?
There's always some background talk about a rolling release, but I like the
cadence of actual releases, with planning and a features process, and
the quality assurance work and release engineering would be hard to
replicate in the rolling model. We have ongoing work on in-place
upgrades
using yum, and further polish to that will reduce the pain of upgrades. The
model I mentioned earlier where OpenStack configuration is shipped as a
Puppet module is also a good direction, since that helps abstract away the
specifics of the underlying system.
Right now, we support upgrades going two releases back, which fits the
Fedora lifecycle, but we also want to make sure people who miss that mark
aren't entirely left behind. One specific thing I'm working on is putting
together tools and resources for the many people still running on Fedora 8
in Amazon EC2.
We've also had a continuous discussion in Fedora about the fire hose of
updates during a release's lifetime. Bringing that under control is
important for real-world use, but we don't want to keep users from getting
new features quickly or prevent developers and packagers from getting their
code out. I'm very much in support of the idea of bundling non-security
updates into monthly sets which can be tested and installed in a more
controlled way. I also strongly prefer to see development targeted at
Rawhide and future releases, with a focus on stability for the thirteen
months of a release's maintenance cycle. Or, beyond that, a focus on
stability for the first twelve months and a focus on smooth upgrades for the
final one.
LWN: What compelling features does Fedora bring to the cloud setting now? What
kind of things do you think need to be done to make Fedora more interesting
in this area?
But, beyond software and beyond technology, we bring an important set of
values and a vision for a free, open, and collaborative future which is
just as vital in a cloud computing future as it is in the older
local-computing model.
What needs to be done to make Fedora more interesting in the cloud setting?
In addition to what I've already talked about, I think most crucially we
need to increase community involvement and build up our user and contributor
base. That's part of my job, too, and I want to help Fedora respond to what
the community needs. I'm working hard to listen, discuss, read, and absorb
as much as possible. We need input from everyone who wants to make Fedora
better, and who is interested in extending the benefits of free and open
source software to the cloud.
LWN: "Cloud computing" often seems like a return to centralized computing with
very little end-user freedom or control. That vision seems mildly
incompatible with the first of the posted Fedora "Foundations." How can
the two be reconciled so that Fedora brings more freedom to cloud
computing?
But, at the user level, when we're talking about software as a service and
about protections for privacy with remote data, it's a legitimate worry.
Fedora needs to offer better alternatives: connections to services which put
privacy and freedom first, the ability to easily stand up cloud services
under your own control, and, finally, a way to work which doesn't
have to be cloud-dependent.
LWN: What sort of tools and/or superpowers has Red Hat given you to get this job
done?
Seriously, though, the primary tool is connections with people – in the
Fedora Project, at Red Hat, and in the open cloud community in general. I'm
very accessible by email, including in the Fedora Cloud SIG and development
mailing lists, and I'm trying to get my 1990s-era IRC habits back up to
snuff (user mattdm on freenode). I'll also be visiting a lot of
conferences, and I hope to see anyone who has read this far, and those of
you I can't meet in person I hope to talk with in other ways.
If you're interested in making Fedora better through cloud computing, or in
making the cloud better through Fedora, please join us in the Cloud SIG. There's no
experience necessary and no formal join process — just jump into the
mailing list and we'll get started.
Many thanks to Matthew for taking the time to answer our questions in such
detail. We are most interested to see where this effort will go.
So when someone emails or approaches you with something they’re excited about, please reply thinking “What can I do to help?. Often I limit my commitment to an encouraging and thoughtful response: a perfectly acceptable minimum. You might want to go further and offer pointers or advice, but take care to fan that delicate flutter of enthusiasm without extinguishing it. Other forces will usually take care of that soon enough, but let it not be you.
Update: Slackware ARM 14.0 is also available.
Distribution News
Debian GNU/Linux
Full Story (comments: 9)
Newsletters and articles of interest
Page editor: Rebecca Sobol
Development
Ian Romanick works on Mesa, an
open-source implementation of the OpenGL specification. His presentation on
the final day of the 2012 X.Org Developers' Conference looked at what he
hoped would be the future of the OpenGL interfaces on Linux. His talk was
broken into three broad areas: the current status of the OpenGL interfaces,
where they should go in the future, and how to get to that future.
The current OpenGL ABI was defined in 2000. It consists of a number of
pieces. The first of these is libGL, which implements three components: OpenGL 1.2, GLX up to version 1.2 (the
current version of GLX is 1.4), and the ARB multi-texture
extensions. Ian highlighted that libGL on its own is not sufficient for
any useful applications these days. The remaining pieces of the OpenGL ABI
are libEGL, and two separate
libraries, libGLES_CM and libGLESv2, for versions 1.2 and 2.0 of OpenGL ES.
There are many problems with the current situation. Applications that
want to use graphics components beyond OpenGL 1.2 have to "jump through
hoops." It's even more work to use GLES with GLX, or to use
desktop OpenGL with EGL. The
implementation of indirect
rendering, a feature that allows OpenGL commands to be encapsulated in
the X protocol stream and sent over the wire to the X server, is
"completely fail": it performs poorly and supports OpenGL up to only
version 1.4—or 1.5 with a lot of effort. The specification requires
indirect rendering to be supported, but the number of legitimate use cases
is quite small. And the presence of this rarely used feature sometimes
creates problems when applications accidentally trigger indirect rendering
and force OpenGL back to version 1.4, leading to user "rage" and bug
reports.
Ian then went through the proposed solution. The first step is to
split libGL, so that the OpenGL and GLX components are separated into
different libraries, called (say) libOpenGL and libGLX. Andy Ritger has
sent out a detailed proposal to the
mesa-dev mailing list describing how the split could be
accomplished. Splitting the libraries will allow applications to mix and
match components as needed, so that, for example, GLES and GLX can be
easily used together by linking with the right libraries. Using both OpenGL
and EGL together would become similarly straightforward. To maintain
backward compatibility for old binaries that look for libGL, it would still
be necessary to provide a legacy version of libGL that glues libOpenGL and
libGLX together.
Among the problems to be solved during the split are how to version the
libOpenGL library and "get away from at least some of the
GetProcAddress madness." That "madness" exists because the current
ABI forces some applications to make calls to various "GetProcAddress" APIs
(similar in concept to the GNU dynamic linker's dlsym() API) in
order to obtain the addresses of multiple functions in the various
libraries that constitute the OpenGL ABI. How the libOpenGL library would
be versioned is an open question. Ian noted that the possibilities included
ELF library versioning or embedding the version number in the library name,
as is done with GLES. He also speculated about whether it would be possible
to bump up the minimum OpenGL version supported by the ABI. The current
implementation is required to support OpenGL versions as far back as
1.2. However, OpenGL 1.2 is now so old that it is "useless", though Ian
still sees occasional bug reports for version 1.3.
Once the library is split, Ian would like to see GLX deprecated. In
addition to the problems caused by direct rendering, adding new GLX
extensions is painful, because support must be added in both the client
and the server. This can create problems in the cases where support
is not added simultaneously on both sides: the client may end up sending
unsupported protocol requests to the server and "they die in a fire." One
recent fire-starter was the GLX_ARB_create_context feature:
support appeared in the X server only in September, more than a year after
client-side support was added to GLX. By contrast, EGL does not have this
problem because support needs to be added only on the client side. In
other words, getting rid of GLX will allow new features to be shipped to
users much more quickly.
A prerequisite for deprecating GLX is to have Linux distributors ship
EGL by default. Most distributions do provide EGL, but Ian supposed that it
is not generally included in the default install. However, Martin Gräßlin
said that KDE started optionally depending on EGL about two years ago, so
it is now part of the default install in most distributions. Later, Ian
noted that encouraging the move to EGL may require the creation of a
GLX-to-EGL porting guide; while there is independent documentation for both
GLX and EGL, there seems to be none that explains how to port code from one
to the other. A lot of the required source code changes can be accomplished
with some simple scripting, but there are also a few deeper semantic
differences as well as a few GLX features that don't have direct
counterparts in EGL.
Another important step is to make OpenGL ES part of the OpenGL ABI. Bart Massey's XDC2012 presentation bemoaned
the fact that developers are not making applications for X. Ian said that
the reason they are not is that they're too busy making applications for
mobile platforms. So, by enabling the OpenGL ES ABI that every developer
uses on mobile platforms, it becomes possible for developers to use X as a
test bed for mobile applications; it also becomes possible to port
applications from Android and iOS to X.
One final step would be to update the loader/driver interface. This
interface defines the way that libGL or libEGL talks to the client-side
driver that it loads, and the way that the GLX module in the X server talks
to the driver to load and route GL calls to it. This will probably be the
hardest step, and it may take some time to resolve the details. As a side
note, Ian pointed out that if indirect rendering is dropped, it will
probably make the task quite a bit easier, because it won't be necessary to
support the loader/driver interface inside the X server.
Ian's presentation was followed by some audience discussion of various
topics. There were some questions about compatibility with old
applications. Ian thinks that compatibility requirements mean that it will
probably be necessary to ship a legacy version of libGL for the indefinite
future. There was some discussion on how to handle multiple versions of
libOpenGL in the future. Some audience members seemed unclear on what
options were available, but others were confident that ELF symbol
versioning, as used in the GNU C library, would be sufficient. Later,
Chad Versace expressed concerns about ensuring that any proposed solution
also worked when using the Clang
compiler and the gold linker. Ian noted
that there will need to be some more investigation of the requirements and
build environments before any final decisions are made.
Bart Massey expressed concern that indirect rendering seemed to be
going away with no replacement in sight. He noted that he'd had students who had
been happy users of indirect rendering for use in certain environments with
limited hardware. Ian suggested that a VNC "send the pixels across the
wire" type of solution might be the way to go. Later, Eric Anholt suggested
that the loader could detect that it is communicating over the wire with
the X server, open the driver as normal, and then transmit images over the
wire with PutImage, with the application being more or less
unaware of this happening.
There are still many details to be resolved regarding the proposed
reworking of the OpenGL ABI. However, there seemed near unanimous agreement
that the proposal described by Ian was the right direction, and it seems
likely that, once some design details have been resolved, the work will
commence quite soon.
The X.Org wiki has a pointer
to the video of this presentation.
Full Story (comments: 59)
The Tizen project has unveiled the first release on the road to its next major release, 2.0. The 2.0 alpha includes both source for Tizen itself and a new build of the SDK. Additions include HTML5 API coverage and a "platform SDK" based on OBS. It is still an alpha, however, and the announcement notes that "there are additional components that we plan to add in the coming weeks, and we will continue to fix bugs and add additional features."
The Blender Foundation has released "Tears of Steel," the short film produced during its Mango open movie project. As with previous open movie efforts, the production process was used to develop and implement new functionality for Blender — this iteration focused on the visual effects pipeline for compositing with live action. The short is available via YouTube or direct download now (as are source files); DVDs are still to come.
Full Story (comments: 8)
Newsletters and articles
The H takes a look at the recent 3.0 release of the Joomla web content management system. Changes include a new administrative interface, use of Twitter's Bootstrap framework, and significant changes under the hood. "Thanks to jQuery, Joomla now supports drag and drop, finally allowing items in the backend interface to be sorted using the mouse. The Mootools JavaScript framework is still included for backwards compatibility, but it is scheduled to be removed from Joomla eventually."
Page editor: Nathan Willis
Announcements
The Foundation has also started a fundraising campaign. "'."
Articles of interest
Education and Certification
Upcoming Events
If your event does not appear here, please
tell us about it.
Linux is a registered trademark of Linus Torvalds