Bias in the Open Source OS World

24 08 2007

I like seeing posts like If you want the features of Solaris, then just use Solaris! cropping up. Not because I think everyone should use Solaris (which I do, but that is beside the point) but because it shows that some people still think about choice of OS rationally. There is a lot of bias among open source operating systems users – most of it completely blind. I won’t pretend to be innocent of this – I was very involved with Brown’s LUG in the past, and was for a time one of the Linux zealots (mostly desktop users) you will occasionally encounter, who treat anyone who favors any other OS with great disdain, and even resentment. Fortunately I mellowed out.

Why should you pick an OS? You should pick the one which best suits your needs. And you should always be open to the possibility that a different OS may serve your needs better than your current one. The main criteria for deciding this tend to be ease of general desktop use, support for common software uses (e.g. Firefox, Thunderbird, office suites), and hardware support.

But in the world of open source UNIX-like operating systems, after configuration, for most desktop users it doesn’t matter. You can use any CLI shell, and you can use any windowing environment. And most open source software will compile on (and is generally available in a package form for) any UNIX-like OS. Most desktop users only go through one main round of software installs, so as great as tools like apt-get are, most people won’t get that much use out of them. For a few the licensing and copyleft ideaology are really key. But for most desktop users, the deciding factor is ease of initial configuration – this is something that more polished Linux distributions* like Debian, Ubuntu, and Red Hat do well, and they certainly deserve credit for it (yes, I said Debian is polished – mostly because of apt-get). Do you really want to be a (possibly unjustified) elitist snob over some nice configuration utilities?

Developers and power users make use of a much wider range of software – compilers, code editors and IDEs, debuggers, more sysadmin-ish command line utilities for resource management and so on and so forth. These are the areas where differences start to emerge. Any of the BSDs, any Linux distribution, Solaris, and with Cygwin even Windows, can run vim/emacs, Eclipse, gcc, gdb and so forth. Want to track down a kernel bug easily, either on a live kernel or on a real, complete crash dump? Solaris is the clear winner: kmdb. Want to track down performance problems easily using a kernel tracing framework that goes through every part of the system and has no performance hit when it’s disabled? DTrace is only available on Solaris, and hopefully soon (eventually?) will be stable on FreeBSD. Need top-of-the-line security? You’re looking at Solaris and OpenBSD. Need fine-grained resource control? Solaris. A copy-on-write shadow-paging filesystem with cheap snapshots and a variation on RAID which can withstand two disk failures in a single disk array? You’re either looking at ZFS on Solaris and FreeBSD, or forking many thousands of dollars over to NetApp (there are other similar filesystems in development, but they’re not very far along). In-kernel support for some variety of native (same-OS) virtualization? FreeBSD, Solaris, and Linux all have their own versions, with their own ups and downs.

Notice any patterns in those lists? Like the fact that FreeBSD has adopted many of the innovations Solaris created and provides as open source? Or the fact that Linux has not? Why is that? Partly it’s licensing – the CDDL and GPL are not compatible, so code written under the two licenses cannot be linked and redistributed. Part of it is the fact that many of the notable Linux kernel developers carry this bias I described earlier. Some of you probably noted the fact that I did not mention SystemTap when I mentioned DTrace earlier. Look at the comment battle on or Adam Leventhal’s comments on SystemTap at OSCON and comments responding to his proposal for a DTrace Linux port. You’ll see why.

Andrew Morton has also made some small news with a wonderful quotes which have already been commented on by someone else.

Don’t get me wrong – I love Linux. I use it almost every day at school. I used it exclusively for a number of years, with great pleasure. But now that I’ve seen what else is out there, I sometimes wonder if it’s lack of innovation that’s holding Linux up, or the attitudes of its leaders and much of its community, of elitism and NIH (not-invented-here) syndrome. There are so many smart people involved, I strongly suspect the latter.

* For the BSDs: I haven’t played with PC-BSD, though I’ve heard good things. I’ve had bad experiences installing FreeBSD. Installing OpenBSD was a breeze, but getting *any* software installed on it was a royal pain in the ass. At some point I want to get around to playing with DragonflyBSD and it’s new vkernel take on virtualization.


STM and Haskell Attention

12 08 2007

Haskell has been garnering a fair bit of attention of late, largely as a result of Simon Peyton-Jonespresence at OSCON (and the ensuing flurry of Reddit postings of related blog posts). He gave a keynote on Software Transactional Memory (STM), which is an alternative to using explicit locks and condition variables in writing parallel programs. He also gave an introduction to Haskell, and a smaller talk on data parallelism.

I’m quite pleased by this recent attention, for a couple reasons.

The first is that I really agree with Simon that STM is going to be the way to go soon*. I also just think it’s damn cool – that’s why I did an independent study on it with Maurice Herlihy last semester, working with a slightly more recent version of Microsoft’s SXM implementation of STM for C# than that available at the link I just gave. Simon gives excellent motivation for STM in his talk, but there are still others if you’re unconvinced after watching it. Consider a datastructure which you traverse in multiple directions – perhaps a grid, which can be traversed in either direction in either dimension depending on the contents at a given location – or perhaps a mutable graph representation. What locking scheme do you use for scalable (read: not coarse-grained) access to a large instance of this datastructure? You could establish a total ordering of nodes and corresponding locks, or do a row/column at a time with an ordering on rows/columns in a table, and acquire locks only in that order. But since you can traverse in any direction, this would sometimes require building lists of locks to acquire, releasing later locks which were acquired in reaching the current point of traversal, and then walking the list reacquiring locks. How expensive is this, possibly acquiring the same lock over and over? How hard is it to write that code correctly? How much effort does it take to document, and for a new maintainer to learn, this locking scheme? All of this makes fine-grained locking a poor idea at best.

How much easier would it be to write such code naïvely and mark it as atomic? And with a proper implementation of STM, how much faster could this be on a large structure than a coarse-grained lock, or repeated acquisition and release of the same fine-grained locks? And this is a bonus on top easier coding for simple concurrent structures (and partial failures), and composition of atomic transactions, which cannot be done with normal locks.

If, like me, you’re curious about how STM is implemented, I suggest you check out a few papers. Understanding of all of these benefits from understanding the basic ideas and terminology of concurrency, as well as a bit of computer architecture knowledge.

There are certainly many, many more implementations, and many more papers, and more ways to tune an implementation than these couple papers spend much time on. Check out the related work in these papers.

I’m also pleased, because I feel Haskell has been underrated by the programming populus at large for a long time.** I’m interested in becoming more familiar with Haskell largely because of some of my pet peeves with OCaml, and because Haskell’s type system particulars are both richer and closer to category theory, which interests me mainly as an intellectual curiosity (and for its relation to type theory in general). Well written Haskell can also beat out many more well-known languages performance-wise (example, definitely poke around the language shootout). And it’s always nice to see a real functional language (read: one in which functional style is the norm, not just an option) getting mainstream attention.

I can only hope that people continue to take note of Haskell.

* Please note, those of you who are itching to start a flamewar about shared state vs. message passing, that STM does not claim to be the be-all and end-all of concurrent programming. It is designed to make programming with shared state easier than it is today. And shared state is not going away, because it’s the model with which chips are currently designed. So at the very least, operating systems, compilers, and some runtime systems will still need to deal with it, even if all other development switches to message passing concurrency.

** I should probably mention that I have not used Haskell a great deal, though this is not for lack of desire. A couple years ago I wrote a short assignment in Haskell (largely to play with lazy evaluation). I’ve also spent a fair amount of time writing OCaml over the past couple years, for the course I TAed and head TAed which uses OCaml to introduce students to currying and static typing (following an introduction to CS with Scheme), and last semester in a group software engineering course (perhaps I’ll elaborate on this experience another time). So I’m familiar with a similar type system.

So I Was On Reddit

8 08 2007

So, apparently, less than two weeks after starting this blog, I ended up on (direct link) for this entry on the fact that most monad tutorials are awful, and offering a pointer to a paper . Apparently if you tag an entry with ‘monads,’ avid Haskell users flock to your blog (I exaggerate of course – just many more people than otherwise read my blog). Really, I suspect that using that tag just got more people to notice, and most of the hits came from Reddit, where I was probably only posted because I had a bit of an opinion, plus a link to Wadler’s paper. Interesting phenomenon.

Apparently I got a peak of 1,015 page views on August 2nd, as well as getting included in the “Blog Noise” section of Haskell Weekly News. Having never really blogged about anything other than my personal life before this summer, it’s interesting to see how certain keywords draw spikes of attention. For that reason, I’m specifically not tagging this entry with ‘haskell’ or ‘monads’ because I don’t want to distract that many people. Just thought it was interesting.

EDIT: Hello again, reddit-ors.

Improving Solaris Resource Management

8 08 2007

Over at my Sun blog, I just recently posted the slides from the KTD (Kernel Technical Discussion) presentation I gave on Monday.

Alternative Languages on .NET and the JVM

3 08 2007

The other day I stumbled across this link on Reddit about the formation a coherent project (the JVM Language Runtime)for implementors of alternative (non-Java) languages on the JVM to get useful information and code for using the JVM. The article points out something I’d taken note of before, which is the difference in Microsoft and Sun’s attitudes towards third parties implementing languages on their virtual machines. Really, until relatively recently, Sun did not really care about such projects. They did nothing to actively stop them, but little if anything to assist them. They really only started helping out (and hiring) members of projects like JRuby after Microsoft took the lead! Microsoft, almost as soon as projects like IronPython appeared, started actively helping them and hiring their developers – what better way to draw programmers to your platform than let them use any language they want? On the other hand, Sun also spent many years trying to push Java Java Java, while Microsoft’s developer tools have always included a variety of languages, most notably Visual Basic and Visual C++ in the pre-.NET era. So perhaps Microsoft just planned support for things like this from the start, because they already needed/wanted support for multiple languages on the same runtime – really, C#, VB.NET, ASP.NET, and a number of other .NET upgrades to major Microsoft product lines ran and inter-operated on the CLR right from the get-go.

For some reason it took Sun some time to see the light on this, and it’s one of the things for which I admire Microsoft (don’t get me wrong – I have all sorts of gripes with other things they do and have done). Mainly that, and the fact that they really spend a lot of money on pure, corporate-hands-off research. Sun also does this, though not as much, with good reason. Sun is doing much better since the pain of the dot-com bubble burst, but still has a ways to go to return to it’s previous state.

Really, I’m just looking forward to the time that I can write a program in Python or Ruby, have it run normally on any *nix (BSDs, OS X, Solaris, Linux, etc.), and run correctly on the JVM or .NET (user’s choice) on Windows through IronPython, Jython (which last I heard was semi-dead), JRuby, and IronRuby. I suppose I’ll have to wait for one of the proper native Ruby VMs for Ruby to be adequately fast – I’m rooting for Rubinius.

See also Microsoft’s framework for webapps, which from what I’ve seen is basically a .NET-esque multi-language Flash competitor (currently Windows and OS X versions): Silverlight.

About That Header Picture

1 08 2007

I did change the default header image for this theme. The original was somewhat bizarre, and reminiscient of a blurry photo of paint sample cards. The image I replaced it with is a cropped version of a photo taken last summer at Land’s End Park in San Francisco. There was, at that time, a stone labyrinth on a flat area on the edge of a cliff, overlooking the mouth of the San Francisco Bay, the Golden Gate, the Presidio, the Marin headlands, and out into the Pacific.

Blog Move

1 08 2007

Well, as I said in my first post I was apt to do, I’ve decided to move my blog. Why? Blogger, for anything other than simply posting or selecting a pre-existing theme, is painful to use. That’s the main reason. In particular, the interface for rearranging page components is basically broken in Safari (my browser of choice).

I’m moving to, because WordPress

  • Has none of the UI wonkiness and breakage which Blogger seems to have
  • For the parts of Blogger which work, WordPress has a nicer UI
  • Supports a clean XML export, usable by me if I do a custom blog engine, or if I install WordPress on my own server
  • Has a broader, nicer variety of default themes. Yes, that’s part of the basis for my selection. Compare the Blogger and theme choices yourself – you’ll agree with me.

I’ll also be updating my domain,, to point there, just as soon as Yahoo! Domains fixes the problem with their domain administration page which is currently preventing me from changing any domain settings…