Mercurial + MLKit + OS X = Failure, plus workaround

21 03 2008

My thesis, as I’ve mentioned several times before, has me hacking up the source to the ML Kit compiler. Because the current build arrangement I’ve been using requires a great deal of RAM, I’ve been using my desktop machine and CS department machines exclusively for building. However, I’m going home for part of spring break this coming week, there’s no computer with appropriate specs at home, and editing code over SSH is rather painful because of the high latency involved. The obvious solution, since I’m using Mercurial to manage my thesis, was to do a checkout on my laptop (a G4 Powerbook lacking enough RAM to build, and lacking the correct processor architecture to test), edit locally, and just push intermediate versions back to the CS systems to build there. This would result in some useless crap in the commit logs, but it beats the alternatives. I had an existing checkout from back in January, when I was using OCaml, so I went to update and saw the following:

Yggdrasil:thesis colin$ hg up
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
Yggdrasil:thesis colin$ hg pull
remote: ####################
pulling from ssh://cs/thesis/
searching for changes
adding changesets
adding manifests
adding file changes
added 44 changesets with 18762 changes to 17825 files (run 'hg update' to get a working copy)
remote: Forwarding you to cslab0a
remote: ####################
Yggdrasil:thesis colin$ hg up
abort: case-folding collision between mlkit/kit/basis/MLB/RI_PROF/Splaymap.sml.d and mlkit/kit/basis/MLB/RI_PROF/SPLAYMAP.sml.d
Yggdrasil:thesis colin$

I saw this, noticed it was a temp file (ML Kit makes an MLB directory for temp files) and hg rm’d all such files in the CS copy, committed, and tried again. Same result. Poked around online, and after a bit, found a message from someone with a similar problem, with a similar case difference between two distinct files, and a response pointing out the problem with Mac OS X’s case-insensitive filesystem.

When I installed the latest OS X, I intentionally opted for case-insensitive because many Mac programs have a habit of breaking on case-sensitive filesystems. And now I can’t have a copy of the ML Kit source on my machine. It’s common practice in SML programs to use all-caps names for files containing signatures (akin to interfaces for the non-ML folk) and using “upper” camel case for implementations. So I can’t just rename a couple files, it seems you just can’t have the ML Kit source on most OS X installations’ local disks.

So was it time to give up on this? No, of course not – I need to work on my damn thesis while I’m home. Screw Apple and it’s crappy filesystem design choice.

Fortunately Apple did make one excellent, excellent design choice with OS X, which was making disk images a standard, common object. So I used the Disk Utility tool to make a disk image containing a case-sensitive filesystem, mounted it, and did a checkout inside that. It’s pretty stupid that I needed to do this, but at least I can work on my thesis over break without running Vim over SSH.

Now if you’ll excuse me, my friend Owen is heading off to Apple’s filesystem team next year, and I need to go give him grief for his group’s terrible design choice.

Solaris and OS X Playing Together

4 11 2007

After winning my iPhone in a raffle, I was told by those who raffled it off that it was purchased before the price drop, and as a result, I could get the $100 credit.  What better to spend it on than upgrading to Leopard?  After all, a system with DTrace, DTrace-enabled Ruby, and a friendly GUI to boot is too good to pass up when you only have to pay $40 after taxes and credit. So I spent about 6 hours backing up my data to my Solaris box.  Why so long?  Well, I could not make Tiger and Solaris 10 play nicely with file sharing, so I had to resort to using scp to move 32GB of data.  Not pleasant.  I’d tried getting Tiger to mount NFS shares, but with little (read: no) success. So naturally one of the first things I did with my machine after installing Leopard, Firefox, and Adium was start poking around in the settings.  I quickly found a “Show Advanced” button in the Directory Utility.  And what appeared?  A tab for network mounts!  All I did was add a quick entry of the form you’d expect, start NFS on my Solaris machine, and voila!  Immediate access to my backups over NFS.  This is great – it means I can easily restore my iTunes collection to the fresh install of Leopard.  The ease with which this was done makes me suspect I missed something glaringly obvious in Tiger. Of course, a few things remain to be dealt with:

  • Supporting read/write access – just unifying the UID of my account on both machines.
  • Denying read access to other clients – probably just changing the permissions in my home directory.
  • Getting my Mac to dynamically resolve the hostname of my S10 box – we use DHCP on my home network, and I don’t want to change this.

Now, here’s to hoping this will get me to do more work on my ruby-on-rails project while I’m flying between coasts the next few weeks! 

A successful NFS configuration on OS X Leopard

Exciting Upcoming OS Improvements

19 10 2007

I must apologize for the lack of posts – the first few weeks of the semester, plus the job hunt, are extremely time-consuming. I have half-finished drafts of 3 articles, but no time to do the solid revising and research they need. I try to make my technical entries very precise, accurate, and backed by links to reputable sources.

But this is just a brief entry, because I’m excited for upcoming updates to three wonderful OSes:

Mac OS X Leopard
This has been anticipated for some time. It ships on the 26th of October if you weren’t already aware, and has some wonderful features. TimeMachine initially excited me the most. I heard about it while watching a periodically updated blog post of WWDC 2006 coverage with coworkers at NetApp, and to us it immediately suggested that Apple finally wised up a bit and implemented snapshots in one of their filesystems. The fact that TimeMachines requires an external hard drive makes it clear that this isn’t quite the case, which is a bit surprising given that it has been acknowledged that Leopard has at least some support for ZFS. Supposedly Leopard will only support reading from ZFS – alas, my dreams of a dual-boot Solaris/OS X Macbook with a shared ZFS pool will have to wait for another day.

More exciting to me is the addition of DTrace to Mac OS X in the form of Instruments, a snazzy GUI on top of DTrace. This is going to be a killer developer application. DTrace is very powerful, and fairly flexible, but has a bit of a learning curve to do more advanced things. I’m very optimistic about how discoverable an Apple GUI can make this.

And of course, after many years of using multiple desktops on Linux, they’re finally in OS X. For those who can’t wait, or don’t want to upgrade just for multiple desktops, Desktop Manager and Virtue Desktops work reasonably well, though for obvious reasons they’re no longer under development.

That said, my Powerbook is finally going, so I’m probably just going to buy a new Macbook next time the hardware is updated, and get Leopard that way instead of shelling out money for an upgrade. If it weren’t time for a hardware upgrade for me, I think I’d probably still do it just to have DTrace on my Mac.

OpenSolaris Project Indiana
What is Project Indiana? Many things, but primarily two things: an effort to create an all-open-source version of OpenSolaris (which currently includes some binary blobs to run well), and a place to prototype things like the new installer, stable ZFS root and boot, and the new package system. It was uncertain when a prototype of all these things would arrive, but it seems that a developer release will be available in the next couple weeks. This is enough to make me hold off on finishing customization on the new workstation I just got; I’m going to wait, and install this development version from scratch. I’m sure I’ll run into plenty of bugs, but that’s fine – it’s exciting! Also, an additional benefit of doing a reinstall is that I can make an extra slice for doing live upgrades of my system, which the preinstalled configuration doesn’t support. I can’t wait.

[Update: Found a very thorough description of Project Indiana.]

KGDB in Linux
Despite being postponed, it looks like a proper kernel debugger is headed into the mainline Linux kernel. Linux has actually had kernel debuggers for some time, but they were external patches. Being in the mainline kernel will mean better stability and will likely increase use among kernel developers.

This doesn’t directly impact end users, because most users don’t debug kernels. It does however affect them indirectly, because it will help kernel developers find (and fix) bugs faster. For kernel developers, a proper kernel debugger is a blessing. I used kmdb extensively this summer working in the Solaris Kernel Group. I can’t imagine how frustrated I would have been without it. Being able to step kernel code makes it almost as easy to debug as userland code (with some exceptions, obviously). Mac OS X has also had a well integrated kernel debugger (two in fact) for some time as well.

I’m really glad Linux is finally going to integrate this – the Apple documentation on kernel debuggers is spartan, and Solaris is still (unfortunately) not as easy to get up and running as Linux, and anyone who wants to hack on a kernel benefits greatly from having a solid kernel debugger. Hopefully this will encourage more people to jump the gap to kernel work, since this makes it more approachable.