Solaris, OpenSolaris and other Musings

Updates...

2006-10-23T23:32:00.000+08:00

Well, Matt commented that a zfs trashcan probably wasn't worth the effort, so I won't be persuing that anymore :(

I do however have a complete working prototype and diffs, so if anyone is interested at all, do drop me an email.

I started a discussion on implementing per file versioning in ZFS, and the long and lengthy thread was insightful to say the least. Most agreed that it was cool, but perhaps a specialized API for user apps to access versioning information would be the way to go, instead of a straightfoward userspace interface like VMS.

While/since all I talk about is ZFS nowadays, I've been reading through the code slowly and over and over again: richlowe thinks a writeup of how certain codepaths operate would be useful to the community at large. While maybe some of you grok C code, and some of you wrote the damn code, I personally find english documentation much easier to parse. It would be nice for those of you who are interested to drop me an email: I'll then do writeups in zfs-code, and it can be corrected by all before I commit it to a wiki (i'm looking at genunix.org baby! )

Later.

Insightful snippet seen on dtrace-discuss

2006-09-21T23:05:00.000+08:00

"...More than anything, what we (or at least I) learned
from DTrace is that if you want to get big wins, you don't make it
incrementally faster to do existing work -- you eliminate work entirely by
addressing its source higher in the stack of abstraction."

Bryan Cantrill, Solaris Kernel Development

Community NIC Driver diskette/CD

2006-09-15T16:09:00.000+08:00

Thanks to J. Keil for the below mentioned: check it out if you want to improve your install experience. :)

Community Boot Driver ITU Diskette/CDROM for Solaris Nevada x86

Interesting sound bite

2006-08-27T19:38:00.000+08:00

“To D-BUS people, every problem seems like a D-BUS problem.”
—Erik Troan

Back to the drawing board

2006-08-22T22:43:00.000+08:00

Matthew Ahrens pointed out several huge gaping holes in my current prototype for "zfs_trashcan". He also gave helpful advice on a better approach.

I've rewritten my design document, and will concentrate on my rewrite of said code this week. :(

On hindsight, I suddenly appreciate a design document. It fleshes out my ideas, and makes me reflect on what I really want to achieve. Following this line of reasoning, I can appreciate why there is a need for the ARC process for code integration into Solaris/OpenSolaris.

My first and currently only prototype came out of me hammering away until I had something working. So yes, I'm an amateur programmer, but I'm getting better.

Stay tuned. :)

sparc-on-intel translator

2006-08-18T19:08:00.000+08:00

Sparc-on-Intel translator due in weeks

Basically a JIT compiler translating SPARC instructions to Intel/AMD instructions.

The question is: how much will it cost you, and does your support contract cover it?

Presenting clicky...

2006-08-14T18:45:00.000+08:00

I decided to do some python and javascript dabbling just for a change of pace.

After 3 days of dabbling, i've implemented a simple web based RSS reader for my own use entitled "clicky".

Tools I used:

webpy : very Pythonic and easy to start with.

script.aculo.us (and prototype.js): used to provide Ajax magic like scrolling, hiding RSS entries as well as reloading feeds.

feedparser: Awesome universal feed parser written in python.

Total number of lines of code in the current implementation:
127 (python + markup)
49 (CSS stylesheet)

(I was a bad boy and mixed markup with code. But it feels oh so good!)

Screenshot attached below for your viewing pleasure.

Moving forward

2006-08-08T22:17:00.000+08:00

Matthew Ahrens has kindly consented to be my sponsor for integration of my 'zfs trashcan' functionality into OpenSolaris. I look forward to much code review and code rewrite on my part. :)

I thought it would be good to summarize why I'm even doing all of this, and in particular this patch.

1. ZFS is sexy.
Enough said about this point.

2. The vast majority of external contributors to OpenSolaris have submitted either trivial patches, or bugfixes (myself included).

This is not to belittle their contributions; however, for our community to grow in the right direction, we need external folks to make large non-trivial patches that add functionality.

Which is why I'm working to get my 'zfs trashcan' work integrated. I sincerely believe that OpenSolaris welcomes contributions from anyone willing to try and I intend to show that that is the case.

I'll blog consistently for the next few weeks as I work with Matt to get this patch into an acceptable state. Hopefully along the way, I'll inspire aspiring contributors to get involved and submit patches, trivial and non-trivial.

To all you young/weekend/covert developers out there: Just do it. Get started with a small patch, talk to ppl in #opensolaris or the mailing lists, and request-sponsor. Just get it going.

Close to completion

2006-08-05T00:46:00.000+08:00

My work on #4852821: user undo has reached another significant milestone.

The core functionality is there: deleted files appear in your trashcan in your_fs/.zfs/deleted, which exposes the same namespace. Files are deleted permanently from the trashcan in FIFO order, and the size of the trashcan is set via the 'trashquota' property.

There are several loose ends to tie up still, but I hope to initiate the whole request sponsor process tomorrow. Hopefully the zfs folks are still interested in #4852821. Even if they're not, it's been a good learning experience. :) I spent the last 2 hours tracking down a bug that got in the way of userland 'zfs' being able to set the size of your trashcan via the "trashquota' property.

DTrace was once again, instrumental in debugging. Hopefully I can scrape up enough cash to get Brendan Gregg's Solaris Performance and Tools: DTrace and MDB Techniques for Solaris 10 and Open Solaris to improve my MDB-foo. :)

An interesting comment on LKML..

2006-07-31T20:46:00.000+08:00

(I mean geez, if you want really high standards before new code is
accepted, take a look at Open Solaris; they have *such* a heavyweight
process, with two mandatory signoffs by core Solaris engineers who
both have to do a line-by-line review, and with a promise of on-disk
and ABI compatibility *forever* ---- that we do more commits in a week
than they do in a year....)
- Ted

ZFS Recycle Bin Phase 1

2006-06-25T19:15:00.000+08:00

I've been working on #4852821: user undo for the past few weeks, and today I've finally reached a milestone of sorts.

nerant@archon [/export/home/nerant]cd /undo4/fs1
nerant@archon [/undo4/fs1]ls -ahl
total 9
drwxr-xr-x 3 nerant sys 3 Jun 25 19:12 .
drwxr-xr-x 3 root sys 3 Jun 25 19:03 ..
drwxr-xr-x 2 nerant other 3 Jun 25 19:13 messages
nerant@archon [/undo4/fs1]cd messages/
nerant@archon [/undo4/fs1/messages]ls -ahl
total 8
drwxr-xr-x 2 nerant other 3 Jun 25 19:13 .
drwxr-xr-x 3 nerant sys 3 Jun 25 19:12 ..
-rw-r--r-- 1 nerant other 17 Jun 25 19:12 msg0
nerant@archon [/undo4/fs1/messages]cat ./msg0
SUPERMAN RETURNS
nerant@archon [/undo4/fs1/messages]rm ./msg0
nerant@archon [/undo4/fs1/messages]ls -ahl
total 6
drwxr-xr-x 2 nerant other 2 Jun 25 19:19 .
drwxr-xr-x 3 nerant sys 3 Jun 25 19:12 ..
nerant@archon [/undo4/fs1/messages]mv /undo4/fs1/.zfs/deleted/messages/msg0 .
nerant@archon [/undo4/fs1/messages]cat ./msg0
SUPERMAN RETURNS

Through some magic vnode and znode hackery, deleted files now appear in .zfs/deleted, with the entire namespace reproduced. :)

I'm looking forward to implementing the quota part of the RFE during the next week or so.

Feel free to email me with your comments! Feedback is appreciated.

OpenSolaris turns 1

2006-06-14T19:41:00.000+08:00

1 year ago, I was nearing the end of my internship at Sun. During that internship, I was able to observe the internal efforts to bring about the open-sourcing of Solaris, aka OpenSolaris.

I was elated when OpenSolaris went live. I still have the initial source code tarballs and build tools stashed safely away on my iPod shuffle. Here was my opportunity to continue working together with smashingly good software engineers at Sun when my internship ended.

I had a short and unproductive stint as a blastwave maintainer, but that was probably due to my own fault.

On the plus side, I've managed to contribute a few small putbacks, with the most recent (about to be) putback being a rewrite of hostname(1) to fix bug 6413595.

Looking back on the past year, we've made great strides as a community.

ZFS was made public before it was even shipped as part of S10U2, and the ZFS developers are ever so ready to entertain emails on the zfs-discuss mailing lists.
The packaging tools have been opensourced. (though I haven't observed much work on this)
JDS as a consolidation has been opensourced
Xorg code drops of what Sun ships with S10 have been made available

Those are just some of the key events which I can recall.

I've had the pleasure of working with Sun engineers, as well as witty and intelligent external contributors, such as richlowe (IRC) . The #opensolaris IRC channel is a delightful mix of technical discussion, help and sarcastic humour.

And what I've found incredibly impressive about all the interactions I've had so far is how polite and civil everyone is. I don't see Sun Engineers flaming non-Sun community members during arguements, or vice versa. Instead, conversations always focus on the technical merits of demerits of a particular approach of providing functionality in Solaris (the ON consolidation more specifically).

I come away impressed: this is a community of very smart folks with many accomplishments under their belt willing to share their opinions in a helpful manner.

Of course, all is not perfect. Namely,

As a community, almost all development efforts and projects are being driven and led by Sun folks.

We need to allow non-Sun developers build ownership of code and projects in OpenSolaris.

It is non-trivial to build vanilla (non-JDS) Gnome on Solaris/OpenSolaris. This means that developers find it difficult to choose OpenSolaris as their development platform of choice. The same can be said for many pieces of software.

Non-bundled OSS can be obtained from multiple sources, such as blastwave, sunfreeware and the companion cd
Essentially, we as a community are wasting time patching/porting software to work on OpenSolaris over and over again instead of pooling our efforts. I've seen plenty of talk and discussion about this, but it all boils down to this: We better fscking do something and work together on this NOW.
Ultimately the more OSS that builds out of the box on OpenSolaris, the more users and developers we can attract to our community.

Hardware support sucks.

This makes it difficult to attract developers.
This is probably because most developers who are hardware geeks already work on Linux and the *BSDs
We need more folks, like you to step up and hack on drivers!

For me, I've been hacking away on a ZFS feature that I hope Eric Schrock will be willing to help me putback. It has been great fun so far, with kernel panics and much ZFS code reading, though my day job is making it difficult to complete it. I hope to have some code to show you folks later this week.

A grateful thanks to all those who have been patient with me this past year. In particular, jmcp, dclarke, dan mick, eric saxe and dan price. Thanks! :)

Happy Birthday OpenSolaris!

Liferea and Poppler updates

2005-08-02T22:50:00.000+08:00

I've put up updated Liferea 0.9.5 packages for testing.

Also, I managed to persuade poppler to build using Sun Studio, so SPARC folks may want to try the updated poppler packages in testing to see whether it is zippier for them.

As usual, feedback is appreciated, especially from SPARC users, as I don't have a SPARC desktop to test on.

Liferea

Poppler

Liferea 0.9.4

2005-07-26T22:40:00.000+08:00

I've put up updated Liferea 0.9.4 packages for testing.

I will most likely put Liferea 1.0 up for inclusion into the blastwave repository ("unstable"), unless there is overwhelming demand for putting 0.9.4 in right now.

As usual, feedback is appreciated, especially from SPARC users, as I don't have a SPARC desktop to test on.

Updated Liferea 0.9.3 packages

2005-07-09T17:38:00.000+08:00

I've put up updated Liferea 0.9.3 packages for testing.

As usual, feedback is appreciated, especially from SPARC users, as I don't have a SPARC desktop to test on.

Evince/Poppler Packages up for testing

2005-07-03T15:39:00.000+08:00

Evince and Poppler packages are up for testing.

What is Evince? Shamelessly ripped from their homepage:

"Evince is a document viewer for multiple document formats like pdf, postscript, and many others. The goal of evince is to replace the multiple document viewers that exist on the GNOME Desktop, like ggv, gpdf, and xpdf with a single simple application."

And Poppler:

"Poppler is a PDF rendering library based on the xpdf-3.0 code base."

Packages are available as linked below (You need poppler to run evince)

Evince

Poppler

As usual, you may need additional packages from blastwave. If you're running Gnome 2.8 from blastwave, you should be set. You will also need CSWgcc3corert and CSWgcc3g++rt (which you most likely already have. Again, pkg-get is your friend)

As usual feedback is appreciated, particularly from Sparc folks, since I don't have a Sparc desktop.

Liferea packages up for testing

2005-07-01T16:23:00.000+08:00

One of the first packages that I'm packaging for blastwave is Liferea.

What is Liferea? Shamelessly ripped from their homepage:

"Liferea is a news aggregator for online news feeds. There are many other news readers available, but these others are not available for Linux or require many extra libraries to be installed. Liferea tries to fill this gap by creating a fast, easy to use, easy to install news aggregator for GTK/GNOME."

Packages (.pkg) are currently up for testing.

You will need the following packages from blastwave (use pkg-get -i )

CSWgconf
CSWggettext
CSWglib2
CSWgtk2
CSWiconv
CSWlibatk
CSWlibgtkhtml
CSWlibpopt
CSWlibxml2
CSWorbit2
CSWpango
CSWzlib

[There is a good chance you already have all these packages if you are using gnome from blastwave]

Please do try out the packages: particularly the sparc packages, as I don't have a sparc machine to test it on. Your feedback is appreciated, as well as bug reports. After sufficient testing (the definition of which is vague), these packages will make it into the blastwave tree, so you can use pkg-get to install them.

Feedback, flames, comments appreciated.