EDIT:

The campaign is over, and all goals were reached!

This post on outlines the current status, and will be updated as work progresses.

If you'd still like to pitch in, there's a Donate button on my homepage -- I will honor perks for donations made before Monday 5th of September 2011, should you request them.

Cheers, and Thank You Everyone,

 -- Nikodemus

Goal #7 reached, big compiler lock removal is go!

Goal #6 reached, Madeira is go!

Goal #5 reached, threading API refactoring is go!

Goal #4 reached, semaphore notification objects are go!

Goal #3 reached, read/write locks are go!

Goal #2 reached, timeouts across the threading API are go!

Goal #1 reached: locking improvements are go!

Hi!

I'm Nikodemus Siivola, an SBCL developer self-employed via Steel Bank Studio Ltd. I've been working on SBCL for about 8 years, and have around 1400 commits in the current tree.

My goal is to make SBCL the number one open source Common Lisp implementation.

Maybe 30% of my work on SBCL has been funded more-or-less-directly by my customers, and the rest I've essentially funded out of my own pocket. During the past couple of years -- since the mortgage crises, pretty much -- consulting gigs have been getting thinner on the ground, and I'm looking for a better way to fund future SBCL development.

This is my first experiment in crowdfunding: if it succeeds, there are other SBCL projects that I plan to fund in the same way -- hopefully involving other SBCL developers as well.

Everyone Loves or Hates Threads

SBCL has native threads on several POSIX platforms, but they aren't quite as robust as they should be outside Linux, and even on Linux the threading API is few pieces short of what I'd like to see there.

Individual platforms have issues of their own, but the major problem shared by most non-Linux platforms is our implementation of mutexes, and their misbehaviour when it comes to interrupt safety. 

The API is missing timeout arguments from several functions, read/write locks, and notification objects on semaphores... to start with.

Introducing Fair Spinlocks

I plan to improve things on non-Linux platforms by:

• Replacing SBCL's mutexes on non-Linux platforms with a ones based on fair spinlocks. My proof of concept implementation shows that this is viable: in my tests fair spinlocks incorporating a backoff consistently outperform POSIX mutexes on Darwin.
• Implementing a userspace solution for semaphores and condition variables for non-Linux platforms.

Futexes will remain in use on Linux: while it would be nice to use identical code on both Linux and elsewhere, futexes are simply too good for our purposes to not to use them.

The initial funding goal is enough to get this done. The results will be incorporated into SBCL and released into Public Domain as usual. (I have already developed the necessary extensions to ticket spinlock algorithm, allowing threads waiting for a spinlock to be interrupted without holding up the queue. What remains is integrating the work with SBCL, and dealing with semaphores and condition variables.)

The Missing Bits and Pieces

Even if you only use SBCL only on Linux, please consider participating. If I exceed the initial goal, I will also be able to work on:

• At $4,000: adding timeout arguments to all blocking functions in the threading API.
  
• At $5,000: implementing read/write locks.
• At $6,000: implementing semaphore notification objects.
• At $8,000: generally refactoring the threading API (while of course retaining backwards compatibility.)
• At $12,000: new library called MADEIRA will be implemented, which is to be a compatibility library for Common Lisp implementations focused on threading and bits not covered by either CFFI or USOCKET. (Basic things like access to command-line arguments, environment, EXIT, etc. Bordeaux-Threads -like functionality, but better defined, with sharper edges. CAS. Remains to be seen how general I can make the support, though. Running external programs -- probably somewhere between TRIVIAL-SHELL and SBCL's RUN-PROGRAM in functionality.)
• At $16,000: scope of the "big compiler lock" in SBCL will be curtailed, so that it will be possible to compile things in parallel. (What is the big compiler lock? SBCL grabs a lock when it compiles anything. The same lock is also used for some non-compiler activities like modifying the class graph. This lock prevents parallel compilation. Even if you don't care about parallel compilation as such, there are other unfortunate consequences: CLOS dispatch (before caches settle down) can involve compiling code. Most of the time BCL is a tiny annoyance, but sometimes it can be a real headache. Sometimes it is an insurmountable barrier. )
  

Spread the Word!

Tell people about this campaign!

Tell people about Steel Bank Studio Ltd.

Tell people about SBCL.

Tell people about Common Lisp.

Let Me Know What You Think!

Even should you choose not to chip in this time, don't hesitate to contact me at nikodemus@random-state.net: I would very much like to hear what you think of this, and what sort of SBCL projects you would be willing to participate in crowdfunding!

Thank you, and Happy Hacking!

 -- Nikodemus

PS. Due to high (and variable!) shipping costs I decided not to offer any material perks of note. If you're after t-shirts or mugs, check out my CafePress shop for some Lisp or SBCL swag -- 50% of the sale-prices there go towards supporting my work on SBCL.