[Music credit: “Give Freely” by The X-Structure]
What is Sandstorm.io?
Update: We did it! We passed our funding goal and made it to 118%! Thanks, everyone!
For the latest updates, be sure to sign up for the Sandstorm mailing list (e.g., be the first to find out when we're taking more pre-orders) and follow us on Twitter @SandstormIO. Thanks again, all of you, for your support!
Sandstorm is an open source project aiming to fundamentally change the way we use web apps, led by ex-Google-engineer Kenton Varda (best-known for Protocol Buffers -- used by hundreds of tech companies -- and Cap'n Proto, its successor).
Sandstorm makes running your own personal server as easy as running your phone. You can install apps like e-mail, document editors, blogging software, and more, using a simple app-store-like interface. No config files, no command lines; everything is done through your web browser.
In other words, Sandstorm gives you the experience you are used to getting from services like GMail or Dropbox, but using the software you choose, on servers you control. This means:
All your data is stored in one place, rather than fragmented across the internet.
Apps do not disappear if the developer stops supporting them.
The developer of an app cannot spy on you, perform experiments on you, or mine your data for advertising purposes.
Sandstorm isn’t just easy to use, it’s also innovative. We are building common features and powerful tools directly into the platform for apps to leverage:
Each app runs in a secure sandbox, so a bad app cannot harm the rest of your server.
Sandstorm provides a unified login system, so you don’t have to log into each app separately.
Apps can easily integrate with our unified sharing model rather than rolling their own. In fact, Sandstorm's sandboxing model makes it possible to securely share any app instance, even if the app itself does not implement sharing at all.
Apps are given infrastructure for federation, so that they can -- securely, with your permission, but also easily, via the web interface -- discover and talk to each other and to other servers.
What we have so far
Sandstorm is already functional today. It is open source and available on Github. You can see it in action in the video above, or by trying the demo. You can install it for yourself, or request an invite to our shared server at sandstorm.io.
For security reasons, we started out heavily restricting how apps can interact with the outside world, and are gradually enabling features. Currently, an app can:
Display a web interface to its owner, and anyone with whom the owner shares it.
Publish public, static content to a separate domain (e.g. a public blog).
Send and receive e-mail (with restrictions to prevent spam and forgery).
Make limited outgoing HTTP requests (e.g. to fetch RSS feeds or interact with a remote API).
Integrate with the unified login and sharing systems.
Be built on any Linux-based technology stack (Node, Rails, PHP, etc.).
We have ported several open source apps to Sandstorm already, including:
WordPress (publishing/blogging platform)
Etherpad (collaborative document editor)
Ethercalc (collaborative spreadsheet editor)
ShareLaTeX (collaborative LaTeX document editor)
Apache Wave (formerly Google Wave)
Groove Basin (music player)
MediaGoblin (photo/media sharing)
Ghost (blog publisher)
IPython Notebook (interactive computation environment)
Tiny Tiny RSS (RSS feed reader)
After a year and a half of work, Sandstorm is almost ready for production use. We only need a little bit of funding to get us the rest of the way. But there’s a lot more we’d like to do, beyond the basic feature set. The more funding we get, the farther we’ll be able to take the project.
If we reach our funding goal, we will:
- Polish off Sandstorm and make it easily accessible to everyone, either on your own machine (free, open source) or on our subscription-based shared host.
- Hire a designer to improve the UI/UX.
- Add the ability for apps to host simple REST APIs, enabling basic federation.
- Make sure Sandstorm supports a compelling set of apps at launch (see above!).
- Make sure the tools are available so that any developer can easily make their apps run on Sandstorm.
We would like to take Sandstorm much further than this. Depending on how much support we get (not just during the campaign, but ongoing), we hope to develop a lot more. Here's a list of some of the ideas we plan to pursue beyond the minimum goals:
Indie App Market: We will develop and run an app marketplace, allowing other indie and open source developers to sell their apps on a “pay what you want” model. This will encourage development of more apps, making Sandstorm itself more useful.
Powerbox: We will develop rich sharing and federation through the use of Cap’n Proto and capability-based security. Users will be able to connect apps to each other through a Powerbox interface, allowing apps to establish secure links without worrying about any of the details of authentication or access control, even when they are not owned by the same user. (For more about the powerbox, see the section on security, below.)
GPG login: We will implement GPG-based cryptographic login. Forget passwords! Create an account based on your public key.
Per-document encryption: We will implement encrypted storage, with each document having a separate encryption key that can only be accessed by the people with whom it is shared. If all those people use GPG login, we’ll make sure that it’s impossible for us to decrypt the document except when a legitimate user is online and accessing the doc.
End-to-end encryption: We’ll build a browser extension (or build on an existing extension) that allows Sandstorm apps to use end-to-end encryption without encryption keys ever reaching the server.
Enabling the Federated Revolution
The web is too centralized. Most -- possibly all -- of the web apps you use today are developed by giant corporations and run entirely in their datacenters. Open source and indie web apps exist, but are rarely used, because they usually require you to run your own server, which few people want to do. Federated networks -- where every user controls their own personal data on their own server -- are currently essentially impossible.
The solution is to make it easy for everyone to have a server that they control and where they can run any app they want. Sandstorm is here to do that.
This doesn’t necessarily mean that everyone runs Sandstorm on their own physical machine. You can do that if you want, but many people would prefer the convenience of paying a subscription fee to use servers running in a datacenter. Sandstorm supports both: one Sandstorm host may support just a single user while another may have a million.
Even if you choose to subscribe to a shared Sandstorm host, you still get to be a part of the revolution: only you -- not your host -- get to decide what apps you run, and if you decide you don’t like your host, you can easily move to a new one and take all your apps with you. It doesn’t even matter whether you and your friend use the same Sandstorm host; it’s just as easy to connect your apps to theirs either way.
Getting Security Right
On a traditional server, every application you install is a new potential gateway for hackers. If any one app has a security bug, chances are a hacker can use it to take full control over your server -- including the secure apps! In such a model, you have to be very careful only to install apps from developers that you can trust to get security right -- and there are sadly few of those.
Sandstorm solves this problem. Under Sandstorm, every application runs in its own secure sandbox, meaning that it is securely isolated from the rest of your system. Even if the app is actively trying to compromise your security, it can’t.
But security is often a double-edged sword. What if you actually want two applications to talk to each other? For instance, what if you want to embed a photo in a document, and so you need the document app to get access to the photo?
To solve this, Sandstorm uses a technique called capability-based security and a user interface called the powerbox. The powerbox is simply an enhanced “picker” interface provided by Sandstorm itself with built-in security. If your document app wants access to a photo, it simply tells Sandstorm “I need a photo.” Sandstorm itself then displays to you an interface where you can pick a photo from among all your apps that contain photos. Once you select one, the requesting app is given access to that photo -- and only that photo.
Compare this to other platforms where an app that wants a photo must first request access to all of your photos, just so it can show you a picker. By making the picker part of the system itself, Sandstorm keeps you more secure.
The powerbox doesn’t just support picking photos, but picking arbitrary “objects”. Any app can define a new kind of object to make available in the powerbox. When you make a selection in the powerbox, you are not just moving data from one app to another, but setting up a two-way revocable connection over which those apps can communicate going forward. This allows apps to be hooked together and “composed” into larger workflows.
Taking it one step further, not only can you hook up your apps to each other, but you can hook them up to your friends’ apps that they have shared with you. In fact, this is exactly how Sandstorm aims to implement federation.
How does Sandstorm work?
Sandstorm’s server-side sandboxing is based on Linux kernel features including namespaces, control groups, and seccomp-bpf -- the same features used by the likes of Docker and Chrome. This allows us to run native Linux binaries without heavyweight virtualization, which allows sandboxes to be fine-grained while also allowing a wealth of existing applications to be ported to the platform relatively easily.
In fact, Sandstorm encourages apps to be designed such that every “document” lives in its own sandbox (where the meaning of “document” depends on the app). In order to make this scalable, Sandstorm aggressively shuts down servers that aren’t in use. No matter how many “documents” you have, you probably never have more than a couple open at a time.
Because each document is in a separate sandbox, Sandstorm itself can implement access control. This means that Sandstorm can offer a unified sharing experience across all apps, rather than have every app implement its own. Every Sandstorm app can be shared, meaning you never have to resort to sharing your password with someone. And you can be certain that no matter how buggy an app is, no one can get access to your data in that app unless you share it with them, because Sandstorm itself enforces security.
Under the hood, Sandstorm applications talk to the platform and to each other using Cap’n Proto RPC. Cap’n Proto is an open source serialization and capability-based RPC protocol created by the Sandstorm developers and already used in production by companies like CloudFlare and Canonical. Cap’n Proto RPC is based on CapTP and other work by Mark S. Miller and colleagues. Their research has shown that capability-based protocols make it easier for applications to have rich interactions with each other while still being secure -- so you never have to choose between doing the “secure” thing and doing the thing that gets the job done -- but popular operating systems have yet to take advantage of this research. We aim to take this research mainstream.
The “Powerbox” UI discussed earlier actually allows apps to connect to each other's Cap'n Proto RPC interfaces. All security is handled by the system, so the apps on either end never need to deal with complicated OAuth handshakes but the user remains secure.
Sandstorm apps are distributed as “spk” files. An spk is simply an archive that contains the application’s entire userspace -- that is, all of the binaries, libraries, and data files needed for the app to run. We avoid “dependency hell” by having apps simply package their own dependencies. spk files are signed by the author so that updates can be cryptographically verified. Anyone can build an spk file, and you can upload any spk file you want to your Sandstorm server, no matter who is the host.
When an application runs, the contents of its spk file are mapped read-only as the root filesystem, with the “/var” directory being the place where the app may store its data. This allows apps to be updated easily -- we just swap in a new spk file while keeping “/var” the same.
Kenton Varda - Team lead
Kenton is a hard-core systems engineer with a background in distributed infrastructure and usable security. In 7.5 years at Google he worked on search infrastructure, security, and sharing models, but was best known as “The Protobuf Guy” for his work on Protocol Buffers, the message serialization infrastructure used by nearly all of Google’s systems and which Kenton himself turned into an open source project. Today, Protocol Buffers is used not just by Google but by hundreds of companies, including the likes of Twitter, Blizzard, and Valve.
After leaving Google, Kenton began work on Cap’n Proto, the successor to Protocol Buffers that is “infinity times faster”. Cap’n Proto is a key component of Sandstorm, providing the platform API and secure inter-app communications. Today, Cap’n Proto is already used in production by companies like CloudFlare, Canonical, and others.
Jade Wang - Developer engagement
Jade runs the developer engagement program at Meteor, where she built a community with active meetups in over 60 cities worldwide in just over a year and a half. Sandstorm is only as useful as the apps available on it, and Jade is here to make sure we attract plenty of developers to write those apps.
Although her primary role isn’t as a coder, Jade has been known to code on the side, and she applies an engineer’s mindset to everything she does. She also hails from a neuroscience background, and founded Chez JJ, the hacker house network, a network of live/work communities for tech entrepreneurs, scientists, and other itinerant geeks.
Jason Paryani - Core developer
Jason is just your average hacker who maintains a personal OpenIndiana ZFS file server, uses a VPN service to get around his ISP throttling Netflix, and has the most tricked-out zsh configuration you’ll ever see -- that is to say, not average at all. When Kenton released Cap’n Proto, Jason was the first person to make it work in another language -- Python -- mostly just because he thought it would be a fun project. Now he’s using that code in Sandstorm itself.
David Renshaw - Core developer
David wrote the Rust implementation of Cap'n Proto and is leading development on the Java implementation. He has a background in math, programming languages, and formal verification, and has previously worked as a test engineer finding vulnerabilities in robotics software.
These guys are not working full-time on Sandstorm but have contributed a great deal of advice and a few patches. Please note that Sandstorm is NOT affiliated with their employers -- they are advising in their spare time.
Andrew Lutomirski - Advisor, Linux kernel
Andy personally implemented or helped to implement some of the advanced Linux kernel features that Sandstorm uses for containerization, and has contributed patches and advice to Sandstorm’s sandboxing strategy.
Brian Swetland - Advisor, platforms
Brian spent nine years leading Android systems engineering, and defined much of the low-level architecture of the platform, starting from initial development in 2004, prior to the Google acquisition of Android. Brian is helping us understand the real-world problems that a platform like Sandstorm will face as it grows.
Mark Seaborn - Advisor, sandboxing
Mark works on Chrome’s Native Client, and as such is an expert at native-code sandboxing, particularly on Linux.
Jasvir Nagra - Advisor, web security
Mark Miller - Advisor, distributed security
Risks and Challenges
Attracting developers is hard. One risk is that developers will be unwilling to target Sandstorm, and therefore the only apps available will be those ported and maintained by the Sandstorm team. Sandstorm could thus be reduced to a fixed-function hosting platform. We think this is unlikely, as many open source apps today struggle to gain traction due to the difficulty of running servers -- these apps can all benefit from targeting Sandstorm. But even if we can’t attract developers, we think the set of apps we’ve ported already form a pretty compelling bundle on their own, and more will come.
A second risk is that Sandstorm will take longer to finish than anticipated. This is a classic risk that applies to all software. However, we already have the base system working today; most of what’s left is simply gruntwork of polishing and productionizing it. The big unimplemented pieces -- the app store, and the powerbox -- are not essential for the platform to be useful. There is nothing new or experimental about the app store; we just have to build it. The powerbox is more experimental, but we already have the infrastructure in place in Cap’n Proto; most of the remaining work is simply UI. We feel confident that we can complete these pieces, and Kenton already has a proven record of leading open source projects and shipping software.
Why the name “Sandstorm”?
Sandstorm provides sandboxing for cloud apps. Sandbox + cloud = Sandstorm.
Sandstorm enables fine-grained application instances to work together in a federated way. In fact, each application instance you create is called a “grain”. We contrast this metaphor of lots of grains moving and interacting against the “walled gardens” offered by the big established providers. Extending the analogy to certain classic works of science fiction is left as an exercise for the reader.
With all that said, we consider the name provisional. It may change. We thought it better to spend our time coding than thinking of names. :)
If Sandstorm is all about running your own server, why are you offering to run our servers?
Clearly, you are a person who wants to run Sandstorm on your own machine. That’s great! But although Sandstorm makes running a server easier, running a physical machine still takes work that not everyone wants to do. If we want federated apps to succeed, it’s important that everyone be able to run them, even people who don’t want to maintain a physical machine.
The Sandstorm subscription service is the answer for these people. It gives you access to Sandstorm with zero setup time, living in a datacenter on a fast internet connection, with automatic backups, and is cheaper to boot (because you’re only paying for a slice of a machine rather than a whole machine).
Long-term, we’d like there to be many options for shared Sandstorm hosts, so that if you don’t like your host, you can just switch. We’d also like to see hosts located in many countries, so you can choose one with data protection laws suitable to your needs.
When is the launch, beta, and alpha?
We expect to launch our shared hosting service in early 2015. If you choose a subscription as a perk, it will start at the launch.
Our closed beta will run shortly before the launch in order to test the shared hosting infrastructure. If you choose a one-year subscription or more, you will be invited to our closed beta, with larger backers being invited first. Beta use will not count against your eventual subscription time, but your beta instance will have the resource quotas you chose as your perk and your beta data will carry over to the live service.
Our closed alpha is already running today! The alpha server has very limited capacity, so we can only offer an invite to backers who purchase a mega instance or larger.
If you plan to run Sandstorm on your own hardware, you can do so today! Check out the instructions on Github.
What is GBrh?
Gigabyte RAM hours. This is how Sandstorm shared hosts parcel out compute resources. A shared host only charges you for time that an app is actually running, which usually means only the time during which you have the app open in your browser. If an app uses 1GB of RAM (on the server) for one hour, then it expends 1 GBrh. If it uses 1GB for two hours, or 2GB for one hour, then it expends 2GBrh, and so on.
A typical Sandstorm subscription comes with 200 GBrh per month. Meanwhile, a typical app server in our experience uses around 100MB of RAM. At such a rate, you would need to keep three apps open for the entire month to go over your limit. Only the most demanding users are likely to hit this limit. The Sandstorm UI will feature a battery indicator to warn you when you are running low.
Sandstorm does not meter CPU time but may dynamically adjust scheduling priorities to keep things fair. Users whose monthly GBrh limit is higher will also tend to get higher CPU priority, but users who hog excessive CPU time will get lower priority.
Of course, if you choose to run your own Sandstorm instance on your own machine, none of this applies.
Why do I have to log in through Google or Github?
In short, because we actually think this is the most secure option we can provide right now, though we want to do better eventually.
Passwords have a lot of problems. People choose bad passwords. People -- even smart people -- are often fooled by well-crafted phishing attacks. And, of course, people regularly forget their passwords. In order to deal with these threats, we believe that any password-based login system for Sandstorm must, at the very least, support two-factor authentication and be backed by a human security team who can respond to hijackings. There must also be an automated password reset mechanism which must be well-designed and monitored to avoid attacks. Unfortunately, we don't have these things yet. Moreover, we don't believe that building a secure password login system is the best use of our time at this early stage.
Another problem with password login is that it makes federation more complicated. When you federate with your friend's server, how does it authenticate you? Not by password, obviously. Perhaps by OpenID or OAuth, but that is again a thing we would need to implement.
For now, by relying on Google and Github for login, we get top-notch security and straightforward federated authentication with very little work. This lets us stay focused on our core product. (We could add Twitter, Facebook, etc. login as well, but we are worried about people forgetting which one they used and ending up with multiple accounts.)
But we don't want things to stay this way forever. Eventually we'd like to support decentralized login mechanisms, like OpenID and cryptographic (PGP/GPG) login. These are stretch goals for us (see the "Goals" section, above), but of course we'll be happy to look at pull requests. :)
Under what license is Sandstorm distributed?
The Sandstorm core implementation is under the Apache License v2.0.
Sandstorm applications can be distributed under any license. We impose no restrictions.
How is this different from [insert alternative here]?
There are a handful of projects that people frequently compare to Sandstorm. We’ve found that they generally fall into a few broad categories:
Some are cloud infrastructure management tools that are really intended for developers or sysadmins, not end users. Modern Linux kernel features make containerization easy. The hard part is making it usable by end users. Sandstorm is all about user interface.
Some are projects aimed at letting you use a physical device in your own home as your personal cloud server. Sandstorm is unopinionated about where it runs. You can run it in your home, but we think many users actually do want their server running in a datacenter, as long as they still have control. We would love to see these hardware projects running Sandstorm. :)
Some are personal “app” platforms that only support apps written in a specific language, against their specific API. Sandstorm supports apps built on any server-side tech stack that works on Linux. We think this is critical for bootstrapping -- early on, the platform will need to be populated by existing open source apps that weren’t explicitly written for it.
Can I run Sandstorm on a Raspberry-Pi-like device?
For now, we are focusing on the x86-64 architecture. This means you can’t run it on a Raspberry Pi (which is ARM-based), but you could run it on a MinnowBoard Max. While we have nothing against the ARM architecture, we feel that because Sandstorm applications use native-code binaries, it is important to avoid fragmentation and developer burden by standardizing on a single target platform for now. We could revisit this decision in the future.
What are the “enterprise tools”?
The Sandstorm Enterprise Tools will make it easy to treat a cluster of machines (whether on-prem or on a cloud hosting provider) as if it were a single Sandstorm instance, with any number of users. This is useful for businesses who wish to host internal business apps in a secure yet low-maintenance way, or for companies wishing to resell Sandstorm hosting at a large scale. Because the enterprise tools are primarily useful only to enterprises, we plan to fund their development through licensing fees. Our Indiegogo campaign is to fund development of the core (open source) Sandstorm platform, not the enterprise tools, but since the core platform is a prerequisite for the enterprise tools, we are offering credit towards the enterprise tools as a perk.
What cut will you take from app market purchases?
For verified open source apps using the pay-what-you-want model, our base fee will be equal to the transaction costs charged by our payment processor, but the buyer will have the option to contribute more to Sandstorm.
For fixed-price proprietary apps, we have not yet decided, but it will be no more than the 30% fee which is typical of other app stores (e.g. Apple’s and Android’s).
Can I contribute in Bitcoin?
If you would like to contribute Bitcoins, you may send them to:
Bitcoin contributions do not count towards our Indiegogo funding goal and are not eligible for the standard perks. However, if we receive at least 10 BTC into the above wallet, we will enable bitcoin as a payment option for our hosting service, and you will receive credit equal to your contribution. There is no deadline for this goal, but people who contribute before we hit the goal will receive an additional 50% bonus credit. In order to claim your credit once the service goes live, you will need to sign a message from the wallet address you used to contribute, so be sure you know how to do that (most Bitcoin clients and web wallets provide an easy interface for this).
I have no money. How else can I help?
Then, tweet/share this campaign with your friends! Short link: