Synthesis Studios

Sometimes the stuff that we build at Synthesis runs in the weirdest places. We love working on the boundary of the digital world and the real world; that means that very few of the things we create are intended for the “usual suspects” like web browsers, servers, or mobile phones. Because of that, debugging said software in said weird places becomes an interesting exercise. I got a cold realization of that last week when I spent a day climbing around a bus with an oscilloscope.

Synthesis has had the past privilege of working with some very interesting companies trying to define the market of “in-car computing.” While the notions sound so simple and grand (a processor in every seat, pervasive media access, high-speed cellular linking the automobile to the world) the real deal of properly getting the bits running are so different than getting them to run on the aforementioned usual suspects. Debugging a question such as “How did that function get called?” means something very different when you’re sitting in a car.

For example: “How did that function get called?” That’s always a fun one no matter where you’re trying to look at software. Engineers spend countless hours with debuggers inspecting code and trying to figure it all out — did the author get all his if-then-else statements right? Did the author have another bug in the code?

In my case, it was neither. It was a wire that came loose, fell down, and shorted something out. Its reminds me so much of why these errors are called software bugs.

At Synthesis we consider ourselves to be lucky — we get called upon to write code that goes in really “different” environments. It really gives us a chance to stretch our brains, arms, freeze a bit in the cold, and have warm, fuzzy, analytical thoughts about true systems.

I attended the Amazon Web Services developers meeting in Second Life this past Thursday. I’ll admit, at first I was a bit skeptical of the format in general. Aside from the “Woohoo! I’m playing games at work!” factor, I was not sure the benefits of using Second Life would outweigh the overhead. My fears were unfounded; I have been using this format, or something similar to it, for almost sixteen years.

The question may arise as to what sort of overhead I’m worried about. Before I was even able to attend the meeting I had to download and install the Second Life client. If I didn’t already have an account I would have to register one as well. All of this took less time than driving to a meeting in the same city, let alone transporting myself to one on the other side of the country.

My other big concern was if the game-like interface get in the way of attending the meeting. I found that it didn’t. Previous experience in video games  lead to a very shallow learning curve with the Second Life interface; the same has been true for everyone else I have spoken to about this. Granted, while launching rockets at your friends requires a different skill set than launching questions at a presenter, many of the mobility and environment interaction ideas carry over.

I think everyone should give this format a try. If Sun Microsystems has their way, it might not belong before meeting in a virtual environment is more than just a gimmick. I’ll be at the meeting tomorrow, Thursday the 27 at 1PM EST on the AWS island, if anyone wants to talk about this. My second life name is Huge Littlething (no pun escapes my grasp).

Arc is a new dialect of Lisp from Paul Graham, founder of Y Combinator and what is now Yahoo Stores. It aims to “make programs shorter” — PG’s unassuming description of what he sees as the fundamental purpose of programming languages. As an example, a function that returns double its argument can be written: [* _ 2].

The initial public release of Arc is itself a paragon of brevity: one directory, fourteen files, and less than 5000 lines of code (which includes a HTML engine and web-server). Check out the installation instructions and the tutorial, which assumes no prior knowledge of Lisp.

What’s fascinating for me, as a programmer and interaction designer, is the deliberate and public attention being given to the core expressiveness of the language itself.

What’s fascinating for me, as a programmer and interaction designer, is the deliberate and public attention being given to the core expressiveness of the language itself. The release is essentially a prototype of the things you can say and how you say them — and not of the efficiency or capacity of the resulting system. The former is a matter of design, the latter of engineering. Features like modules and Unicode character support were omitted, because although they’re fundamental to the implementation of the language, they’re not fundamental to its design. While the two undoubtedly affect each other, it seems appropriate to focus on design first. After all, what’s the point of a great implementation of a language that no one wants to use? (Although one could argue that this is the current state of Java.)

Some people have criticized Arc for being nothing more than a thin wrapper atop Scheme (in which the release is implemented) or for not having tackled enough difficult problems. As PG has pointed out in his rebuttals, he wants to provide the best language from the point-of-view of the people who are going to use it, whether the implementation is short or long, easy or difficult. In any design process, you want to do as little as you can to allow people to experience the thing you’re designing. This lets you iterate rapidly, try lots of ideas, and create a better result than you would by toiling away on your own. In the case of a programming language, of course, experiencing the prototype means writing programs in it - and for this you need an implementation. Don’t, however, confuse the prototype with the product. The former exists to generate new designs; only the latter is primarily intended for use. If you try Arc, do it with the intention of improving the language, not (necessarily) of building systems with it. It’s not yet ready for the latter.

Arc has some aspects that should help it to evolve into an effective way to build systems. Foremost among them: it has a leader.

Arc does have some aspects, however, that should help it to evolve into an effective way to build systems. Foremost among them: it has a leader. To see the importance of this, compare Python to Common Lisp. Python is run by one person (Guido van Rossum), giving it stability and consistency without inhibiting growth and change.

Common Lisp, on the other hand, has a fourteen-year old standard that’s an amalgamation of the work of many people — both inconsistent and obsolete. Anything not in the standard has likely been done in differently in each implementation, so many programs don’t work in more than one. This creates a fragmented eco-system of libraries, limiting the resources you can build upon. While it’s small now, if Arc can stay unified under the direction of Paul Graham, we should end up with a large, evolving, and compatible set of libraries and tools. In fact, we may finally get an acceptable Lisp.

My other hope for Arc (besides becoming a great language itself), is that it will generate broader interest and activity in the area of programming language design (as distinct from implementation). While there are lots of great new languages out there, most of them simultaneously evolve the language and its implementation, and limitations of one affect the other. By encouraging us to start with the design and then figure out the implementation, Arc should lead to languages that are better in both.

Since this fall, Synthesis has been working on a Facebook application that lets users create groups and use text messaging to hold threaded conversations privately within the group. We used several different technologies while developing this that made the architecture decidedly different from the norm for web applications.

For starters, simply being a Facebook app means there’s another layer between the server and the browser.

For starters, simply being a Facebook app means there’s another layer between the server and the browser. Facebook apps appear as part of the Facebook site, providing hooks to lots of features within Facebook itself, but at the cost of needing to work around under-documented features. The application is built using canvas mode, meaning that our server returns snippets of HTML and FBML (Facebook Markup Language) to Facebook’s servers whenever the user makes a request, and Facebook then renders it for the user as if it were a Facebook page. Facebook provides a ton of help if you’re writing your app in PHP, but we went with Grails in order to leverage the scalability of Java. This meant that documentation was sparse in a few key areas; we found that the easiest way to go was often porting code from Facebook’s PHP libraries. There is, however, a great third-party alternative to Facebook’s Java API that’s open source, has javadocs, and surprisingly enough, provides better compatibility with Facebook’s API.

For hosting and dynamic storage, we chose to built it on top of Amazon’s Electronic Compute Cloud (EC2) and Simple Storage Service (S3) services. The biggest reasons for going with them are that they provide lots of configurability, ease of setup, competitive pricing, and the ability to scale easily if and when the application takes off virally.

EC2 is great because it allowed us to upload a custom disk image configured to have exactly the software configuration we need. So, when the number of users starts increasing, we can turn on more instances and configure them to share the load. The details make this much more complex, of course, since the application itself needs to be built in a way that makes such scalability possible. However, the ease and flexibility of having custom images ready and waiting to be turned on when needed is a huge plus.

However, there are two big downsides to using EC2.

First is that the instances have no permanent storage. Each instance offers 150 GB of local storage, but it’s tied to that particular instance. When the instance is shut down (or crashes), it goes away. Storing data on S3 helps, but S3 uses block oriented storage that’s pretty different from a typical file system, making it difficult to simply treat it as an infinitely sized NFS mount.

Second is that the IP address assigned to the instance isn’t permanent. This leads to a similar problem: if anything references that IP address (either through DNS or any other third party service) and the instances goes down, those references will be broken. (Update 4/3/08: Cool! Amazon fixed this, now it’s possible to reserve a static block of IPs and dynamically assign them to instances.

We solved the first of these problems by implementing an aggressive backup strategy using S3. S3 is effectively an infinitely large storage space with all the replication and high availability features that Amazon’s own web site is built on. In the future the “correct” answer may well be to go with Amazon’s simple DB service. It’s currently in closed beta though, and the model it’s built around isn’t ideal for real-time updates, since there’s a lag between when a request is made to store data and when the request is actually performed.

Frequently changing DNS addresses are common enough to have a standard solution: dynamic DNS.

The second problem, of impermanent IP addresses, is much more straightforward. Frequently changing DNS addresses are common enough to have a standard solution: dynamic DNS.

This application highlights the evolution of web apps as both the underlying infrastructures become simultaneously commoditized and far more customizable, and as individual sites like Facebook begin offering APIs, making them into their own proprietary platforms.