I don’t understand the reasoning of so many ‘climate change skeptics.’

Let’s pretend it’s not about earth, for a moment, and just some black box, hovering in a vacuum, with a big lightbulb shining next to it. Practically all its energy comes from the lightbulb (the rest from the residual heat within, and some dim source of central power), and practically all of its cooling consists in radiating infrared back outward. Now you put a layer of glass, which infrared cannot penetrate, over the black box, and wait, and see what happens.

The infrared is significantly absorbed by the glass, largely radiated back to the box, and thus the largest channel for cooling — and pretty much the only one that will work over a long period of time — has been attenuated.

Now replace the black box by earth, the lightbulb by the sun, and the glass by CO2.

The black box would have to be totally weird in order to keep from heating up. It might, for some time, somehow redirect some of the heat into less observable sections of its mass (e.g. the lower levels of the earth’s oceans, which have a much greater heat capacity than its atmosphere), but this cannot last forever. It might also become more reflective, absorbing less light (e.g. the earth’s clouds, desertification)? Except clouds are totally observable, and an opposite effect comes from the melting snows and ice caps and constructed asphalt we add in urban areas: all of which have radiance and albedos observable from the outside (e.g. our satellites). Finally, the black box radiation is proportional to the fourth power of the temperature, so even if the percentage of radiated power that reaches the outside of the glass is diminished, if the temperature of the primary radiative bodies becomes less even, such that ∫T(new)^4 dA >> ∫T(old)^4 dA, the temperature can stay roughly constant. Other than that, there’s close to nothing that can be done: that box will very probably rise in temperature, and almost certainly the climate will change.

Skeptics correctly points out that the lightbulb varies in power output. And the black box is moving a bit relative to the light — further away or closer by — shinier or cooler or more black parts facing the light at any given time. They also point out that the glass isn’t the only thing surrounding the black box — for example they have noticed also a shiny layer of dust on the glass (aerosols), and an even bigger layer of glass underneath the glass we’d place (water vapor). And they point out that the layer of black paint appears to be, in a great proportion, liquid, and with a high heat capacity, and churning cyclically, and that there’s a lot of it, so that in any one instance a cooler or a warmer parcel of that liquid is showing.

None of this changes a thing about the fact that if we put yet another layer of glass on the box, the smart money is on it heating, and certainly on it changing. How could it not? At this point the onus is totally on these climate change skeptics to suggest a means by which the box is supposed to stay exactly the same.

Which brings up an interesting point. Maybe it is not so necessary that the Earth stays the same. Maybe there are credible arguments that explain that, really, the box won’t change that much, and for the teeming, glass manufacturing cultures of microbes living under the glass that these changes are not really such a big deal.

Some scientists, who I respect very much — Freeman Dyson for example, make this very argument. I respectfully disagree with him, as I think that there’s far too much risk in disrupting the biosphere, and that the disruption, famine, and loss of ecosystems and species that have already occurred are too great a price to pay, that oil wars, tyrannies, and people dying of respiratory illness from coal plants aren’t exactly positive either, and estimates of the probability of some catastrophic event happening, like say, Greenland melting, the consequences of which are too dire to imagine, range somewhere between 10% and 80%.

But that’s a philosophical disagreement. One might say that instead of engaging in ‘climate change’ skepticism, Dyson and others are engaging in ‘climate problem’ skepticism.

Too often, what we have with ’skeptics’ is a scientific disagreement: the great majority say either that it is happening, but only as part of natural variation, and they had nothing to do with it, or that it isn’t happening at all. Which, at this point, seem more like the antics of a child screaming ‘I didn’t do it,’ or putting their hands to their ears, singing ‘la la la, I can’t hear you!’ than of a calm and reasoned scientist — or skeptic — examining the assumptions of a majority opinion. Their conclusions are already drawn.

—
PS:

For those already sold on the problem, my startup, LightSail Energy, Inc. is an exciting, well-funded startup in the $100 billion field of green tech energy storage. We are located in Oakland, California. We are seeking to fill several Mechanical Engineering positions. Applicants should have at least 5 years experience in product design of mechanical components.

Please be familiar with at least some of the following:

engine design, fluid dynamics (analytical, experimental, computational), heat transfer, thermodynamics, fluid power, pistons and seals, and multiphase phenomena.

Candidates should be comfortable with 100 kilowatt to multi-megawatt systems. Our needs range from mathematical modeling and design of experimental apparatus during the Research and Development phase, to designing for manufacturability and reliability. The ideal candidate will be a hands-on design engineer who possesses a high level of creativity and innovation required to be a valuable asset to the company.

Interested? Please send your resume to jobs@lightsailenergy.com.

Where laughter becomes more hilarious than whatever half-forgotten thing preceded it.

Also known as H.E.H.

I discovered today, to my amazement and horror, that the otherwise exceptionally advanced and useful Mathematica 7 has only a single level of undo in its notebooks. And no redo.

Furthermore, the lone autosave feature, hidden deep, is triggered upon every cell evaluation. Every time you evaluate something in a notebook, it will save the file – overwriting the last version.

I spent an hour looking to see what was the matter. I was convinced that it must be my own obtuseness causing me not the find or see or properly use the actual undo function there present. Instead, I found several newgroup threads in which people exclaim their amazement that such vital features as multiple undo and revision control hadn’t made it in, and apologists of various employ, Wolframites and others, who wrote back about the academic difficulty inherent in making multi-level notebook undos for notebooks fed with dynamic data.

I’ve got news for you Wolfram Research. It doesn’t matter. What people care about is their input data. What people don’t want about is destroyed work – the product of their labor, channeled through from their mind and out through their fingers, typed into little, nicely formatted cells. Almost everything else can be reconstructed from that input. The cases in which the inability of a simple multilevel undo to capture changes in dynamic data could possibly destroy work make up a fraction of a percent. Catastrophic hard disk failure is more likely.

I checked for a while to see if anyone found a solution more advanced than to manually save often, and use standard revision control. I couldn’t find anything.

Thus, I present my complete hack of a solution. It seems to work acceptably well — as in, I think I can rely on it to save my work, most of the time, if it really came down to it. But use at your own risk:

1. Get Dropbox. (getdropbox.com) This is a file sharing and revision control system that’s built right into the operating system, Windows, Mac OS X, Linux. One creates a folder, shares it over Dropbox, and every file in it is mirrored, automatically, on their servers. In particular, they keep track of revisions: so far as I can tell, every saved file is queued to be uploaded.

2. Enable Autosave. One can set this for the current notebook by evaluating the following line: SetOptions[SelectedNotebook[], NotebookAutoSave -> True]. This setting persists across saves.

3. Now every time one evaluates a cell, the notebook is saved. Further, while the file is apparently overwritten locally, the revision history (which I believe is, in most cases in full,) has been uploaded to Dropbox, or queued by the client.

4. To recover a revision, one can check the revision history in the Dropbox web interface.

5. Standard diff facilities are less useful, because the notebook files are highly structured. However, there’s a structured diff built into Mathematica, Notebook Diff in the AuthorTools package. One loads the package by evaluating << AuthorTools`. One can run this command NotebookDiff[nb1,nb2], where nb1 and nb2 are either notebook objects or their filenames with full paths.

This seems to work well enough for now. In the meantime, Wolfram Research, get your act together! You are driving people crazy.

I am also putting together a way to use the Units` package throughout (much more of) Mathematica, in particular, in conditionals and numerical functions. In practical engineering it could often be said that converting units is one’s primary occupation. For this reason the units package came to me with extreme promise. Built into the algebra package as it was, it could be used, in an extremely simple way, to check dimensional and unit consistency. However, it is not natively compatible with numerical functions: one cannot plot expressions including units, or expect results from comparison operators, and they can’t be used with functions like Max and Abs, or within the solver for numerical differential equations. It would be so wonderful if it could. I think I’ve figured out a way forward there, but we’ll see. The basic idea is to set to 1 all base units in a particular unit system — SI or CGS. This is likely to work for mechanics, however due to the differences in how the CGS and SI systems define electromagnetic quantities, making it work well for electromechanics is either a subtle project, or a doomed one. [Edit: David Park's ExtendUnits package appears to do exactly this. It's $30, though. http://home.comcast.net/~djmpark/UnitsPage.html]

Also, I don’t know if anyone’s really following this, but we’re not working on vehicles specifically anymore; LightSail is directing itself towards energy storage as a first product. Our website is woefully out of date. Vehicles or vehicle systems may be a further product and direction of ours, but by focusing on energy storage we can really prove out the core technology without the additional hassles of becoming an automotive company.

you have saved me from an eternity
of what if
with one moment of yes

– M. Volkova

Once upon a time in Northern Mexico, my mother was sick. She had come down with a cold. Sitting on her porch, an old, tiny Mexican man walked up to her and said, “Beautiful lady, why are you looking so sad?”

“I have a cold!” she said. “I am tired and chilled. I feel like nothing.” Hearing that, he took a small flask from his jacket. “Take a swill of this. You will feel better.” It burnt on the way down. “Tequila perhaps,” she thought, “if it is, it is the best I have ever tasted.”

She said nothing. With a smile, the man shifted his hat, turned his body, and walked away.

The next day, he visited my mother, sitting on the same porch. “Are you feeling better?” he asked, reaching for his flask. “Maybe a little,” replied my mother, quaffing the fiery liquid with uncertain haste. “Good,” he said. And with that, he pulled the collar of his jacket taught, smiled, and walked away.

In a blur the days passed. The old man visited the next morning, and the morning after that, and on and on for a week. Finally, he asked, “Are you feeling better?”

“I am!” she said. “Good,” he replied. He shouldered his bag and turned to go.

“Wait!” she faintly cried, “Won’t you let me sip once more of your medicine?”

“More tequila?” the man said concernedly, furrowing his brow. He waved his hands in circles towards her. “There is no need. You are cured.” And with that, he gave a smile and a wink, he turned, and walked away.

The clock stuck twelve. It’s October 30th. In a heartbeat I emerged an adult in the eyes of American law. In an alternate universe, I danced the night into a hazy sunrise. But I left celebration to Haight St. patrons, their addled revelry spilling muffled through the crack in my window. Tonight, we work.

Dawn, a night and two weeks later. It was ready; the design for the both the engine and the drivetrain, encoded in a scattered handful of drawings and documents, one wiki, two heads, and a thousand lines of physical simulation code. The first test: powering a scooter through a staccato ride amid frenzied Manhattan traffic, calculating, by the hundredth second, the will of the engine, and the vehicle’s reply. We’d follow a path devised to track emissions from humming, throbbing combustion engines, byproducts of fuel burnt in tiny explosions sparked every second by the thousands.

But nothing save cool air would our machine exhale. Compressed air, ‘a thermomechanical battery’ of sorts, is cheap, long lasting, and quick to recharge (one need only open a valve, and if impatient, run a pump, the tank will fill in seconds.) What’s more, it’s efficient. A batteries charge begins life in mechanical form, in a spinning turbine if charged off the grid, or in the inertia of a vehicle, during regenerative braking. This is then converted into AC electrical current, which is converted into DC current, which, finally, is converted into mechanical energy, losing power at each step. To power the engine this whole process runs in reverse! But compressing or expanding air keeps mechanical energy mechanical (so long as temperature is kept reasonably constant.) In powering vehicles it is superior to the most advanced battery systems known. That is, in every parameter but one.

Historically, the low energy density of compressed air had crippled any attempt to venture further than a couple dozen miles; physics, it seemed, demanded tanks of excessive proportions to travel longer. At 300 bar (’scuba pressure’), compressed air could release only half a percent as much energy as the same volume of gasoline burnt. We understood, however; it was an efficiency war. We knew that conventional vehicles were incredibly wasteful. There were many battles left to fight.

The Laws of Thermodynamics¹

“You can’t win.”
“You can’t break even.”
“You can’t give up.”

We hunted losses relentlessly. We were repaid with a series of compounding improvements, each building upon another, reversing the conventional patterns of efficiency losses endured by vehicles for more than a century. Finally, in a brilliant and unusually compact layout by Steve Crane, we found room to replace the paltry 1.3 gallon gas tank with one ten times its size. Nights yielded to our toil, and, slowly but surely our enemy routed.

“We’ve cracked the code,” we exclaimed. “The city is ours to conquer.” On the highway, whatever benefit earned by our scooter’s light weight, low rolling resistance and ultra-efficient regenerative braking would be dominated by air resistance.² But air resistance falls quickly with speed, and in the stop-start motion of the city our combined inventions would give our scooter an efficiency historically unmatched.

I keyed in the last few drivecycle parameters, drew a shallow breath, cocked my head, and pressed the enter key. The simulation lasted only a moment, but in that time, my little scooter ran more than one hundred and twenty miles, the equivalent of dozen rides between Wall Street and the Bronx on a single tank. “We’re in business,” I said. With that, and for all of a New York Minute, the questions, worries and restlessness retreated from our hearts. We huffed. “What’s next?”

[1]: To paraphrase C. P. Snow. Hat tip to Jonathan Smith.

[2]: Scooters are not particularly aerodynamic vehicles. Ordinary scooters have a drag coefficient of nearly 0.9, and a frontal area of 0.6 meters squared. We hope to achieve a drag coefficient of 0.6, similar to faired motorcycles ridden upright, but due to the rider’s position this will be difficult: some have described the aerodynamics of a scooter as like a “brick attached to a parachute.”

I have here several charts of driving cycles. These are standard plots derived from real traffic data, of velocity versus time. Unfortunately I cannot find the data anywhere. So I hatched a plan: maybe there’s software that will reconstruct data from a graph or chart? Does anyone know? If not, I’ll just write it myself and open source it. It seems like a generally useful thing in engineering and science. It deserves an application (maybe even a web application).

Many believe that technology simply gets better over time: that every class of invention can improve endlessly into modernity. That is not so. Most of the hard constraints on technology are imposed by physical or mathematical laws. These remain constant. Those who truly understand this may work, instead of towards the solution of individual problems, towards timelessness, and the ideal platonic form.

Excavated at Choghazanbil Ziggurat, near Susa, late 2nd Millenium BCE.

Jenny Boriss and the Mozilla Labs team have helped spark a wildfire of discussion. What can we do to make browser tabs better? I’ve been considering this question for a while now (heck, I practically *live* in Firefox), and Aza Raskin’s recent posts have finally inspired me to weigh in.

This may sound strange, but if you look at it a certain way, the introduction and popularization of tabbed browsing represents a tipping point in the history of computers. And, as they have altered and improved our ability to multitask, they represent crucial advances in the history of human thought.

There’s no fundamental difference between a tabbed web browser and an operating system taskbar. They are tools for multitasking. They are intended to make accessible each distinct thing which we seek to do, on our computer, so that we might easily glance at, remind ourselves of, or focus our attention on, whatever is most profitable, intriguing, or demanding of attention. Tabbed browsing popularized as web documents and applications overwhelmed their desktop counterparts. Computers had altered part of our working memory. Now, the web plays a leading role.

Computers have caused such an increase in our ability to multitask that they have precipitated what is one of the greatest generational shifts in patterns of thought since the scientific revolution. Quite literally, when we alter the way we multitask, we alter our minds.

Dangers

This is not without dangers, of course. We can still only keep so many things in mind (7 plus or minus 2, for most of us). Computers can display more tasks and more information than we can handle, and attention remains a finite resource. Computers are only a level two cache. If we don’t carefully control how we drink from this firehose, our attention may shatter, rendering us totally useless. For some, in today’s YouTube generation, this seems to have happened.

If tabbed browsing is a tool for thought, then we might expect that even small alterations in tab behavior will affect habits of mind. Consider the popular method of conducting research. One begins with a search, and lands on a Wikipedia page, or blog article. As one scrolls down the page, we middle click, opening tab after tab in the background. But where do they go? If the tabs land go directly to the back of the list, ordering tabs by age, we end up with something kind of strange. People tend to move to the next tab in the list, and so, the tendency is for our poor researcher to perform what is known in tech circles as a breadth-first-search.

There’s something to be said about this. Breadth first surfers only skim topics. They’ll cover many in one sitting, certainly. But they’ll tend not to learn in much detail. They will, in other words, learn many sides of many things, superficially. Shallowly. This could be one of the major reasons why, online, so many people talk so much about things they know so little about with zero apparent knowledge of this fact. Draw your own conclusions.

If, by contrast, tabs are opened directly behind the parent tab, we lure our researcher into a trap of a different sort. A depth-first-search.

Usain Bolt wins the 100m! Who’s he? Let’s check the Wikipedia… ah, it says here he was born in  Trelawny, interesting name — it’s a parish? Isn’t that like a religious thing? Hmm, it can be, but maybe it’s a civil parish, which is, hmm, a subdivision of the United Kingdom? Huh? Oh — I guess it’s part of the Commonwealth. What’s with that name anyway… seems kind of random… for “the promotion of democracy, human rights, good governance, the rule of law, individual liberty, egalitarianism, free trade, multilateralism, and world peace“, I guess those are good, especially… individual liberty… heh… I wonder how many times Dubya has trumpeted that one… Dubya… Maybe this has a list of Bushisms… which are apparently malapropisms… there’s a mouthful… where was I?

It’s easy to joke about, but it’s no laughing matter. It happens to the best of us. It wastes our time.

A Personal Note

People don’t always read straight through the tab they have open, and move onto the next. They zip around. Or they’d like to. Or, at least, I like to. When tabs first came out, I used to fill my browser with bajillions of them. You’d barely be able to click on one, much less recognize one by it’s hidden icon or title. There were tabs a pixel wide. Less, if I could get away with it!¹

So what did I do when I wanted to switch back to, gmail, say? I couldn’t find the old gmail tab. That would be impossible. So, I’d open a new one!

Soon, I’d have a zillion gmails, which would slow my computer to a crawl. And the tabs would be almost completely useless: it was totally impossible to switch around, all I knew if that there a lot of articles I wanted to read in there, and if I just kept working on the pile, then eventually the titles would see the light of day and I could use my tabs again. Usually I just crashed the browser. Oy.

Firefox 3 changed a lot of that, but not entirely for the better. Instead of trying to squeeze fifty tabs into one space, it just hides them offscreen. But the same problems come up: I still opened googles of gmails, and my browser would slow to a crawl. Like cold molasses, running up a hill. Honestly.

I’ve learned a little since then, and I’ve made some improvements. Some are technological — little plugins, restoring my sanity. The first is ‘aging tabs‘, by Dao Gottwald. It’s a brilliant little Firefox plugin that makes tab headers fade with age, from light to dark. It helps by reminding you what you were reading most recently.

The second is Tabhunter. Those familiar with Quicksilver or Enso will recognize its function: you just press ctrl-alt-t, type part of the name or URI of the open tab you’re looking for (like ‘gm’ for gmail)², and a list of matching tabs is displayed. Press up or down and enter to select the tab, and you’re there. There are a few minor problems with the design I think; for one, it’s modal (a box pops up, and you need to hit enter to dismiss it), and it saves your old patterns instead of clearing the input, requiring more keystrokes than I’d appreciate, but overall, it’s been a huge boon for me. It makes it much easier to switch between tabs, and I love it.

Finally, I use fullscreen mode. I found having a list of tabs staring me straight in the face had the effect of luring me away, towards something else, whenever things got hard. Now, with fullscreen mode and Tabhunter, I can concentrate on my work or reading, and switch around, only to do something definite, when I need to. This changes everything.

Suddenly, keyboard shortcuts became best of friends: ctrl-1 shifts to the front of the list, ctrl-9 shifts to the back, ctrl-w closes the current tab, ctrl-shift-t opens the last one you closed (or the second, third, fourth last one, in order), ctrl-d bookmarks, ctrl-l focuses on the Awesome Bar, ctrl-k zips to the search bar. alt-left or alt-right to go backwards or forwards. It’s way faster. The first time I started to surf this way, I had nearly 60 tabs open. I just kept reading through, fully completing each task in front of me, dismissing it when done (closing by ctrl-w), moving onto the next item in the list. I was absolutely startled, in just an hour an a half, my browsing was complete. I almost couldn’t believe it.

This has changed my life. You might laugh, but too many times I’ve stayed up, dozens of tabs tormenting me, unable to close them, afraid of sleep, of losing mindstate. I browse faster, concentrate better, and learn more. And now I am free.

Concentration

The lesson, I think, is that computers hold great powers.³ They can remember what we need to do, and what we need to know, freeing our minds from needless memorization, allowing us to concentrate on the task at hand. Or, they can present to us an endless stream of distracting tasks, tweets, and you-tubes, shattering our attention, and any hope of getting work done.

With each distraction, my mind thrashed. Heisenthoughts collapsed. Mindstate was lost. And for what? It takes far too long to start upon challenging work. Frequent taskswitching rendered me barely capable of any challenging work at all.

With the tab window hidden, my world changed. No longer was I tempted endlessly by the idle promises of articles unskimmed, tweets unheard, posts missed. I simply had to concentrate on the thing in front of me, and judge it worthy of time and attention, or not. It worked like magic.

I stopped being distracted by thoughts that there might be something marginally better that I could be doing. Instead I just did it. When I was done with it, I closed the tab. It might be, much more closely, a breadth-first-search, but, damn it, it worked.

Troubles

The trouble with this approach, I found, was that I was literally following a breadth first search with my browsing, which was not always what I wanted. Often, Tabs would present themselves in exactly the wrong order. Yet I’d still be tempted to go through them.⁴

People keep tabs around. They use their tab system as a to-do, to-read, or to-check-list. They keep tabs around as applications — things like gmail, and Facebook, and Pandora, which you might as well be logged into all the time. But keep too many things around, and you create a massive cognitive load. By hiding my tab list, I lessened that cognitive load. I outsourced prioritization to my computer. But my computer wasn’t any good at it. I switched tasks less often, which allowed me to concentrate, which helped me read, and helped me work. But it probably didn’t direct me towards the best thing I could have been doing. I worked, but, in some sense, on the wrong thing.

Priorities

I explained this dilemma to my friend Joel, breadth-first/depth-first analysis and all. He said that I had a linked list of task, and what I needed, instead, was a priority queue.

Duh.

Okay. So that’s goal one. Tabs are tasks: make it easier for us to prioritize our tabs, and we make it easier to prioritize our work.

Memory

Tabs serve as a kind of memory. We use tabs as a way to remember things we intend to do. But what aging tabs shows us is that we can also use tabs to remember things that we’ve done: specifically, what we’ve recently been looking at. What else might we be able to show?

First, let’s give ourselves a great big space to work. Suppose, say, when we hold down ctrl-tab (quasimodally), a great big transparent tab display appears over our window. We can zoom in or out, and drag the display around. We can even give it momentum: physics, just like the iPhone. We can toss it around. On it, what might we show?

Let us channel Edward Tufte for a moment. Let us maximize information density. Let us minimize data ink.

A Paper Mockup for FoxGlide

We can show content. We can display a title, with the favicon, atop a thumbnail of the tab. This is much better than just showing the favicon, as screenshots are far more likely to be distinct, and inform the user of the nature of their content. We can also show which bookmark folders that tab has been placed in or which tags the tab has adopted.

We can show order. Lay the tabs out horizontally, in the order in which they were opened. (Just like the current default, in Firefox).

We can show time. Near the top of the screen, make a small ticker, and mark the horizontal axis with small, vertical ticks: by clocktime, at distant regular intervals, and with icons at every change in activity.

We can show link structure. Connect every parent by an arrow to their children.

We can show relatedness. If two tabs share a domain, if they commonly are switched between, or if they share a large amount of semantic content, draw a faint line between them, to be highlighted when the tab is selected. In this way, we can automatically cluster groups of tabs used for a shared task or activity, and remind the user of tabs habitually used together.

We can show history. Turn the ticker at the top into a lifestream. At each tick, denote the activity by a little icon. Suppose you’ve done a bunch of typing: this is a writing tab, we can represent this with a little ‘key’ icon. Suppose instead that you’ve just scrolled and read: show scroll buttons, up and down. Suppose we’re idle: show an ellipsis. Suppose we open a new tab: show a sprout. Suppose we open a tab in the background, creating another branch: show an arrow branching into two.

We can show age. Let the tabs fade with disuse, as in aging tabs.

We can show status. Make the title of as yet unviewed tabs blue, as in unvisited hyperlinks. Make the title of visited tabs purple. And instead of wiping closed tabs off the display, make the titles brown, and let them inhabit a ‘history gutter’ nestled quietly at the bottom of the screen — to view it more fully, pull-up. when a tab is closed, maintain the arrows, connected to parents and children, but fade them so that they do not distract. By keeping closed tabs accessible, we seamlessly integrate browser history with the tab display, enabling browsers to enhance our memory still further.

Finally, we can show priority. Let the vertical axis represent priority. Higher implies more pressing. Priority can be set automatically: Age lowers priority. Activity raises it. Or you can set it manually. You can drag each tab along the vertical axis to set the priority, or you can strike a number, to alter the priority of the tab currently viewed, or otherwise selected. We can even use a hybrid approach: after a while, an algorithm harness the implicit, ambient information from our browser use, to learn which sites, keywords, and activities you consider high priority, and set priority for every other site you visit accordingly.⁵

To make it easier to set the priority for large groups of tabs, the priority of related tabs, for example, parents and children, influence each other. This implies that, while raising the priority of one tab, the arrows connecting parents and children pull the related tabs up elastically. We can also uncover tab relatedness by whether they share a domain, or if we would like to get really fancy, we could even link tabs by shared usage of ’statistically improbable phrases’, as Amazon might.

Navigation

With the tab display active, we can quickly type to find the tabs we want. Type ‘wiki’, and all the tabs matching ‘wiki’ are shown, filtering the others out, which fade away. Hit left or right to progress through the history of open tabs: in the same order as is currently used by Firefox. Hit up (or enter), and we switch to the the highest priority tab (that isn’t being currently viewed) matching the filter we typed. Hit down, and we instead decrease priority, stepwise.⁶ It’s like Tabhunter, but using less keystrokes.

The labs team has already come up with a decent navigation system. They suggest that we put navigation in the Awesome Bar. If someone goes to the Awesome Bar and types a pattern that matches an open tab, pressing enter will take them to it, while pressing option enter will open a new tab.

I worry about this a bit. As much as I like the Awesome Bar, keeping focus on it is modal. And you need to select it (ctrl-L, or a mouse click), and then type in the name, making it a bit more complicated.

Also, it messes with important prior behavior. Most people will expect to go to somewhere on the current tab when they hit enter on the awesome bar. Changing this behavior might confuse users, conflicting with previous program models, doing so in a way that does not make evident that program behavior has changed.

Finally, compared to the approach I describe above, it simply doesn’t show as much information. It’s not as useful as it could be (note: see the mozilla labs wikipage on visual navigation for more ideas.)

Status

Where might you be able to download this? You can’t, yet. It is, for now, just a twinkle in my eye. But we’re on the cusp of something. The Mozilla team is just now in the process of deciding what new tabbed browsing features they want to build into Firefox 3.1, and are soliciting ideas. If you like this concept, or think, at least, that some of the ideas are worth considering, or if you have your own ideas, please, get involved in the conversation!

Also, if you’re interested in helping to do some mockups or some Firefox extension coding, please let me know (email me at daniellefong at daniellefong dot com). It will stay a small, open, simple project; this won’t be my fulltime thing. But for me, at least, I cannot ignore the chance we have to deliver ourselves from cognitive overload. To free us to explore the web more fully. To enhance tools for thought, and thus enhance thought itself. As whimsical as that might sound, this hope shines far too bright for me to simply let go.

Notes:

[1] – In other words, without crashing the browser. Opera was much better in this regard than Mozilla.
[2] – This actually works with regular expressions too.
[3] – With great power comes great responsibility.
[4] – This was not the only problem. sometimes, I’d see a link, and I’d want to follow it, just a little ways, to see what was there. But if I opened it in the background, I’d have to finish all the other tabs just to get there. Ahh! Instead, I just followed the link without opening a new tab, with the hope that I’d remember to return to the parent page and alt-left backwards. Soon I found the hotkey for flipping to the end of the list (ctrl-9). That helped a bit. I could open a tab, and flip to it in back, though I’d have to use Tabhunter to get back where I started.
[5] – It’s *Xobni for Firefox*! The web is the killer app! (I promised myself I’d never say things like this, but I actually kinda like it.)
[6] – Note that under this key scheme, ctrl-tab doesn’t switch to the next tab like it used to. That would now be done with ctrl-tab-right.

Thanks

to Alex Lang, Nick Pilon, and Joseph Perla for reading drafts of this, Sasha and Patrick from BaseShield, for the encouragement, the Mozilla Team, for running this wonderfully open design process, Blacktree and Humanized, for showing me how good navigation could be, Jenny Boriss, for sparking the tabs discussion, and Joel Muzzerall, for the inspiration.

PS

I’ve given some thought to the name. I, personally, like FoxGlide, which also exhibits the wonderful feature of my having parked it.

Please Leave Your Comments!

This post has sparked some interesting on Hacker News. What do you think? Does the design pass the Phone Test? Are the angles I haven’t considered? Are like features you’d like to see?

heisenthought,

\ˈhī-z^ən-thȯt

noun.

1. A thought subject to mental collapse triggered by interruption.

2. A mental event or sequence characterized by coexistence of loosely related, sometimes contradictory, sets of knowledge or tasks. Highly dispersed in knowledge space, though it may support well defined momentum. Apt to tunnel through imposing technical or energetic barriers.

Collapse triggered upon the observation of the state of current knowledge by a third party. During the beginning states of observation, keen thinkers engage in a mad scramble to save mindstate, which may at some later time be partially reloaded. Status reports are heisenthought’s natural predator. Occasionally, members of a team will spontaneously ‘go dark’, in an attempt to think through some difficult problem — sometimes the only strategy which will work to extricate oneself from local extrema in solutionspace.^1,2 Caution should be taken when observing said people, as doing so will both collapse their progress into a single, likely suboptimal path, and be likely misinterpreted as disloyalty, hubris or laziness, since they have disengaged from discussion with no apparent progress.

3. A mental event or sequence characterized by firmly specified and demarcated generating bodies of knowledge, and ill discernible forward movement. Induced by overspecification of problem, method, or solution. Collapse upon observation of current direction of progress by a third party. “‘Are we moving forward on this?’ ‘The heisenthought collapsed; we’re moving, just not in the right direction…’”

4. A coherent collection of thoughts sufficiently isolated from random outside influence. Obeys the relation Δknowledge Δmomentum ≥  ħ/2.

5. A tongue-in-cheek term invented to cast off workplace ‘pings’, ‘check-ins’, ’status-reports’ and ‘how’s it goings?’.

Notes:

[1] – For example, read the interviews with Max Levchin in chapter 1 and Blake Ross in chapter 29 of Jessica Livingston’s Founders at Work.

Max discusses how he went dark for a year, and came up with the something that really mattered — a way to fight fraud.

We had this merger with a company called X.com. It was a bit of a tough merger because the companies were really competitive—we were two large competitors in the same market. For a while, Peter took some time off. The guy who ran X.com became the CEO, and I remained the CTO. He was really into Windows, and I was really into Unix. So there was this bad blood for a while between the engineering teams. He was convinced that Windows was where it’s at and that we have to switch to Windows, but the platform that we used was, I thought, built really well and I wanted to keep it. I wanted to stay on Unix. [...]

I had this intern that I hired before the merger, and we thought, “We built all these cool Unix projects, but it’s kind of pointless now because they are going to scrap the platform. We might as well do something else.” So he and I decided we were going to find ourselves fun projects. [...]

It was me acting out, but it was kind of a low time for me because I was not happy with the way we were going. Part of having a CEO is that you can respectfully disagree, but you can resign if you don’t like it that much. But then eventually I became interested in the economics of PayPal and trying to see what’s going on in the back end, because I was getting distracted from code and technology. I realized that we were losing a lot more money in fraud than I thought we were. It was still early 2001. If you looked at the actual loss rates, they were fairly low. You could see that we were losing money, but, given the growth of the system and the growth of the fraud, fraud was not that big of a problem. It was less than 1 percent—it was really low. But then, if you looked at the rate of growth of fraud, you could see that, if you don’t stop it, it would become 5 percent, 10 percent of the system, which would have been prohibitive.

So I started freaking out over it, and this intern and I wrote all sorts of packages— very statistical stuff—to analyze “How did it happen; how do we lose money?” By the end of the summer, we thought, “The world is going to end any minute now.” It was obvious that we were really losing tons of money. By midsummer, it was already on a $10 million range per month and just very scary.

[2] – Blake discusses how, by necessity, they closed up the source and process during the early stages of work on Firefox.

Phoenix was basically a fork of the Mozilla code base that we controlled. We closed off access to the code, because we felt it was impossible to create anything consumer-oriented when you had a thousand Netscape people in search of revenue and a thousand open source geeks who shunned big business trying to reach consensus. We just wanted to close it off and do what we thought was the right thing. We went through a couple name changes, Mozilla offered us more support, and that’s kind of how it all got started.

Thanks to Marc Chung and Joel Muzzerall for reading drafts of this.

Our company is going to have graph paper placemats. Jealous? ;-)

analogies, once preserved as mere notes, find harmony as poetry

organizations
relationships
communities

as living entities

as where no one cell holds the soul of a person
no one person, incarnate
may form all essence of a community
nor live throughout its lifetime
wholly part

as one cell expires
one soul retires
others, some new, fill their place

generations cycle
yet something remains
an living entity unto itself
though it may be true
that some people or ideas
resemble vital organs

the liver, to filter
the stomach, to process
the lung, to supply
the mind, to direct
the heart, to power

too often
transplanted,
perfectly good hearts
from one being to another
from one community to another
raise immune reactions
edicts like allergens
mobs like histamines

without disarmament
no transplant succeeds
no hire, restructuring, or revolution carried through
without somehow disabling
or surviving
societal antibodies

for some, it seems
while becoming part of a great organization
or movement
or organism
they enter through digestive tracts
to supply the raw stuff
the best
carbon, iron, calcium, oxygen
skills, goals, ideas, selves
are prepared
and cooked
chewed
dissolved
filtered
processed
and made, finally, into a part of something new
to fit some other master plan
a genetic blueprint
an ideology
a mission
a politic

for some
no greatness of which they are part
could come to redefine them
their raw stuff
the parts comprising
can find no better molecule
no further local minima
no structure more solid
between surrounding stomach walls
they are indigestible

or too delicate
too beautiful
too unique
to be eaten

their soul to realize
as part only whole
of something profound,
meaningful,
believed in
or as the kernel of a gem
of another body
of an essence of its own
starting anew

I base my action upon prediction. Every technologist should. I try to see how the world will be, and then try and see within that future what place I may come to hold.

So prediction is fundamentally at the heart of a technologist’s work. At the highest level, we must predict to find what work focus on, and what future to aim for.

You might then think that prediction, as a skill, is worthy of practice. And practice it gets. In living rooms, in pubs and classrooms and yearbooks and dial-in talkshows and newspapers and blogs and comment threads and slashdot and every polluted corner of our existence, you find evidence: prediction is practiced all the time.

There’s a problem. In most areas of the technologist’s pursuit, it’s easy to see whether you’ve done well. Code should compile. Planes should fly. Cars should go. Bridges should stay up. We have a lot of honesty in our discipline, much of it because we are blessed with tests that we find hard to fool.

A typical test for predictions, on the other hand, is whether the story sounds good at the pub. You make some exclamation. People nod and clap. Everyone forgets.

This would be fine if you’re just looking for some conversation. But if you are, like technologists fundamentally in the business of creating the future, it becomes lot more troublesome. We are left to ignore predictive incompetence until reality slaps us coldly across the face. We are flying blind.

Taking a cue from Trevor Blackwell, I’ve decided to inject some rigor into my life: when I make predictions, instead of casting them abstractly into the air, I’ll post them here: einfall.slinkset.com. And I won’t delete my predictions — if they turn out wrong, I’ll keep them there, as permanent reminders to learn from.

Through accountability, honesty. Through honesty, improvement.

Thanks to Trevor Blackwell for the inspiration, and John and Brett from Slinkset for the List Hosting.

Notes: a friend of mine noted that most of my predictions seem ‘pessimistic’, in the sense that they take the form of ‘X will not Y.’ I would have to agree with him. But this is largely a byproduct of how these predictions were made – they’ve come from studying some field, working in it for a while, and coming to the creeping realization that one or more of the current approaches were doomed. Besides, much of the skill of experts comes from the ability to ignore false trails.

Further Reading: An excellent site for major predictions (often with significant wagers) is Long Bets.

During the back and forth of exchange with a technical recruiter, he finally asked me what I was looking for. And so the floodgates opened.

This may sound weird, but I pretty much choose employment based on the promise of quality work. Other factors fade into irrelevance.

When I say quality of work, I mean more than the work environment, more than the magnitude of technical challenges, and more than the IQ of those I’d be working with. I want the opportunity to walk paths with the greatest hope of leading to first-class work. Nobel-prize winning kind of work.¹ This force guides me, and so inevitably I tend upstream of technological change. Money and prestige are mere proxies for what really want: to develop and inspire fundamental changes in the way people live. That doesn’t mean I need tackle the greatest problems humanity now faces (yet). What matters is that I, personally, have a reasonable approach. So I must always remind myself to, as Richard Hamming says, ‘plant the little acorns from which the mighty oak trees grow’ — because small projects can, swiftly and strikingly, grow momentum and value.¹

I find little value in submitting myself to some company culture. I instead mean to develop my professional values, ambitions, and goals: for example, I would like to develop new methods, make them available by open sourcing them and make them popular by evangelizing them. I’d love to be given the chance to teach what I’ve learned. Excellent people bring ideas and perspective to a communities of makers. Given time, I think this will evolve naturally into a company culture worth having.

Most big companies grow faster than they could build trust, to a size greater than strong values can be supported. Natural culture is the product of alignment of creative philosophies, and in BigCo, this is too often replaced with virtual company nationalism. Fascism even. I find this is more than distasteful. I haven’t really learned how to work within it at all. I could devote my efforts to such an organization only were there deeply meaningful work to be done. And why bother?

Angling to be upstream of technological change, I bait unusual questions and find surprising answers. Give me the choice between a VP position at a big five media company with oodles of benefits, and, say, work at an early netscape or google for a totally minimal salary, and I’ll choose the latter every time. I’m pulled towards organizations where I can learn about organizing, rather than learning about institutional tradition. It’s not important for me to learn about how to run a large organization: if ever I do, I won’t follow of the paths of current captains of industry. Instead, I intend to help grow large, leaderless, open organizations, and so I’d do almost anything for a chance to work with Caterina Fake, or Linus Torvalds.

I want to work on something I find deep personal meaning in. I strongly believe in supporting open culture. I don’t think I’d work for long in games or entertainment unless it could influence some social change. I worked at MochiMedia because it made possible an income stream for small independent developers where none existed before. This finally opened up professional game development from BigCos. Now, much innovation in gaming emerges from bedroom studios. Independent game developers can now commit to their art in a way they before could not.

Similarly, I’d work at YouTube rather than Hulu, even though one’s a startup and the other isn’t, because they’re more interested in involving everyone in the process. As Clay Shirky says, they’re interested in ‘finding the mouse’.

I want to work somewhere where I can truly make a difference. Why am I working in technology at all? Archimedes once said, ‘If you give me a lever and a place to stand, I can move the world.’ Technology is my lever. I need only find place to stand. This makes me wary of startups that try to do good, but aren’t particularly focused on doing it efficiently. I wouldn’t work for most charities. There’s too little pressure on them to focus — the tempering influence of market competition is replaced by government demands for ‘accountability’, which arn’t nearly so powerful.

There are numerous ‘ecogreen’ websites out there that try to promote simple, green ways of living. These may be virtuous, however, in terms of minimizing environmental impact I think they’re somewhat irrelevant.² Saving plastic bags won’t lift a toe on our carbon footprint unless we find ways to either cut down on air and automobile travel, or do it more efficiently. And on carbon footprints — global warming is, I think, a red herring — there are thousands of nasty effects of pollution from, say, coal-fired power plants that will hit even if global warming doesn’t occur (though I think, probably, it will). Too much of China now wears breathing masks.³

I can see myself dedicating myself to the right company, so long as our goal, philosophies, and ambitions align. Yet these are stringent requirements. So far, then, I’ve found it necessary to reserve some energy and time for my own projects. So I must be open with companies: with most, I want only consulting work, to help them with some particular project, idea or problem. And I want to be completely, totally honest with everyone about it, because so far, the high road has never let me down.

Sincerely,
Danielle

———

Notes:

[1] – From the classic talk by Richard Hamming, You and Your Research. I don’t, particularly, apologize for my ambition here. Why shouldn’t I try to do first class work? The Nobel prize winning part is purely incidental. But this is the kind of work I mean — a significant contribution, one that people can build upon.

[2] – The free, online book Sustainability – Without the Hot Air is an excellent read. It is the first thing I’d suggest to someone interested in seriously starting into environmental matters. I shouldn’t claim that small contributions to green living are completely irrelevant — each does have some small effect. Perhaps raising the issue of green living in our collective consciousness will have an effect greater at second order than I imagined. But so many of our behaviors are misplaced. Many people, for example, go out of their way to buy ’sustainable’ products at Whole Foods, say, when in reality, longer vehicle trips do more damage than almost anything you could buy. Many things are sold in a way to make you feel good about buying them. They don’t have any real effect!

[3] – This is worded provocatively, but pollution in China is a growing catastrophe. See ‘As China Roars, Pollution Reaches Deadly Extremes’ and ‘Where Breathing is Deadly’.

———

Thanks to Alex Lang, Ma’ayan Bresler, Nick Pilon, Colin Percival, Michael Nielsen, and Joel Muzzerall for reading drafts of this, and Charles Beatty, for sparking it.

PS: Certain misconceptions have been raised. Some feel that this is one demand of an over privileged generation. I reply to this here. Additionally, I am not, in fact, abandoning my startup. But I do need money, and a visa, so I am looking into either employment or seed funding.

Recently, unusual features of the cosmic microwave background, a ’snapshot’ of the early universe, have raised issues with our understanding of the Big Bang. A Caltech team has shown how we might fix our theories. They suggest that there might have been an asymmetry in the energy that once powered the big bang. If this is correct, anomalies in the CMB may be traces of structure from a time before our explosive beginnings.

True to form, when a discussion appeared on Hacker News I rushed to comment, and this article erupted from that attempt. The current scientific understanding of our cosmic origins is a mystery to the public at large, but it was only after I noticed the bewilderment of my fellow hackers that I realized how poor a job we scientists have done in conveying the motivation behind our discoveries.

This article represents an attempt to replace that sense of bewilderment with that of wonder. I want more than to explain what cosmologists believe. I want give people a deep sense of why we believe it, of how we’ve come to our current understanding, and of why we care.

Look close, and it seems the universe is lopsided.

The cosmic microwave background (CMB) is like a snapshot of the early universe. It was once all hot plasma, gas so hot that the atoms inside it were broken up. Because it was hot, it emitted light. Because it was dense, it was opaque: the light emitted couldn’t just pass through, instead it had to bounce around. But once cool enough, the universe became transparent: all the light could now travel freely. It was as if the photographic shutter of the universe was lifted.

The light from this moment became the cosmic microwave background radiation. Because the universe seemed to have cooled at almost exactly the same time everywhere, the CMB is, unlike almost everything else in astronomy¹, a picture of the entire universe at almost exactly the same moment in time. It is the best picture we have of the structure of the early universe.

The universe appears to have expanded evenly since then. We know it’s expanding now. Light is like a wave. Since the speed of light is constant, an illuminated object moving towards us has its wave crests squish together, turning bluer, and an object moving away from us has the distance between crests expand, turning more red. This is called a red shift. Since he knew the colors of certain celestial objects, Edwin Hubble was able to observe that the further something is from us, the more red-shifted its light, and therefore the faster it is speeding away.

Since we know that the early universe was hot, dense and small, and we know now that it’s cooler, sparse, big, and expanding, we can reasonably deduce that, long ago, there was a Big Bang. The universe exploded.

Strikingly, the CMB is almost the same everywhere you look. There are minor fluctuations, but even they seem to have the same distribution everywhere. The CMB, our best picture of the early universe, is extraordinarily smooth. It is one of the smoothest things ever observed in nature. This might not seem like a mystery. You might imagine that anything expanding, hot and dense would look roughly the same in all directions. It needn’t. Nebulae are formed by exploding stars, and they aren’t particularly smooth. In fact, in nature, it would seem, more often than not, that explosions are messy.

In 1981, Alan Guth suggested what might be called a ‘recipe for a universe’: inflation theory. Until then, nobody had come up with any good ideas for why the universe was so smooth and even. It is as if God² had pressed the entire universe with a cosmic clothes iron.

Guth said, suppose you started with pretty much any initial universe. Suppose you also had an extremely strong, extremely smooth field of energy. If this field started dumping energy into the rest of the universe, it would also evenly expand space itself.³ The universe would undergo a period of exponential expansion — inflation — having the effect of flattening and smoothing the rest of the universe. Inflation is God’s clothing iron.

A flat, smooth universe isn’t the only thing that inflation predicted. For example, at small physical scales, quantum mechanical fluctuations persist. During inflation these fluctuations are blown up as well, and these would seed, almost entirely, the cosmological structure of the universe. We see these fluctuations in the CMB. According to inflation, they are tiny quantum fluctuations blown up to a cosmic scale. They are, quite literally, the ancestors of our galaxies.

It wasn’t just that there were fluctuations. Inflation theory predicted a very specific distribution and type⁴. When people finally had the technological capability to check, that’s just what they found. The universe appeared, at a cosmic scale, astonishingly consistent with this simple theory. Yet recently our observational capacities have improved. A CMB survey called WMAP has uncovered several surprising and unexplained features, not all of which fit well with the our previous inflation theories.

If you divided the sky in half by tracing the orbit of the earth around the sun⁵, and compared, in each half, the size of big fluctuations, those between 3 and 5 degrees wide, you would come to the conclusion that one side has fluctuations outweighing the other by an alarmingly large amount. One side of the universe is bumpier than the other. Moreover, the difference is larger than would be accounted for by randomness, at least 99 times out of 100.⁶

This asymmetry looks real. It has been checked against every known experimental error and background effect astrophysicists have been able to think of. And if it is real, our previous inflation theories, with one field of energy to inflate the early universe, won’t work. They can’t account for this anomaly.

The authors Erickcek, Kamionkowski, and Carroll don’t merely point out this problem. They posit a solution. They describe another inflation model, consistent with our new observations. They suggest the universe had not one, but many fields of universe inflating energy. There’s just one problem. At least one of these fields needs to be asymmetric.

Where could such an asymmetry come from? It is possible that we’ll never know. Cosmology offers us the hope of uncovering consistent, compelling stories of our origins. Thousands of independent observations fit neatly in cosmology’s book. But while we may discover a few lost pages from our first chapters, we may never know all reasons why our book was written in the first place.⁷

Nevertheless, the authors make an exciting point. Wherever the asymmetry in the inflation field came from, it must have existed before inflation. It must have existed before the big bang. We had once imagined that time before our explosive beginnings would forever remain a mystery. Yet hidden in the CMB are hints of times earlier still. In this wonderful piece of work, the authors carefully consider what anomalies in the CMB could mean. And in the process, they may have discovered a way to look farther into the past than ever before.

Notes:

[1] – Since light moves at a finite speed, when we see something far away, we’re seeing light emitted in the past. What we see of something a light-year away is (at least) one year old.

[2] – I mean ‘God’ here as in a figure of speech. Feel free to substitute ‘Mother Nature’, ‘Allah’, or the ‘Flying Spaghetti Monster’ while reading.

[3] – What does it mean, exactly, for energy to expand space itself? It’s roughly analogous to blowing up a balloon. We know that the gravity of the universe, just like the elastic outside walls of a balloon, pull its contents inward. In a balloon, air pressure pushes against that inward force of the walls. During cosmic inflation, the inflationary force pushes against gravity. There’s one important difference though. We don’t actually know what the inflationary force is. Air blows up our balloons, but we have few clues as to what blew up the universe.

[4] – The quantum fluctuations predicted by inflation follow a nearly-scale-invariant random Gaussian distribution. These fluctuations show up in the CMB, and for the most part follow these predictions pretty closely.

[5] – The line dividing the two halves of the sky here is called the ecliptic.

[6] – Formally, this statement is true at at least the 99% confidence level.

[7] – There are some questions forever beyond our grasp. Even if we knew from where the Big Bang had come, we could always probe further, and ask where that came from.

An article on one bright young man, Moshe, recently appeared on Hacker News. For a long time I’ve been meaning to write about the subject, and what was to be a simple comment morphed into this essay.

The story of educational acceleration is an old one. Curious, bright children learn and explore rapidly on their own, and interactively with their parents. The world is like a playground for the growing mind. The child takes in everything. Eventually, these children find themselves mired in school’s morass. There are new adventures: more kids, older kids, a new environment. Yet kept in one place, individual attention of parents replaced by lectures from often overtaxed and uninterested teachers, their minds are left to go fallow. While some of school is new, and quite enjoyable, boredom and obedience, for the curious child, is torturous, a fact which lucky children and mindful parents come to confront.

Alternatives appear: skipping grades, dropping out, home-schooling, gifted programs, science fairs, participating in the popularity game, sports, focusing on musical or athletic achievement, playing hookie, becoming jaded.

After entering junior high I pretty much stopped responding to the world at large. Life rapidly degenerated. I quickly dropped out, and luckily my parents didn’t make me go back. At that time both of my parents were very busy with work, and so homeschooling couldn’t work for long. We discovered that college was much cheaper than private school, which didn’t seem very good anyway. We argued my way in.

I was given a huge amount of freedom. Like Moshe’s parents, and like many others, mine urged me to slow down — advice I tried to take, though my eyes were always bigger than my appetite. I was amongst a culture that respected me for good reasons. I shared so much more than with my age peers. I was learning so much. A growth spurt hid my relative youth — soon nobody suspected I was so young. I truly enjoyed my time in college. Nothing brightens a teenager’s world more than responsibility, respect, and freedom.

There are dangers. Early fame is a minefield, for many reasons, in many fields. I think it’s hard for most people to live up past successes, the ‘promise’ they had shown. But there are worse dangers. Two extremes haunt the gifted:

It is easy to feel that what one has achieved is in some sense artificial. That one has been built up out of nothing — that really, respect is in a large part undeserved. ‘Imposter syndrome’ is a common term for this feeling. People can feel themselves to profoundly wanting. Phony. But this is by no means exclusive to the academically accelerated. It follows early achievement where ever it can be found.

Alternatively, one can fall into a trap of picking activities on the basis of their impressiveness. One can miss out on more intellectually simulating, enjoyable, and valuable things in life, all in order to participate in a vapid achievement contest. One strives to be the youngest smartest fastest most daring most broadly educated highest iq’d phd’d nobel prized ivy leagued quantum genius prodigy in the world ever.

This happens way too often. And it affects all of us. One can feel so small next to the prominent. Long shadows can reach deep into other’s egos. Perhaps worse, all of us have felt a need to be great at times. It can become overpowering.

It’s a trait that is unfortunately encouraged by some parents who should know better. It’s certainly encouraged by society at large. The wiser guardians of bright kids, sometimes say that raising them is like having to constantly pull on the brakes. They must. The world keeps pushing them.

One of the saddest things is that it’s all futile anyway. Inevitably, you grow up. Prodigies become ex-prodigies. This is lost on too many people.

Beyond questions of ego and achievement, there’s another problem with acceleration. Those who advance so quickly do so by letting delight pull them. One falls in love with the rhythm of important, challenging work, with the excitement, with ideas, and with the culture of the mind.

But you can’t live life like this forever. One must learn other lessons while growing up. At some point, we all wish for a pause button. We all need time to get our identities together. We all need time to relax, and there is much that we need that cannot be provided by an overscheduled life. It is far too easy, with successes behind you, with fame around you, with drive and courage within you, and with a love of your work pulling you, to let that pause button go, and to refrain from asking for respite before hitting rock bottom. I know. It happened to me.

It felt like the world was ending. I felt like I was abandoning my love and my duty. I felt incredibly selfish. And I was exhausted.

It was not so bad, it turns out. My life rallied. It is difficult to explain how much better I feel. And surprisingly, taking a break, and taking time to reset myself personally, has even started to improve my work. Much of that many be a function of happiness. But part of this may help even the happiest of people. Pressure to do great things can rush life. It is so hard to pursue daring thoughts in a life too hurried.

So my advice to Moshe, and anyone who finds a glimmer of recognition in these words: stay curious, let your love of something pull you, but do not hastily give all of yourself and find there is nothing left to give. Don’t engage in the achievement rat race. Listen to praise from those you respect, but ignore it from those you don’t. Push back when society eggs you on. Focus on things that matter the most to you alone. Ignore fears of being an imposter — be authentic to yourself, that is enough. Keep your hands on the brakes. Enjoy your freedom of time and mind. Pursue daring thoughts when the urge strikes you. And when your heart begs pause of the world, listen.

Further Reading:

Those who’ve spent time with me over the past few months know both how absorbed I’ve been in the catalysis of our startup, and how poor I am at concealing my admiration for YCombinator. We had poured startling effort into building our product, honing our idea, refining our pitch. But our focus was, perhaps embarrassingly, almost entirely toward a single goal. Getting into YCombinator. It was constantly in our minds. Ample encouragement followed months of work. On occasion, I could be found exclaiming my certainty to the universe. We’ll get in. We’ll make sure of it.

The letter arrived silently. I’m embarrassed to admit that my body read like a chapter on the stages of grief. Shock. My stomach churned as I turned inward. Admonishments ‘not to take rejection personally’ meekly confronted universal doubts. Egos struggled against a rethinking of everything. Hours of discussion lapsed. Plans of what to do were floated, accepted, rejected, forgotten. Night passed to sunrise before sleep. Denial. I woke up recalling a story of one rejection mailed out accidentally, to a startup later to succeed. Thoughts strayed from their success – all I could register was the possibility of a mix-up. Anger. I stewed. ‘It doesn’t matter what they think. I know where they’re coming from, I know what they must think of us. They’re wrong! And we don’t need them anyway.’

Finally, acceptance.

YCombinator has it’s own personality. Their opinions are their own. They have perspective and wisdom and the will to apply it. But they have no more dominion over truth than the rest of us, nor are they immune to the blindspots that all beings must endure, nor would they suggest otherwise. It’s taken much for me to fully understand this

For years I’ve contemplated PG’s word as philosophy. It has changed the way I think. As PG writes, heroes are those of whom you’d ask “what they’d do in the same situation,” as for years I’ve asked myself. He is a hero.

But PG isn’t my conception of him, no matter how close his words have touched me. YC isn’t what I know of it, no matter how right I’d judged it to fit. And we are not what any others, no matter how wise, can ever hope to judge us.

Once can’t come to truly know people by their writings alone. Yet in my imagination I felt I could. Upon rejection, it couldn’t feel like a group in Mountain View had overlooked our ideas in favor of ideas more compelling to them. Their judgment felt universal. It took hours to move past the looming question: ‘How are we deficient?

Our idea seemed nothing but wonderful. Yet after taking a few steps away, and looking back, there are perfectly valid reasons for skepticism. PG is even on record stating that building applications for people interested in local events is a ‘perennial tarpit’. This isn’t a YC idea, even if it was YC inspired.

It started with a question: could software help people connect in real life? Nothing we knew about was any good. Nobody sane would sift through lists of events online. Dating sites mostly sucked. Put people in the position of a judge, and romance cannot bloom. Chat rooms sucked. There are trolls and perverts everywhere. And the internet at large has no location. Rarely would you find someone nearby. Even more rarely could you expect to know someone by their online persona: my YC dilemma writ large. And we knew, or believed, or hoped. There must be a better way.

But how could they believe us? With neither users nor customers nor a finished product, we have no proof of what is perhaps best described as a social experiment. All we have is code, determination, and the belief in our hearts that we are on the right track.

That belief can only come from imagining our success as we do. But our imagination is not theirs. Without proof, others cannot take the same leaps of faith. It is so easy to take rejection as a challenge to ourselves. It is only a rejection of our rendition. We are unproven.

How were we deficient? Perhaps we don’t see the problems others have seen: traps laying in wait for us just beyond the bend. Or perhaps it is because our imagination, try though we might, could not be made to fit with our heroes, and we were so certain of our shared viewpoint that we were blind to this possibility.

Much attention is being paid to YCombinator clones of late, and there are murmurs of the movement forcast more springing up. When the first batch appeared, they pointed out that YC would eventually struggle with deal-flow. Plenty of good startups were being passed over, so why not fund those? Too little effort was invested in explaining why one might prefer them to YCombinator, dooming them to the appearance of funding YC rejects. Later attempts focussed on a different limitation: bright entrepreneurs must contend with, in addition to harrowing expense, culture shock, and commitment, an immigration system unwilling to receive those planning to work for themselves.

Seed funding is limited by a third resource. For proven teams one needn’t far sight to bet dollars on success. But to bet on the earliest of early startups, run by strangers, not friends, toiling without traction, with working, complete technology, takes more than guts. To back such a startup requires they imagine the success for themselves.

There is hope for new players in the global seed funding game. But none will succeed as YCombinator does, winning successes from strange ideas with unproven players, unless like YCombinator, they cultivate a personality of their own, and call upon their imagination to see as the bright minds applying dream.

The name problem has been with our band of hackers for a while. At least we were not alone: judging by the perennial popularity of the topic on Hacker News, it would seem to stump many.

On such matters, an appeal to a higher power is appropriate. My friends use a variety of divination techniques, such as flipping a coin, tarot, or peyote. I, however, found myself reading an infrequently referenced blog post by Paul Graham (an orphan of the collapse of infogami).

“…as happened with lofts, the features that initially repelled people, like rough concrete walls, have now become a badge of coolness. Weird names are now cool, if they’re the right kind of weird. Nothing could be less cool, at this point, than calling a startup “cool.com.” A company with a name like that could not have arisen organically. “Cool.com” smells of a media conglomerate trying to create a web spinoff.”

and

“My favorite recent startup name is probably Writely. It looks so natural that even though it isn’t a word, you feel it should be. Anyone thoughtful enough to come up with a name like that is probably going to have good software.”

Even ordinary people have an extraordinary ability to glark meaning for a word newly encountered. A word that feels natural enough to exist in speech (‘I’m feeling ever so writely’) is quite a goal to aim for. People are sure to remember that.

I threw together a ruby script to create domain names from some simple rules and then check whois. I multithreaded it for throughput. (Ruby threads are easily invoked but apparently the threading system is not so powerful.)

My friend Nick pointed out that our startup targets the young, urban, and restless. I tried the suffix ‘libre’, meaning freedom, and slapped letters from A to Z and roots meaning city in front. Didn’t quite work. At some point, I ended up with collibre.com (it’s still available if you want it), though my partner in crime Alex said it was too much like collaboration, or a library. Finally, he said, ‘what about the root -opolis’. We plugged it in.

Hopolis.com wasn’t taken. We liked it. (don’t go there yet. It is only parked.) This name has only grown on us since then.

Anyway, yesterday I overheard some business types arguing over names. Someone mentioned capital. I thought to myself “hot diggity, let’s impress ‘em and get funded.” Alas, they were merely business students plotting a revolution in granola distribution (direct to the customer! Take that middle man!) Still, my script came up with better names than they did. Who’d have known so many things ending in ‘anola’ would be untaken?

So tonight I was hacking on hopolis, and six hours in, I started to shift into filling out the YC application. Now it’s almost done; only the questions I want to mull on and really ace are left. The sun’s rising — what’s there to do? Procrastinate, obviously. What better way to procrastinate but to update my neglected blog and save hackers from naming catastrophe?

I decided to post it here, and below. (the link goes to the Google Code project).

# Scans DNS for cool names, put together with prefixes,
# affixes, and middles.
# Usage: ruby DNSScan.rb
#
# Edit the domain composing code as you will. This is
# perhaps best used to make up words, eg. writely.
# Do not pass untrusted data to this program, whois
# has buffer overflows, so this may be insecure.

pre = ['Fox', 'Web', 'Net', 'Thought', 'Read', 'Work', 'Surf', 'Browse']
mid = ['']
suf = ['Stream','Flow','Link','Line','Focus','Scope','Glide','Fly',
       'Flight','Path','Wing','Eye','First','View', 'Sight','Vision',
       'Hunt','Wind']

# A single whois scan
def scan(x)
  found = system('whois ' + x.to_s + ' 2>1 | grep -E \
          'No match for|connect: no route to host' - \
          1>/dev/null 2>/dev/null')
  puts (found ? 'Free: ' : 'Parked: ') + x
end

# constructs domains (Note: It would be great if ruby had
# an outer join, so I could do (pre*mid*suf).each{...} )

domains = []
pre.map{|i| mid.map{|j| suf.map{|k| domains << i+j+k + '.com' }}}

# Makes threads (avoids blocking)

thrNum = 40
thrArr = (0..thrNum).map{
       Thread.new{ scan(domains.pop) while domains.any?}
}

while (thrArr.any?{|ta| ta.status})
  Thread.pass
end

Note: There are web services like this. They’re great, but they don’t seem to work fast enough, and I personally would rather have the name construction rules in code.

Terry Tao’s recent post on a classic logical puzzle has seeded a bloom of activity in the nerdsphere. A friend of mine introduced it to me over mugs of steamed milk in the graduate college coffee house; I was telling him of recent work I’d been doing on the soundness of emergence of Nash equilibria, and where sufficient conditions arise in real life.¹ It was an enjoyable conversation, though I wasn’t afforded the luxury of working it out for myself. If you’d like to struggle through the problem on your own, read Terry’s post, and only return after the break when you think you’ve solved it. (why am I writing this? It started as a comment on news.ycombinator.com, and I couldn’t help myself)

This problem is subtle, and wording is important, so I’ve reproduced the statement from Terry’s blog (emphasis mine) [editor's note: Terry didn't intend for a particular subtlety, and so has reworded his main post. He says that it had an 'unexpectedly interesting subtlety in its formulation, but was not the puzzle I had actually intended to write'. He's posted the original here]:

There is an island upon which a tribe resides. The tribe consists of 1000 people, 100 of which are blue-eyed and 900 of which are brown-eyed. Yet, their religion forbids them to know their own eye color, or even to discuss the topic; thus, each resident can (and does) see the eye colors of all other residents, but has no way of discovering his or her own (there are no reflective surfaces). If a tribesperson does discover his or her own eye color, then their religion compels them to commit ritual suicide at noon the following day in the village square for all to witness. All the tribespeople are highly logical and highly devout, and they all know that each other is also highly logical and highly devout.

One day, a blue-eyed foreigner visits to the island and wins the complete trust of the tribe.

One evening, he addresses the entire tribe to thank them for their hospitality.

However, not knowing the customs, the foreigner makes the mistake of mentioning eye color in his address, remarking “how unusual it is to see another blue-eyed person like myself in this region of the world”.

What effect, if anything, does this faux pas have on the tribe?

I’ve italicized the statement hosting the subtlety. It is known to everyone that if any tribemember discovered his or her eye color, they would dutifully commit suicide at noon the following day. And yet. The statement cleverly leaves open whether every person knows that every other person is as logical, or as devout, or as committed to the duties of ritual sacrifice, as they are.

To jump the gun, the statement doesn’t explicitly state the logical nature and devoutness of all tribespeople as common knowledge.

Hopefully examples will deliver me from vagueness. Consider tribes with one, two, and three blue eyed people on islands Galileo, Hippasus, and Russell. ^2,3,4

On Galileo, after the gaff, the sole blue eyed unfortunate notices that none of his people share his tint of iris, and reconciles himself to a midday doom.

On Hippasus, one day later, our fated blue-eyed duo, fully aware that the other would have extinguished were he or she the only blue eyed resident, are forced to draw the conclusion that they are blue eyed, and so they disappear.

But on Russell, confusion sets in. Three blue eyes islanders know the two others. They know that each would commit suicide if they discovered their eyecolor, and they know that on the previous two days, no such thing happened. It would seem that the group would meet a bitter end. As the reasoning usually goes, each of the three, knowing the other two, would infer that the others continued survival could only mean the existence of other blue eyes on the island — theirs!

But wait. Let’s examine this more closely: take the blue eyed islanders, let’s call them, err, Alice, Bert, and Cecil. Alice sees Cecil and Bert; she knows that they as well as the rest of the tribe, and knows they’d indeed follow protocol if either discovered their eye color. What she doesn’t know for certain is whether Bert knows that Cecil would kill himself over eye pigment (even though he in fact would). Therefore, she doesn’t know for certain that a tribe with two blue eyed members would homogenize itself on the second day, she only knows that this would be true were she one of the members. She therefore cannot deduce that she has blue eyes, and the tribe’s taboo keeps information from spreading any further. Our trio is saved. ⁵

What would happen if all islanders knew that all islands followed the same rules (as is explained in the variant posted by xkcd)? Instead of confusion, Alice would be certain that Bert and Cecil wouldn’t have survived were they the only blue eyed people, and so she must be the last. They all would die on the third day. Indeed, any group of blue-eyed ⁶ islanders under the same conditions would all meet their fate the same way.

There are other variants, Anatoly brings up a very interesting metaproblem where proactive tribe’s members might decide to save the others through early self-sacrifice. The analysis of this, and other metaproblems, goes very deep. It will have to be the subject of another post.

In any case, I hope I’ve thoroughly explored the magic. Or at least spoiled it!

As Terry writes,

“The remarkable thing about the puzzle, to me, is how such a subtle and seemingly academic change in the knowledge environment eventually propagates to have a concrete and dramatic effect.”

Notes:

[6] A common misconception is that the brown eyed people would die too. But that’s not quite right since none of them are sure that their eyes aren’t actually green, grey, or magenta.

The following occurred to me on a run about two years ago:

It’s not given much press, but the the Halting Problem is intimately related to Gödel’s First Incompleteness Theorem. Indeed it produces it as a correllary. Historically, Gödel’s incompleteness results were proved by hacking arithmetic into a Turing complete system, and this is still how they’re explained today.

There’s a one-to-one bijection between computability of a function and provability of a statement. Hence, the short, and generally accessible proof that the Halting problem is not in general computable for an arbitrary input is also a proof of the ‘most important, surprising result in logic’, namely, that some results, which have may have a perfectly valid truth-value outside a system, cannot be proven within it. One only needs the notion of a computer to follow this line of thinking, which is, in essence, what Gödel did. But the Halting problem is much easier to grasp. I’ve had children understand it, though it does take some walking through!

The interesting thing about the Halting problem is that it’s unsolvable in full generality, independent of whatever special capabilities the system has available. To see this clearly, consider the proof.

Question: Does there exist a (halting) program H which, given any program P, figure out if it would halt, for any input I?

Assume there exists such a program H. Construct a program T as follows.

(Program P, Input I) => (Boolean Halts):
if H(P,I) is true run forever
otherwise halt

Now, call T on itself, with itself as an input. Our assumption presupposes that H always halts. If T would halt on input T, then T will run forever. And if T would run forever on input T, then T would halt. This is a contradiction, so no such program H would exist.

This becomes philosophically and theologically relevant because it doesn’t at all depend upon the form that H might take. Suppose one has funky laws of physics, or happens upon a ‘book of truths’. Or suppose one is some kind of omniscient god, able to just ‘know’ whether something halts or doesn’t. All of these can be wrapped up by saying they’re part of H, whatever H may be, and even if you assume that H¹ always exists and is well-defined for some set, you cannot also have H be well-defined for programs which include calls to H.

In short, an omniscient entity can’t exist. Not even a god. When speaking amongst nontechnical types this conclusion is usually met with some form of drama: praise or otherwise.

‘Omniscience’, as it happens, doesn’t survive close inspection.

Notes:

[1] Taking the existence of H as given, convention names it a ‘halting oracle’.

The central limit theorem states that if you have many small, independent, random variables, then their sum is distributed approximately as a bell curve. Strikingly, almost everything is made up of many small parts, and these parts don’t tend to influence each other very much.

So much of what can measure seems to fit a bell curve. This is why the normal distribution works. Because this assumption tends to work well, it is usually taken as a matter of course. Students are taught it, lecturers preach it, researchers apply it, and startlingly few stop to question it.

Suppose the variables are not small, or suppose they’re not independent. Suppose, under certain conditions, the value of one variable would seriously effect another. Suppose we’re talking about the buildup of snow on a mountain slope. Most of the time, snowflakes can gradually build, without significant effect. But once enough builds, you don’t find snowflakes resting calmly upon a drift. What you find is an avalanche.

The sum total of snowflake movement isn’t what we might expect. The snowflakes on the top used to be lightly packed by the new, gradually coming down. The snowflakes on the bottom used to just sit there. But they’re not just sitting there. They’re moving fast, and they’re moving down.

The central limit theorem doesn’t always hold! If you have a model where, most of the time, a change in one part doesn’t  effect another much, but some of the time it really does, then you can’t assume that your outcomes will follow a bell curve. Entirely different outcomes are possible.

Traditionally, when you have some measurement that’s far outside of normal, you call this an outlier. In much statistical analysis, these are ignored or thrown out (for example, even the most extraordinary measurements won’t effect a median). This is useful if you want to study the ordinary behavior of something. But sometimes, the information you gain from outliers is by far the most interesting. Some of the outliers are expected in normal distributions (bell curves). But some of them are outliers because the model doesn’t apply. Some outliers are avalanches.

We started from an iffy assumption. Not everything is made up of independent random variables. Parts effect each other. Sometimes it’s violent, like a chain reaction, or our avalanche. And sometimes, it’s magical. Life is an example of this. If the chemicals that made up our bodies didn’t bond so strongly to one another, our DNA would unwind, and you’d be a puddle.

Statisticians try to account for this. One example on the tips of our tongues lately: finance. Roughly speaking, the ‘beta’ used in Finance means the volatility of a stock with the linear correlations with the company’s existing portfolio factored out. Despite this, there seem to be ’six-sigma events’ happening all the time, things which, according to the theories, and ‘ordinary’ date, really shouldn’t be happening at all. ¹ What’s going on?

When small effects just add up simply, you can model them by what’s called a linear model (so called because if you add up small things along an XY graph, you’ll get a line). This doesn’t always work, and in fact, most interesting phenomena are non-linear.

Models are limited. They break down. And one can’t really account for every possibility. Financial markets can collapse due to dustbowls, and furthered by widespread investor panic. The destructive power of armies during world wars can be dwarfed by exhaustion and a powerful flu, though normally flu is beaten with chicken soup.² And planets full of ten story tall reptiles can be wiped out by meteorites. No number of small rocks would matter — usually they just ping off them. Bet they didn’t see that coming.

It isn’t too long before one fears running directly into Godel’s Incompleteness Theorems: checking all assumptions isn’t merely inconvenient, for most interesting problems, it’s impossible.³

The next time you hear about your monthly six, or seven, or eight sigma event, keep this in mind. Outliers are where the model breaks down. They happen more often than standard models would expect, and they often point out problems in the understanding of the system. If they start happening all the time, start mistrusting your statisticians. And your wall street.⁴

One can show the recursiveness of the model verification problem. Suppose one says to you: drug x worked better than drug y, with a 95% confidence. You might reply: what’s the confidence of that confidence? Somewhere along the line, they’ll have to say 100, and then they need to prove consistency. One can show that there are theories with unknown external variables and relationships (as are dealt with in statistics) aren’t even formally recursively enumerable,⁵ but even if they were, no such theory could contain a statement of its own consistency.

Back on planet earth, is there anything we should be worried about? Some who talk about the climate claim that, since basically the temperature of the earth just goes up and down naturally, it’s not really anything to worry about. But outliers can mark where models break down. Where systems change. This is a chart of temperature over the past few centuries. You may notice, to the right, an outlier. Maybe it’s telling us something.⁶

Further Reading:

– The Psychology of Human Misjudgment – Charlie Munger

“Now let’s talk about efficient market theory, a wonderful economic doctrine that had a long vogue in spite of the experience of Berkshire Hathaway. In fact o­ne of the economists who won — he shared a Nobel Prize — and as he looked at Berkshire Hathaway year after year, which people would throw in his face as saying maybe the market isn’t quite as efficient as you think, he said, “Well, it’s a two-sigma event.” And then he said we were a three-sigma event. And then he said we were a four-sigma event. And he finally got up to six sigmas — better to add a sigma than change a theory, just because the evidence comes in differently. [Laughter] And, of course, when this share of a Nobel Prize went into money management himself, he sank like a stone.”

– Wikipedia on the Spanish Flu.

– For the curious, a quick way to show this is by adding a ‘randomness’ oracle to a turing machine, and showing that you still can’t solve the halting problem.

– Wikipedia on the 2007 subprime mortgage financial crisis.

– As wikipedia states, a recursively enumerable language is a formal language for which there exists a Turing machine (or other computable function) which will enumerate all valid strings of the language.

– (update) For further musings on uncertainty and randomness, a terrific book is Nassim Taleb’s The Black Swan. A related essay is available here. Also intriguing is the working paper “Extreme Sample Selection Bias: Conditions that Cause the Correlation Between Two Variables to Switch Signs” by Tim Groseclose

Konstantin Lopyrev writes (on the TopCoder forums):

“Everyone always discusses algorithms and such on this forum. However, there is one trait that is perhaps equally important as knowledge of algorithms – good problem solving strategy. By that I mean not jumping into solutions too quickly, thinking through everything before writing any code and such. Can anyone share what they’ve figured out that helps them? I’ve noticed that I have several flaws when I solve problems in general. I am too quick to start writing code. Also, I don’t think through my algorithms thoroughly. Also, there are many other things. Does anyone have similar flaws that they’ve already passed. If you have, please share your strategies for getting better at problem solving in general. Anything would be helpful, since I have the TCHS tournament coming up and I want to go home with some sort of prize.”

I wrote back, on May 10th ‘07:

One thing I want to start doing is to make estimates of the implementation time of parts of my solution, how difficult each will be to code correctly/debug, and how sure I am that my method will work in the first place.

I’ve gotten very good at making estimates of time and space constraints. Now they are second nature. But knowing how best to solve a problem practically is still another area which I need to improve.

For example, for the last hard problem on TopCoder (circa May 9th, sorry, you’ll have to register to see it). I broke the problem down. Six steps toward victory! Five of those steps were really quite simple, though a few bugs crept in. The sixth was brutal — I could solve the problem by implementing a complicated comparison function object which recursively called itself on sorted sets of children. It was really crazy. To get it all working properly took nearly eight hours after the match.

Each bug, if not isolated, tends to increase debugging time by something like the square of the number of interacting components: the rate at which the number of dependencies would grow. Perhaps it would be simpler to either independently verify each component (by having pre-written tested functions or methods), or to organize the program, inasmuch as possible, as a linear pipeline or other simple acyclic graph. In fact, one might take a few seconds just to sketch the program idea out — how strongly coupled is it? What can I be sure of? How much of a pain will it be to debug? Can I make it less so? If not, can I anticipate debugging requirements and build in information gathering tools?

Furthermore, I was lured by that fact that, conceptually, ‘most’ of the problem could be coded simply. This blinded me to the fact that the code that was doing all of the work was almost completely unsupported by the auxiliary framework I had envisioned, and that the whole thing was a tightly entangled mess. Maybe the other coders can handle it. But I think it’s mostly because they know ways to keep things from failing — keep bugs isolated and visible enough to squash.

What I really needed to do was find a data structure or recurrence that was simple enough to verify piecemeal. I cannot be trusted to write bugless code. In fact, I should guard against it by breaking things down. This has the added benefit of making more clear what I need to do. I would need to remember less. Increasing my effective working memory has an impressively super-linear effect on my problem solving ability, how fast I can learn things, and also how much fun I’m having. I’ve come to realize this only recently, while studying for PhD qualifiers using index cards, scrap paper, two levels of books, one rough, one ‘canonical’, a wiki system of notes and a flashcard engine we made for it.

The last thing — and I think this is really more rudimentary, but I’ve yet to learn my lesson, know exactly what your variables are. In all of my experience this is the worst problem I have. Again and again I find myself with a few minutes to spare before the end of the coding phase (for readers unfamiliar with TopCoder, to get any points you need to have a solution that is (almost) exactly correct, submitted before the 1:15hr deadline). There is some minor error which I’ve detected. The tests give one more in some cases, one less in others, maybe it fails on an edge case. The tinkerer in me asks ‘hey! what happens if I turn this knob? Cross that t? Increment that i?’. So, I try it. Does it ever work? No. Not one damned time.

It’s voodoo programming, sure, and I make fun of it when I see others do it, but there’s just something — as that clock is ticking down, for some reason it feels like the competition is not about making sense of problems but just trying loads of things really really fast. And when I’ve defined an ‘r’ or an ‘x’ or a ‘q’ as some kind nifty function, where I only know the ‘approximate’ relationship with what I’m doing (up to a least significant bit, or maybe it’s part of the state of my program’s implicit automata that I used to speed up coding), when I’ve done this and the program nearly works except for this minor case or that minor case, it takes me nearly forever to disentangle what something is from what it is supposed to be. Even though I feel like I’m so close, when the pressure is on I just can’t handle it. It’s just way too complicated to do in my head when I keep having to watch the clock.

——

One advantage of TopCoder is that you’ll be constantly stretching your abilities. You’ll be solving many more problems than you otherwise might, and because of the time pressure you’ll learn faster, and it will stick better. I came across apropos comments by Philip Greenspun

1. A person won’t become proficient at something until he or she has done it many times. In other words., if you want someone to be really good at building a software system, he or she will have to have built 10 or more systems of that type.
2. A person won’t retain proficiency at a task unless he or she has at one time learned to perform that task very rapidly. Learning research demonstrates that the skills of people who become accurate but not fast deteriorate much sooner than the skills of people who become both accurate and fast.

and

Whatever the training task, the pace must be ruthlessly brisk. The learner should be expected to build at the same pace as an experienced developer. The difference between the learner and the wizard is that you expect the learner to make a lot of mistakes. The system as built may be awkward or not handle error cases properly. That’s okay. Training research shows that if you get speed now you can get quality later. But if you don’t get speed you will never get quality in the long run. We practice this technique in 6.916, Software Engineering for Web Applications, our course at MIT. Each student builds five database-backed Web applications during the 13-week semester. The first few that they build, during the course of the problem sets, are not necessarily elegant or optimal, but by the end of the semester they’ve become remarkably proficient, especially when you consider that each student is taking three or four other classes.

This rings remarkably true. Periods where I’ve needed to learn a skill in a short period of time have inevitably resulted in me learning them well. TopCoder is a terrific example. So was the high school math I needed to catch up on when I found myself in the university math department at 12. Cramming seems to have gotten a bad rap. I learn best intensely.

It’s not overarching. One of the most striking counterexamples is my friend Joel. When I met him in first year, he had already read code complete, and was working on his own games. He painstakingly engineered every assignment to be bug free. Extensible. Readable. Coherent. Most of us rationalised such behavior as making sense only because later assignments built upon the previous.

Joel lost the floppy he kept his work on. Most of us would have been upset. He was unphased. He did all his engineering for practice, and was happy to be able to do it again.

On TopCoder, he isn’t always the fastest. But he is the only competitor I know with a perfect accuracy rating. I don’t believe this skill was developed under time pressure. But I do believe that it entailed long hours, and sincere commitment to build something well. That is quite rare.

——

It’s been a while since I’ve done any of these contests: things keep getting in the way. Making rent, and taking care of other basic necessities has impeded my time for fun. Hopefully my skill wouldn’t have deteriorated in the interim. I actually have hope that I’ve unlearned more of the bad habits than my good. :-)

Further Reading

Misof’s description of the solution to the “RussianCheckers” TopCoder problem highlights a crucial part of the problem solving process: understanding how to reliably, quickly, and accurately implement a particular solution. The problem statement gives all the information one needs to solve the problem, but the task of implementation is so hard that only two competitors manage. Petr, the winner of the match, has a solution that’s a paragon of readability.

One morning around the graduate college dining hall, there was a gathering of physicists, finance students, and economists. The physicists are always quite amazed by those people who decide to forgo the life of the ivory tower, and choose to strike out into the real world, and so could not be kept from asking what the economists actually did. Furthermore, we could not be kept from wondering aloud what type of mathematical models they built and polished, and whether any of them had a physical interpretation.

One of the economists scratched his head, drew a sip of black coffee from his porcelain cup, and mumbled something about how a large proportion of the physics department of Harvard University was hired by a trading company, with the lure of riches beyond the pale of the meager imaginings of the physicists (“you mean I can afford a house?!”).

But what did they do? Why them? What did the trading company see in them? I offered, from my experience in TopCoder, that trading companies appear to be mostly interested in hiring people who were very smart, and that maybe physicists at Harvard fit the bill.

“Uhmmm, no, not quite. I think it was something along the lines of specific expertise… something about, quantum… quantum field theory, maybe?”

I closed my eyes. Quantum field theory — gads! They’re going to use that for economics? Renormalization, Feynman diagrams, antimatter. I was aghast, finance would certainly be fun this way, but how on earth would anyone use it?

As my incredulity faded, it dawned on me. “Aha!” I exclaimed, “I think I know what they do! In Quantum Field Theory each normal mode oscillation of the field is interpreted as a particle, and all of these oscillations, coupled together, form the field. But we can just as well interpret an economic ‘price function’ as a field, and interpret each individual market player as a linear oscillator! The mathematics is exactly the same, so we can use all of the methods of QFT! It’s brilliant!”

I asked my friend across the table: “You must tell me. Did it actually work?”. He shook his head. “I don’t know. All of the results would be lost to the company. Perhaps if the stock rose inordinately it might say something, but how or why it happened would always be a mystery.”

Dejectedly, I pondered this. For perhaps the ex-physicists are happy, with their six-figure salaries and shiny urban lifestyles. But if I was them, well, I doubt I’d be able to handle keeping my application secret. And so I discovered yet another reason why I can’t live in the real world. I’m an intellectual show-off.

Michael Nielsen writes in to point out that Jim Simons, of Chern-Simon’s theory, is in fact the founder of Renaissance Technologies, a prominent hedge fund.

Tonight it’s winter in Berkeley. 53 degrees and raining, and outdoors, warmed by a heat-lamp, sheltered by an awning. I draw spiced apple cider through my lips. Classical music plays. An earbudded minority vote silently with their ears. Old men watch hooded students roll down the hills towards Telegraph Ave, Berkeley’s epicenter of hippiedom. Moist, newspapers ink the hands of activists, busily plotting the victories in the years long struggle to ’save the oaks’. A young man lids a drink and smiles at me. Separated by glass, headphones, and 12 feet, I smile back. We wave.

There’s something magical about this place.

I don’t know anyone here. To arrive I flew four thousand miles from my place of growth. This place isn’t home. Yet there are few places that attract me so strongly. Modern life has been made private. And in doing so, life’s become a little lonely.

Builders of great cities have long understood that life would, but for misfortune, consist of more than work and one’s home. The vibrancy, energy, and community grown in what are sometimes called ‘third places’ played part in much of the world’s social, political and intellectual revolutions. The roles that the Roman forae, French salons, and English learned societies played in scholarship has been tremendous, as has been the influence of American chautauquas, worker’s taverns, and artist’s ghettos in social and political spheres. These public, accessible, talkative, comfortable playful places are magnets for folks of many stripes. Creativity can thrive there. Unconstrained by work’s implied unity of purpose, and decoupled from the tight bonds around one’s family and home, third places give marginal people, ideas, and voices room to grow, people to hear them, perspectives to challenge them, and food to help keep the conversation going.

Yet in much of the world, third spaces are dying, or being replaced by poor substitutes. Nearly half the USA lives in a suburb. Some try to define suburbs with density requirements, or as satellites of more major cities. But that doesn’t seem to capture the idea: almost all of San Jose is a suburb. Los Angeles, despite its skyscrapers, feels desolate in the same way, albeit in a different intensity as Jacksonville, Florida. We’ve heard it all before suburbs have too much pavement, too much repetition, too many box stores, too many cars and vans and soccer moms and private schools. But nothing so much makes a suburb if not a lack of third places. It may be the defining characteristic. There’s nowhere to go. There’s nothing to do.

So people fight back.

They watch TV; sitcoms and reality TV, predominantly: short, canned glimpses into lives where people are living. The same people, born of similar neighborhoods decades later, would watch YouTube over Friends, prefer to text instead of talk, and paste pictures in MySpace instead of a scrapbook. They’d hangout in different places, most marginal, most online. Yet despite these differences, the core behavior remains the same. People reach out subconsciously and compulsively to the world. They have a social itch, and nothing in their known universe will let them can scratch it the old fashioned way. Times have changed. Things are just too far away. Instead, they substitute.

Centuries ago, Shakespeare’s melanchoic Jaques proclaimed:

“All the world’s a stage,
And all the men and women merely players;
They have their exits and their entrances;
And one man in his time plays many parts…”

If the world can be a stage, then the web is valued stagehand. Millions now live essentially online. Working means sitting behind a computer. Home means sitting behind a computer. Hanging out means sitting behind a computer. The web has made possible access to millions more places than urban planners had dreamed of. It deeply affects our culture. Yet, in many ways, it comes up short. The web doesn’t serve apple cider, nor can it provide heat lamps for the evening chill. Eyes I catch won’t smile back at me. I won’t be able to catch eyes at all. To how many does it occur that when reading the New York Times, or Reddit, or YouTube, hundreds of eyes are following the same pages they are? Does it occur to them that comments on popular postings represent the opinions of only an assertive minority? Does it occur to the millions of highschoolers using Wikipedia that they can change a page at will? That thousands of others have requested the same references for the same civil war paper as they? So much interactivity of the physical world has been lost in the transition to the web. How much can we get back?

——

PS: To all you aspiring entrepreneurs, ThirdPlace.com appears to be parked. The tagline?

ThirdPlace.com

The Leading Competition Site on the Net

——

Further reading:

rokhayakebe describes online places as cities in a comment on Paul Graham’s Cities and Ambition

“I think the biggest shift in deciding where to live is happening “online” rather than “offline”. I am an online nomad.

I never lived and never will live in MySpace. I do not like the MySpacian’s message (Hey let’s try to see who has more friends and hookups).

I sometimes spend time at Facebook. I lived there for a little while until I realized I am not so much into keeping in touch and I had no friends in the few hours I spent in college. When they open their borders, that’s when I found that I do not like the Facebookies message (You should throw more pies and send more kisses)

I vacationed at Twitter, but it is not really my cup of tea. I still don’t get their message (Life is a popularity contest).

So Where do I live? Well, I live mostly in HN. Although I sometimes get into arguments with the habitants, I have yet to find another city that beats the intelligence, vibe, energy and support I witness here. I take a daily ride to Techcrunch City and NYT, but I make sure I come back home to HN and mingle with the people who live here.”