How Law Shapes the Business Landscape, and a Patent Puzzle

•February 11, 2010 • 5 Comments

If there’s one thing I’ve been surprised by while trying to start startups, it’s the extent to which the business landscape is shaped by law.

Skin Cancer

One of my first serious startup business efforts involved skin cancer: melanoma, specifically, by far the most malignant and dangerous type. It turns out that one can really do a pretty good job in terms of detecting melanoma in the early stages, when it’s still relatively easy to treat. This is in part because it is a cancer on the surface of the skin, and typically the cancerous, rapidly multiplying melanocytes produce an excess of melanin in patterns of characteristic irregularity. In other words, one can spot melanoma in ugly moles. If (a) one knows what to look for and (b) actually does look, in the vast majority of cases you can catch the disease before it spreads, and thus, save a life. The World Health Organization pegs melanoma deaths at 48,000 per year.1

The ABCD's of Melanoma

The ABCD's of Melanoma. Part of the ABCDs for detection of melanoma. On the left side from top to bottom: melanomas showing (A) Asymmetry, (B) a border that is uneven, ragged, or notched, (C) coloring of different shades of brown, black, or tan and (D) diameter that had changed in size. The normal moles on the right side do not have abnormal characteristics (no asymmetry, even border, even color, no change in diametry).

The chain breaks in both places. There are papers2 that estimate the sensitivity of physicians to melanomas at 86%, which is not bad, if they happen to look, but not great either. The big problem is that it’s by no means certain that the doctor will look: most melanomas are still found first by the patient. That is, if they’re found at all before it’s too late.

A friend of mine refused to see a dermatologist, despite her friends urgings. Once she finally did, the cancer had grown out of control. She did survive, but she had to suffer through an extremely painful excision followed by reconstructive surgery followed by a recovery period nearly a year long. Prevention and early detection are the key weapons against disease.

This could have been prevented. There’s a heck of a lot you can learn from a photograph, even one taken from a cellphone. From that photograph, a trained individual can determine whether the mole is a cause for concern — whether you’re really just fine or whether one should be examined more closely by a specialist. There are even papers showing how an algorithm can be made to be as accurate as physicians in diagnosing the cancer.3

It seemed like a great hack. Within a week or so, I rallied some friends of mine, threw together an application to Y Combinator, and tried to put a business together.

Whatever the technical challenges were, I was unprepared for the legal and social challenges I was to face. The partners at Y Combinator did like the idea on some level, but declined to fund us. In Paul Graham’s words

“The trouble with the melanoma detection idea is that you’d spend most of your time dealing with legal and regulatory crap. That sort of work doesn’t really take advantage of your skills.”

There were other problems too (we were rather cavalier about the quality of images we’d get in practice from widely varying cameras, lenses and lighting conditions), but in the end it was the specter of law and regulation that cowed us. I still maintain that we could build this, and while perhaps it might not have been the absolute best focus for my efforts or launching point for my career, I still think we could have done a lot of good.4

What concerns me most isn’t that we couldn’t get the melanoma detection idea to take off. It’s that anyone trying to work in the field of medicine is going to have to slog through legal and regulatory crap. There’s a lot that can be done with simple little pieces of software and hardware to help people look after their health. Detection is the first line of defense. But as long as we persist in the current regulatory environment, where detection aids must disclaim whatever they imply or slog through the same lengthy and expensive FDA approval process regardless of their capacity for harm, entrepreneurs, engineers, doctors and scientists are going to be limited in the good that they can do, as well as the bad. Lesson one.

The Electrical Grid

It doesn’t take too long before the familiar seems sane, and what we grow up with seems to have always been, bequeathed to us by ancestors blessed with the virtues of invention, vigor and foresight. I remember, as a kindergardener, when my mother took me to the local power plant, chimneys three hundred feet tall, how awed I was by the sights and sounds — the enormous burners and boilers, steam blasting through turbines powering vast generators, electrons pumping through transformers in gray substations of metal ringed insulators, transformers, cylinders, a geometrical landscape adorned with fins, cathedrals of wire, lines buzzing. Deep sounds metal clangs reverberated through the boiler room, amid hot steel, as my tiny feet traced halting paths along innumerable iron grates. Five years old, seeing this, I imagined the builders of these machines surely must have known what they were doing. Promethean gods walked the earth: behold! This was their handiwork.5

It shocked me to find out how the electrical grid really worked.

The grid, as it exists today, has almost no capacity to store energy. This means that at every instant in time, the amount of energy going on to the grid must almost exactly equal the amount of energy coming off. Otherwise, the grid rapidly fails in its operation, rising too high or too low in voltage, or shifting frequency, or succumbing to noise, spelling disaster to everything dependent upon a smooth, steady stream of power for its operation. That is to say, nearly every device of the modern world.

And while we have some provisions to redirect power on the grid (primarily, giant, motor driven switches of a kind not unlike those which control your household lightbulbs), our ability to throttle the amount of power we put onto the grid is slow and expensive. Gas turbines are the power plants that can be controlled most readily, however their power must be throttled over a period of nearly 15 minutes. When they do this, however, they operate away from their most efficient operating point, meaning that lots of fuel is burned, power is expensive, and more CO2 goes out the stack into the air. Hydropower stations can be throttled nearly as readily as gas turbines, but they can operate in a wider range before they lose most of their efficiency. Coal, petroleum, and nuclear plants, on the other hand require periods on the order of days to speed up and slow down — it takes about that long for them to relax to a thermal equilibrium, keeping thermal stresses within safe bounds.

However, the electrical demand is quite unpredictable, and it can vary significantly in a period much shorter than the 15 minutes it takes for our fastest power plants to significantly alter their power output. So to respond in faster timescales, power plants must run as ’spinning reserves.’ This is where fuel is burning and the turbine is spinning, constantly, in expectation of a fluctuation. There are now massive power plants, burning fuel this very second, for no purpose other than to smooth out the rapid fluctuations on the grid. I hope I don’t need to point out how crazy this is.

What’s more, electrical supply from renewables is very unpredictable, especially whatever’s coming from distributed power. Wind power is the least expensive (in fact, according to Prof. Marc Jacobson of Stanford University, considerably less expensive than coal power!)6, but in terms of supplying controllable, usable power, wind is terrible! Wind power varies rapidly and widely, is extremely difficult to predict, is quite correlated over large regions, and, what’s probably the worst, wind power comes mostly in the dead of night, when nobody needs it.

See the sudden drop? That's why we need energy storage if wind is to be a major part of our grid.

The entire world has wired itself up with an electrical grid that is fundamentally insane, and as we plug in renewables to solve the other problem with the electrical grid, controlling it is only getting harder.

We need a way to store energy. The current state-of-the-art7 is pumping water up a hill to store energy (essentially using a hydropower plant backwards), and letting it run down to retrieve it. This works well and is relatively inexpensive if you have the ideal geography, but those plant sites are mostly taken up: from now on pumped hydro energy storage is either going to be a lot more expensive or a lot more exotic (current plans call for giant undersea bladders, or the use of enormous underground aquifers).

Pumped Hydro Energy Storage

Pumped Hydro Energy Storage

There are a number of other energy storage technologies, including electrochemical batteries, flywheels, and compressed air. There are good people working on every one of these approaches, and from our perspective, technically, there’s a good chance that at least one of us will find a solution allowing for energy storage so efficient and inexpensive that renewables, especially wind power, are economically competitive on the world scale, and on an unsubsidized basis.

However, among the many obstacles we find along the way appears the government. Despite their problems, technologies now exist that can both regulate the grid on short time scales (that is, dampening the second to second and minute to minute fluctuations) and, over a longer period, buy unneeded power in bulk at low prices and sell it when needed at high prices. If, on a single device, you can only provide the short or long term service, the economics don’t make sense. Luckily, it is technically feasible to do both simultaneously, on the same device, and thus the economics for the current technology can work out, in at some cases. The electrical grid can be buffered by energy storage, and utilities could make money doing it, to boot! How sensible.

Unfortunately, it’s illegal!

Let me explain.

Transmission assets, like roads, railways, telephone networks and the electrical grid, have been shown to naturally tend towards a monopoly. In the case of the electrical grid, after a number of political fights the following deal was struck: the government would grant the electrical grid transmission monopoly to a single entity (in some cases more specifically the last few legs leading from the power plants to your home) in exchange for certain powers. In particular, the utility could, using its monopoly pricing power, impose practically any price on the unwitting public, which would pay just the same — they have almost no choice and electricity demand is notoriously inelastic. However, it would be restrained to charge a price only up to a certain regulated return on their capital, and no more. The regulatory commission has other powers, such as being able to define which sorts of investments and projects can proceed, and it places on the utility certain contractual demands to supply power.

Therein lies the first problem. In effect, were one selling power equipment for the grid, one’s real customer is not the utility, but the regulatory commission. It’s in the utility’s interest to buy the most capital intensive equipment that the regulatory commission will allow, and to incur the greatest expenses. After that, it may jack up prices such that it achieves its regulated rate of return, but now, due to its greater expense, it is now earning a greater profit as a greater entity for more investor dollars.

I never imagined such a perverse set of incentives. There are only three reasons for which utilities regulated in this way will look beyond the most expensive corner of the status quo. Firstly, in order to reduce risk on their capital investments, secondly, to explore the use of alternative technologies and broaden their strategic options, and finally, out of the goodness of their hearts. They’ve actually done a lot on that last count; that renewable energy is deployed at all seems to me shining evidence of a desire to do good. But utilities are conservative organizations, they’ve already figured out how to run their capital investments with very low risks (lower than most new technologies could possibly manage) and dollars for technology exploration are in short supply.

So, there’s the first shocker. Utilities with a regulated rate of return (I think this is most of them, though I’d love a better quantification for this) have essentially no incentive to save money, and hence no incentive to try out new money saving technology. If we’re to sell to anyone we have to sell to the regulatory boards, who are even more conservative. Oy.

Secondly, as the government is fond of doing, parts of the electrical utility business have been broken up, and they are not even allowed to talk with one another and share information, much less operate with the same hardware. The transmission department at PG&E is kept sealed off from the generation department at PG&E. Unfortunately, despite the fact that it is ludicrous, that it is patently insane to operate an electrical grid without energy storage (just as it’s insane to operate a network with no buffer, a computer with no cache or memory), there is no energy storage department at PG&E, or at least it doesn’t fit into the existing regulatory structure. There are no requirements for a certain amount of energy storage, and even the things that you could do to save money for the utility are unattractive for the reasons described above, and even those are unlikely with current technology to be good investments because they are both risky and it is illegal to operate the same device for both the generation department (people are saying now that storing cheap energy at night and selling it at a higher price during the day is called ‘generation’ for some reason) and the transmission department (regulating short term fluctuations in the grid and storing energy to ease congestion is called ‘transmission’ for some reason). So, a sensible and practically necessary thing (energy storage) is not allowed to be run in a manner which with existing technology which could save the public and the utility money and could enable the economic competitiveness of renewables. But that doesn’t matter because utilities aren’t trying to save money anyway.

Wow.

A couple of these problems can be avoided in the new, deregulated, ‘ancillary services’ markets. These deregulated markets, humorously enough, are markets for regulation — utilities and electric co-operatives pay a premium for MW scale power slices, so as long as you’re in the right spot and can deliver power on extremely short notice (people are pushing this as short as 4 seconds) you can stand to make a fair bit of money on a deregulated market selling to utilities. There are a few companies (Beacon Power is a prominent one) that try to make systems to do this now: in addition to the technology they’ve developed they’ve put in an incredible amount of political effort to get utilities to open up and deregulate their (*snicker*) regulation markets.

Unfortunately, the economics are still somewhat marginal. Current energy storage technologies are really expensive compared with the wasteful throttling of fossil fuel power plants. Better technologies (we’re working on one) might be able to make a really substantial profit. And if you can do energy arbitrage (buying power low, selling it high), or do something else that’s useful with your energy storage device, you can make even more money. Unfortunately, as above, it’s illegal to sell regulation services and do energy arbitrage on the grid — you have to do something else. (If any investors are reading this, don’t worry, we’ve got a few tricks up our sleeves!)

Someday, by our actions and with the help of others, enough of these problems will be solved so that the electrical grid can be buffered and made sane, that the public will save money by using energy storage, and renewable energy will be free to take over the world.

Someday.

Until then, we’ve got a lot of work to do. Here I thought we were working on a complete, technical solution to solve a massive problem in the world. A straight shot. And what do we find? Legal and regulatory crap, my nemesis, as defined by Paul Graham.

Lesson two.

So it goes.

A Patent Puzzle

Inventing things is what I do. It’s an inseparable part of who I am. I can’t get three chapters into a book before I have an idea for a new technology, product, startup, or hack. Parties are brainstorming sessions. I pester romantic partners as we fall asleep.

Some of my ideas are good. Some, not so good. Those that pass all my filters probably each have a decent shot at solving a major problem in a way fundamentally better than what’s come before with an approach that’s within my eventual capability to execute. Maybe it’s a little better than 1 in 3. I can’t really know until I try. But because I have so many ideas, and possibly because of practice, they number in the hundreds. If they were they only idea I had, I could really see myself giving them a go. And succeeding. I am not at all unique in this.

One thing that helps tremendously is to be able to talk about one’s ideas. Maybe even bring another person in to help work on bringing them to life. Oftentimes one’s ideas are too much for one to handle at some time, or maybe one is missing some crucial missing piece that needs just the right person or the right piece of knowledge.

If good ideas are too plentiful to fully utilize by oneself, if the difficult part of profiting from a good idea is bringing them to life, and if being open about them aids in this process, then ideas should be free.

Or, at least, it should not make one unduly vulnerable to reveal one’s ideas. One should at least be free to let one’s ideas free.

My friend, let me tell you about the patent system.

If I hold a patent on something (a widget), it does not allow me to build, license or sell the patented widget. It merely allows me to prevent others from building, licensing, or selling that widget (or anything else that falls under the widget classes and widget constituents for patented widget uses under the widget claims).

If my widget involves some device (a sprocket) that someone else (Eve) has patented, Eve can prevent me from building, licensing, or selling my widget unless I purchase a license for Eve’s sprocket.

If my something absolutely requires some crummy little item (a doohickey) that I forgot to mention or didn’t quite yet invent in my papers, my blog posts, my patent applications or my brochures, the sum total of my disclosure to the public domain and the patent office, and Eve notices this, she can patent said doohickey and the rational use of her doohickey and extort an outrageous price from me before I can sell any widgets at all.

Herein lies the problem.

Race to the Sea!

The Race to the Sea of the Second World World, 1914.

The Race to the Sea of the First World War, 1914. While the war was still mobile, both sides attempted to find an open flank. This began a race that ended when it reached the sea. Before these lines were drawn, armies were mobile. Afterward both sides became mired in grueling siege trench warfare from nearly unassailable positions. An analogy with patent strategy is apt: intellectual property is territory, and you do not want to be surrounded.

If you mention almost anything specific about what you’re doing and how you’re doing it, then you’re making yourself vulnerable. Eve can now take a look at what you’ve done, run through possible improvements, and possible requirements that you either have not claimed or disclosed or have not invented yet, and patent the ones that will hurt the most. Once you’ve given away your position, she can surround you, choking you off from the possibility of profiting from your work.

The important patent on the sewing machine was the sewing pin. With a tiny little hoop through the head. It is easy to forget the little things.

In the current legal climate, there are three options.

1. Stay hidden indefinitely. Protect everything that you’ve got by trade secret. If you personally (with or without corporate backing) expect to be able to bring your ideas to their full potential, this might be best for both you and the world.

2. Race to the sea. Patent your work — stake your ground, and reveal your position. Once your first patents are filed, you have to, urgently and carefully, check to make sure you’ve thought everything through. Are the relevant uses and markets patented? Are all of the incorporated or required devices available as commodities, or public domain, or inexpensively licensable, or do you own those patents yourself? Are you absolutely sure that you haven’t left anything critical out — that there’s nothing that you still need to invent that someone else might patent first? Are substantial improvements and variants of your design included in the first patent? There is a relatively short delay (on the order of a year) between when you file a patent and it appears for publication. If any of the vulnerabilities describe above exist, you’ve got until the publication date to protect yourself before Eve can make her attack. The clock is ticking.

3. Use your judgment, talk and write as freely as you desire, and cross your fingers. Hope that no one patents around you, and no one litigates.

I personally would love to be able to open source many of my patentable ideas. Engines. Desalination plants. Solar collectors. Refrigerators. Waste water mining. Wind turbines. Boats. Planes. A crowd beacon. Medical devices. Heat lamps. Translucent coatings. Metallurgical processes. Bioreactors. You name it. But the legal climate does not make it very safe for me to do so. If I write about them too freely, Eve might patent around me — now neither I nor anyone else can pursue my idea without encumbrance. If I protect myself by racing to the sea, I need to engage in an extraordinary investment of time, energy, and money on an idea that I probably crystallized in three seconds. It’s clear that even in the best of circumstances I’d be rate limited, but for almost all practical purposes this would become an untenable demand on my time (unless I have the resources of Nathan Myhrvold at my disposal.) Finally, I can protect myself by keeping my ideas trade secrets. Unless they are one of the few which I actually find the time and resources to seriously work on, this accomplishes exactly nothing. There is no pile of precious ideas I keep to stroke at night. I do not hoard them.

Some members of the free software community have been brainstorming ideas which have some relevance to tackling this problem. They include the Patent Commons and retaliation provisions in the GPL v3.8 Unfortunately they really do not provide full protection. Patents can only prevent people from building or selling things. They are weapons. Patent trolls, on the other hand, don’t build or sell anything. There is nothing to retaliate with, nothing to counter-sue, there is no center, no target. Patent trolls occupy an ecological niche in the legal landscape equivalent to cell based terrorist organizations in today’s political world. A sufficiently determined and evil person could extract billions of dollars from the world and destroy billions of dollars in wealth via long and painful lawsuits, simply by surrounding the patents and inventions that the world now relies upon. Perhaps at this time our only real protection is how grueling and soul destroying that path to fortune is versus the creation of wealth.

The patent system is supposed to encourage innovation. But I think what it’s done is cause everyone to overvalue ideas, and that has shifted the balance of power away from those who want to turn their ideas into something concrete and towards, marginally, those who have the ideas in the first place, but much more worrisomely, those who wish to wage war with them.

I’d love a discussion on how to fix this system. Specifically, not what to fix, but how: what specific actions can we take to make a difference? Quite apart from this, what should I do? Suppose I want nothing more than to do good for the world, to let my ideas, if they so deserve, blossom fully into life. Just talk about them and hope there’s no lawsuit? I do not want to be thinking about this!

Lesson three.

More legal and regulatory crap. Maybe I shouldn’t be so surprised: without law, there’s no formal property, no real capitalism, pretty much just anarchy, really. Of course it undergirds business. But what’s strange is that it’s constraining my thought. I just want to talk to people about my ideas, help the world, and be good. This shouldn’t be so hard!

So it goes.

Post Script: Trevor Blackwell of AnyBots and Y Combinator gives a thoughtful reply on Hacker News as to how the regulations that come into being are so malformed.

Big companies are smarter than gov’t regulators, they understand their business better, and they have a longer time horizon. So when the government comes around to regulate them, they think “OK, how can we turn this into a huge barrier to entry for new competitors?” They have large lobbyist and strategy budgets. They generally win.

The most misregulated industries in the US are energy, medical, and transportation. So there are lots of glaring inefficiencies, but they are there for a reason. Technologists assume the reason is stupidity and that clever inventions can fix things. Frustration ensues.

References

1 – Lucas, R. Global Burden of Disease of Solar Ultraviolet Radiation, Environmental Burden of Disease Series, July 25, 2006; No. 13. News release, World Health Organization

2 – Delays in diagnosis and melanoma prognosis (II): the role of doctors. Richard MA.

3 – Neural Network Diagnosis of Malignant Melanoma From Color Images, Ercal F. et al.

4 – I was told later by an entrepreneur formerly employed by the FDA that with a sufficiently awesome disclaimer one can put nearly anything on the market (“Nothing this program tells you means anything”) though I am really not sure how well this works in practice. I have read horror stories of the offices of a breast cancer detection aid being raided by the FBI — the case had to go up the the supreme court before they were acquitted.

5 – I do remember, however, challenging my tour guide in the waiting room when my cartoon host ‘Zappy the Electron’ claimed that both electricity and electrons traveled at the speed of light!

6 – Jacobson, M. Z., and G. M. Masters G. M., Exploiting Wind Versus Coal

7 – State-of-the-art if one is considering cost effectiveness, and not energy density! Pumped hydro plants are gargantuan.

8 – This is a good discussion: Copyleft versus Patents: The Open Source Legal Battle, By François Lévêque and Yann Ménière

Climate Change Skeptics

•October 11, 2009 • 13 Comments

I don’t understand the reasoning of so many ‘climate change skeptics.’

Let’s imagine the climate in question is not Earth’s, for a moment, and is instead the climate of a black box, hovering in a vacuum, with a big lightbulb shining next to it. Practically all its energy comes from the lightbulb (the rest from the residual heat within, and some dim source of central power), and practically all of its cooling consists in radiating infrared back outward. On the surface of this box tiny microbes are busy manufacturing and installing a layer of glass, which infrared cannot penetrate, covering it. We now wait, and see what happens.

The infrared is significantly absorbed by the glass, largely radiated back to the box, and thus the largest channel for cooling — essentially the only one capable of sustained cooling in the long term — has been attenuated.

Now replace the black box by Earth, the lightbulb by the sun, and the glass by CO2.

One would imagine the black box to have very strange properties were it not to heat at all. It might, for some time, somehow redirect some of the heat into less observable sections of its mass (e.g. the lower levels of the Earth’s oceans, which have a much greater heat capacity than its atmosphere). Yet this cannot last forever: there is only so much ocean. It might also become more reflective, absorbing less light (e.g. the earth’s clouds, desertification)? Yet an opposite effect comes from the melting snows and ice caps and constructed asphalt we add in urban areas: all of which have radiance and albedos observable from the outside (e.g. our satellites). Finally, the black box radiation is proportional to the fourth power of the temperature, so even if the percentage of radiated power that reaches the outside of the glass is diminished, if the temperature of the primary radiative bodies becomes less even, such that ∫T(new)^4 dA >> ∫T(old)^4 dA, the temperature can stay roughly constant. Other than that, there’s close to nothing that can be done: that box will very probably rise in temperature, and almost certainly the climate will change.

Skeptics correctly points out that the lightbulb varies in power output. And the black box is moving a bit relative to the light — further away or closer by — shinier or cooler or more black parts facing the light at any given time. They also point out that the glass isn’t the only thing surrounding the black box — for example they have noticed also a shiny layer of dust on the glass (aerosols), and an even bigger layer of glass underneath the glass we’d place (water vapor). And they point out that the layer of black paint appears to be, in a great proportion, liquid, and with a high heat capacity, and churning cyclically, and that there’s a lot of it, so that in any one instance a cooler or a warmer parcel of that liquid is showing.

None of this changes a thing about the fact that if we put yet another layer of glass on the box, the smart money is on it heating, and certainly on it changing. How could it not? At this point the onus is on these climate change skeptics to suggest a means by which the box is supposed to stay exactly the same.

Which brings up an interesting point. Maybe it is not so necessary that the Earth stays the same. Maybe there are credible arguments that explain that, really, the box won’t change that much, and for the teeming, glass manufacturing cultures of microbes living under the glass that these changes are not really such a big deal.

Some scientists, who I respect very much — Freeman Dyson for example, make this very argument. I respectfully disagree with him, as I think that there’s far too much risk in disrupting the biosphere, and that the disruption, famine, and loss of ecosystems and species that have already occurred are too great a price to pay, that oil wars, tyrannies, and people dying of respiratory illness from coal plants aren’t exactly positive either, and estimates of the probability of some catastrophic event happening, like say, Greenland melting, the consequences of which are too dire to imagine, range somewhere between 10% and 80%.

But that’s a philosophical disagreement. One might say that instead of engaging in ‘climate change’ skepticism, Dyson and others are engaging in ‘climate problem’ skepticism.

Too often, what we have with ’skeptics’ is a scientific disagreement: the great majority say either that it is happening, but only as part of natural variation, and they had nothing to do with it, or that it isn’t happening at all. Which, at this point, seem more like the antics of a child screaming ‘I didn’t do it,’ or putting their hands to their ears, singing ‘la la la, I can’t hear you!’ than of a calm and reasoned scientist — or skeptic — examining the assumptions of a majority opinion. Their conclusions are already drawn.


PS:

For those already sold on the problem, my startup, LightSail Energy, Inc. is an exciting, well-funded startup in the $100 billion field of green tech energy storage. We are located in Oakland, California. We are seeking to fill several Mechanical Engineering positions. Applicants should have at least 5 years experience in product design of mechanical components.

Please be familiar with at least some of the following:

engine design, fluid dynamics (analytical, experimental, computational), heat transfer, thermodynamics, fluid power, pistons and seals, and multiphase phenomena.

Candidates should be comfortable with 100 kilowatt to multi-megawatt systems. Our needs range from mathematical modeling and design of experimental apparatus during the Research and Development phase, to designing for manufacturability and reliability. The ideal candidate will be a hands-on design engineer who possesses a high level of creativity and innovation required to be a valuable asset to the company.

Interested? Please send your resume to jobs@lightsailenergy.com.

The Humor Event Horizon

•March 30, 2009 • 1 Comment

Where laughter becomes more hilarious than whatever half-forgotten thing preceded it.

Also known as H.E.H.

Acceptably Nonterrible Revision Control for Mathematica

•March 11, 2009 • 2 Comments

I discovered today, to my amazement and horror, that the otherwise exceptionally advanced and useful Mathematica 7 has only a single level of undo in its notebooks. And no redo.

Furthermore, the lone autosave feature, hidden deep, is triggered upon every cell evaluation. Every time you evaluate something in a notebook, it will save the file – overwriting the last version.

I spent an hour looking to see what was the matter. I was convinced that it must be my own obtuseness causing me not the find or see or properly use the actual undo function there present. Instead, I found several newgroup threads in which people exclaim their amazement that such vital features as multiple undo and revision control hadn’t made it in, and apologists of various employ, Wolframites and others, who wrote back about the academic difficulty inherent in making multi-level notebook undos for notebooks fed with dynamic data.

I’ve got news for you Wolfram Research. It doesn’t matter. What people care about is their input data. What people don’t want about is destroyed work – the product of their labor, channeled through from their mind and out through their fingers, typed into little, nicely formatted cells. Almost everything else can be reconstructed from that input. The cases in which the inability of a simple multilevel undo to capture changes in dynamic data could possibly destroy work make up a fraction of a percent. Catastrophic hard disk failure is more likely.

I checked for a while to see if anyone found a solution more advanced than to manually save often, and use standard revision control. I couldn’t find anything.

Thus, I present my complete hack of a solution. It seems to work acceptably well — as in, I think I can rely on it to save my work, most of the time, if it really came down to it. But use at your own risk:

1. Get Dropbox. (getdropbox.com) This is a file sharing and revision control system that’s built right into the operating system, Windows, Mac OS X, Linux. One creates a folder, shares it over Dropbox, and every file in it is mirrored, automatically, on their servers. In particular, they keep track of revisions: so far as I can tell, every saved file is queued to be uploaded.

2. Enable Autosave. One can set this for the current notebook by evaluating the following line: SetOptions[SelectedNotebook[], NotebookAutoSave -> True]. This setting persists across saves.

3. Now every time one evaluates a cell, the notebook is saved. Further, while the file is apparently overwritten locally, the revision history (which I believe is, in most cases in full,) has been uploaded to Dropbox, or queued by the client.

4. To recover a revision, one can check the revision history in the Dropbox web interface.

5. Standard diff facilities are less useful, because the notebook files are highly structured. However, there’s a structured diff built into Mathematica, Notebook Diff in the AuthorTools package. One loads the package by evaluating << AuthorTools`. One can run this command NotebookDiff[nb1,nb2], where nb1 and nb2 are either notebook objects or their filenames with full paths.

This seems to work well enough for now. In the meantime, Wolfram Research, get your act together! You are driving people crazy.

I am also putting together a way to use the Units` package throughout (much more of) Mathematica, in particular, in conditionals and numerical functions. In practical engineering it could often be said that converting units is one’s primary occupation. For this reason the units package came to me with extreme promise. Built into the algebra package as it was, it could be used, in an extremely simple way, to check dimensional and unit consistency. However, it is not natively compatible with numerical functions: one cannot plot expressions including units, or expect results from comparison operators, and they can’t be used with functions like Max and Abs, or within the solver for numerical differential equations. It would be so wonderful if it could. I think I’ve figured out a way forward there, but we’ll see. The basic idea is to set to 1 all base units in a particular unit system — SI or CGS. This is likely to work for mechanics, however due to the differences in how the CGS and SI systems define electromagnetic quantities, making it work well for electromechanics is either a subtle project, or a doomed one. [Edit: David Park's ExtendUnits package appears to do exactly this. It's $30, though. http://home.comcast.net/~djmpark/UnitsPage.html]

Also, I don’t know if anyone’s really following this, but we’re not working on vehicles specifically anymore; LightSail is directing itself towards energy storage as a first product. Our website is woefully out of date. Vehicles or vehicle systems may be a further product and direction of ours, but by focusing on energy storage we can really prove out the core technology without the additional hassles of becoming an automotive company.

Yes

•December 31, 2008 • 1 Comment

you have saved me from an eternity
of what if
with one moment of yes

– M. Volkova

Launch

The Cure for the Common Cold

•November 16, 2008 • Leave a Comment

Once upon a time in Northern Mexico, my mother was sick. She had come down with a cold. Sitting on her porch, an old, tiny Mexican man walked up to her and said, “Beautiful lady, why are you looking so sad?”

“I have a cold!” she said. “I am tired and chilled. I feel like nothing.” Hearing that, he took a small flask from his jacket. “Take a swill of this. You will feel better.” It burnt on the way down. “Tequila perhaps,” she thought, “if it is, it is the best I have ever tasted.”

She said nothing. With a smile, the man shifted his hat, turned his body, and walked away.

The next day, he visited my mother, sitting on the same porch. “Are you feeling better?” he asked, reaching for his flask. “Maybe a little,” replied my mother, quaffing the fiery liquid with uncertain haste. “Good,” he said. And with that, he pulled the collar of his jacket taught, smiled, and walked away.

In a blur the days passed. The old man visited the next morning, and the morning after that, and on and on for a week. Finally, he asked, “Are you feeling better?”

“I am!” she said. “Good,” he replied. He shouldered his bag and turned to go.

“Wait!” she faintly cried, “Won’t you let me sip once more of your medicine?”

“More tequila?” the man said concernedly, furrowing his brow. He waved his hands in circles towards her. “There is no need. You are cured.” And with that, he gave a smile and a wink, he turned, and walked away.

Tick Tock

•November 15, 2008 • 8 Comments

The clock stuck twelve. It’s October 30th. In a heartbeat I emerged an adult in the eyes of American law. In an alternate universe, I danced the night into a hazy sunrise. But I left celebration to Haight St. patrons, their addled revelry spilling muffled through the crack in my window. Tonight, we work.

Dawn, a night and two weeks later. It was ready; the design for the both the engine and the drivetrain, encoded in a scattered handful of drawings and documents, one wiki, two heads, and a thousand lines of physical simulation code. The first test: powering a scooter through a staccato ride amid frenzied Manhattan traffic, calculating, by the hundredth second, the will of the engine, and the vehicle’s reply. We’d follow a path devised to track emissions from humming, throbbing combustion engines, byproducts of fuel burnt in tiny explosions sparked every second by the thousands.

EPA New York City Cycle

But nothing save cool air would our machine exhale. Compressed air, ‘a thermomechanical battery’ of sorts, is cheap, long lasting, and quick to recharge (one need only open a valve, and if impatient, run a pump, the tank will fill in seconds.) What’s more, it’s efficient. A batteries charge begins life in mechanical form, in a spinning turbine if charged off the grid, or in the inertia of a vehicle, during regenerative braking. This is then converted into AC electrical current, which is converted into DC current, which, finally, is converted into mechanical energy, losing power at each step. To power the engine this whole process runs in reverse! But compressing or expanding air keeps mechanical energy mechanical (so long as temperature is kept reasonably constant.) In powering vehicles it is superior to the most advanced battery systems known. That is, in every parameter but one.

Historically, the low energy density of compressed air had crippled any attempt to venture further than a couple dozen miles; physics, it seemed, demanded tanks of excessive proportions to travel longer. At 300 bar (’scuba pressure’), compressed air could release only half a percent as much energy as the same volume of gasoline burnt. We understood, however; it was an efficiency war. We knew that conventional vehicles were incredibly wasteful. There were many battles left to fight.

The Laws of Thermodynamics1

“You can’t win.”
“You can’t break even.”
“You can’t give up.”

We hunted losses relentlessly. We were repaid with a series of compounding improvements, each building upon another, reversing the conventional patterns of efficiency losses endured by vehicles for more than a century. Finally, in a brilliant and unusually compact layout by Steve Crane, we found room to replace the paltry 1.3 gallon gas tank with one ten times its size. Nights yielded to our toil, and, slowly but surely our enemy routed.

“We’ve cracked the code,” we exclaimed. “The city is ours to conquer.” On the highway, whatever benefit earned by our scooter’s light weight, low rolling resistance and ultra-efficient regenerative braking would be dominated by air resistance.2 But air resistance falls quickly with speed, and in the stop-start motion of the city our combined inventions would give our scooter an efficiency historically unmatched.

I keyed in the last few drivecycle parameters, drew a shallow breath, cocked my head, and pressed the enter key. The simulation lasted only a moment, but in that time, my little scooter ran more than one hundred and twenty miles, the equivalent of dozen rides between Wall Street and the Bronx on a single tank. “We’re in business,” I said. With that, and for all of a New York Minute, the questions, worries and restlessness retreated from our hearts. We huffed. “What’s next?”

[1]: To paraphrase C. P. Snow. Hat tip to Jonathan Smith.

[2]: Scooters are not particularly aerodynamic vehicles. Ordinary scooters have a drag coefficient of nearly 0.9, and a frontal area of 0.6 meters squared. We hope to achieve a drag coefficient of 0.6, similar to faired motorcycles ridden upright, but due to the rider’s position this will be difficult: some have described the aerodynamics of a scooter as like a “brick attached to a parachute.”

Reconstruction of Data from a Chart or Graph

•October 26, 2008 • 5 Comments

I have here several charts of driving cycles. These are standard plots derived from real traffic data, of velocity versus time. Unfortunately I cannot find the data anywhere. So I hatched a plan: maybe there’s software that will reconstruct data from a graph or chart? Does anyone know? If not, I’ll just write it myself and open source it. It seems like a generally useful thing in engineering and science. It deserves an application (maybe even a web application).

Unified California Drivecycle

Timelessness

•September 13, 2008 • 12 Comments

Many believe that technology simply gets better over time: that every class of invention can improve endlessly into modernity. That is not so. Most of the hard constraints on technology are imposed by physical or mathematical laws. These remain constant. Those who truly understand this may work, instead of towards the solution of individual problems, towards timelessness, and the ideal platonic form.

Wheel, Iran, from 2nd Millenium, BCE

Excavated at Choghazanbil Ziggurat, near Susa, late 2nd Millenium BCE.

Enhancing Multitasking to Enhance Our Minds.

•August 24, 2008 • 12 Comments

Jenny Boriss and the Mozilla Labs team have helped spark a wildfire of discussion. What can we do to make browser tabs better? I’ve been considering this question for a while now (heck, I practically *live* in Firefox), and Aza Raskin’s recent posts have finally inspired me to weigh in.

This may sound strange, but if you look at it a certain way, the introduction and popularization of tabbed browsing represents a tipping point in the history of computers. And, as they have altered and improved our ability to multitask, they represent crucial advances in the history of human thought.

There’s no fundamental difference between a tabbed web browser and an operating system taskbar. They are tools for multitasking. They are intended to make accessible each distinct thing which we seek to do, on our computer, so that we might easily glance at, remind ourselves of, or focus our attention on, whatever is most profitable, intriguing, or demanding of attention. Tabbed browsing popularized as web documents and applications overwhelmed their desktop counterparts. Computers had altered part of our working memory. Now, the web plays a leading role.

Computers have caused such an increase in our ability to multitask that they have precipitated what is one of the greatest generational shifts in patterns of thought since the scientific revolution. Quite literally, when we alter the way we multitask, we alter our minds.

Dangers

This is not without dangers, of course. We can still only keep so many things in mind (7 plus or minus 2, for most of us). Computers can display more tasks and more information than we can handle, and attention remains a finite resource. Computers are only a level two cache. If we don’t carefully control how we drink from this firehose, our attention may shatter, rendering us totally useless. For some, in today’s YouTube generation, this seems to have happened.

If tabbed browsing is a tool for thought, then we might expect that even small alterations in tab behavior will affect habits of mind. Consider the popular method of conducting research. One begins with a search, and lands on a Wikipedia page, or blog article. As one scrolls down the page, we middle click, opening tab after tab in the background. But where do they go? If the tabs land go directly to the back of the list, ordering tabs by age, we end up with something kind of strange. People tend to move to the next tab in the list, and so, the tendency is for our poor researcher to perform what is known in tech circles as a breadth-first-search.

There’s something to be said about this. Breadth first surfers only skim topics. They’ll cover many in one sitting, certainly. But they’ll tend not to learn in much detail. They will, in other words, learn many sides of many things, superficially. Shallowly. This could be one of the major reasons why, online, so many people talk so much about things they know so little about with zero apparent knowledge of this fact. Draw your own conclusions.

If, by contrast, tabs are opened directly behind the parent tab, we lure our researcher into a trap of a different sort. A depth-first-search.

Usain Bolt wins the 100m! Who’s he? Let’s check the Wikipedia… ah, it says here he was born in  Trelawny, interesting name — it’s a parish? Isn’t that like a religious thing? Hmm, it can be, but maybe it’s a civil parish, which is, hmm, a subdivision of the United Kingdom? Huh? Oh — I guess it’s part of the Commonwealth. What’s with that name anyway… seems kind of random… for “the promotion of democracy, human rights, good governance, the rule of law, individual liberty, egalitarianism, free trade, multilateralism, and world peace“, I guess those are good, especially… individual liberty… heh… I wonder how many times Dubya has trumpeted that one… Dubya… Maybe this has a list of Bushisms… which are apparently malapropisms… there’s a mouthful… where was I?

It’s easy to joke about, but it’s no laughing matter. It happens to the best of us. It wastes our time.

A Personal Note

People don’t always read straight through the tab they have open, and move onto the next. They zip around. Or they’d like to. Or, at least, I like to. When tabs first came out, I used to fill my browser with bajillions of them. You’d barely be able to click on one, much less recognize one by its hidden icon or title. There were tabs a pixel wide. Less, if I could get away with it!1

So what did I do when I wanted to switch back to, gmail, say? I couldn’t find the old gmail tab. That would be impossible. So, I’d open a new one!

Soon, I’d have a zillion gmails, which would slow my computer to a crawl. And the tabs would be almost completely useless: it was totally impossible to switch around, all I knew if that there a lot of articles I wanted to read in there, and if I just kept working on the pile, then eventually the titles would see the light of day and I could use my tabs again. Usually I just crashed the browser. Oy.

Firefox 3 changed a lot of that, but not entirely for the better. Instead of trying to squeeze fifty tabs into one space, it just hides them offscreen. But the same problems come up: I still opened googles of gmails, and my browser would slow to a crawl. Like cold molasses, running up a hill. Honestly.

I’ve learned a little since then, and I’ve made some improvements. Some are technological — little plugins, restoring my sanity. The first is ‘aging tabs‘, by Dao Gottwald. It’s a brilliant little Firefox plugin that makes tab headers fade with age, from light to dark. It helps by reminding you what you were reading most recently.

The second is Tabhunter. Those familiar with Quicksilver or Enso will recognize its function: you just press ctrl-alt-t, type part of the name or URI of the open tab you’re looking for (like ‘gm’ for gmail)2, and a list of matching tabs is displayed. Press up or down and enter to select the tab, and you’re there. There are a few minor problems with the design I think; for one, it’s modal (a box pops up, and you need to hit enter to dismiss it), and it saves your old patterns instead of clearing the input, requiring more keystrokes than I’d appreciate, but overall, it’s been a huge boon for me. It makes it much easier to switch between tabs, and I love it.

Finally, I use fullscreen mode. I found having a list of tabs staring me straight in the face had the effect of luring me away, towards something else, whenever things got hard. Now, with fullscreen mode and Tabhunter, I can concentrate on my work or reading, and switch around, only to do something definite, when I need to. This changes everything.

Suddenly, keyboard shortcuts became best of friends: ctrl-1 shifts to the front of the list, ctrl-9 shifts to the back, ctrl-w closes the current tab, ctrl-shift-t opens the last one you closed (or the second, third, fourth last one, in order), ctrl-d bookmarks, ctrl-l focuses on the Awesome Bar, ctrl-k zips to the search bar. alt-left or alt-right to go backwards or forwards. It’s way faster. The first time I started to surf this way, I had nearly 60 tabs open. I just kept reading through, fully completing each task in front of me, dismissing it when done (closing by ctrl-w), moving onto the next item in the list. I was absolutely startled, in just an hour an a half, my browsing was complete. I almost couldn’t believe it.

This has changed my life. You might laugh, but too many times I’ve stayed up, dozens of tabs tormenting me, unable to close them, afraid of sleep, of losing mindstate. I browse faster, concentrate better, and learn more. And now I am free.

Concentration

The lesson, I think, is that computers hold great powers.3 They can remember what we need to do, and what we need to know, freeing our minds from needless memorization, allowing us to concentrate on the task at hand. Or, they can present to us an endless stream of distracting tasks, tweets, and you-tubes, shattering our attention, and any hope of getting work done.

With each distraction, my mind thrashed. Heisenthoughts collapsed. Mindstate was lost. And for what? It takes far too long to start upon challenging work. Frequent taskswitching rendered me barely capable of any challenging work at all.

With the tab window hidden, my world changed. No longer was I tempted endlessly by the idle promises of articles unskimmed, tweets unheard, posts missed. I simply had to concentrate on the thing in front of me, and judge it worthy of time and attention, or not. It worked like magic.

I stopped being distracted by thoughts that there might be something marginally better that I could be doing. Instead I just did it. When I was done with it, I closed the tab. It might be, much more closely, a breadth-first-search, but, damn it, it worked.

Troubles

The trouble with this approach, I found, was that I was literally following a breadth first search with my browsing, which was not always what I wanted. Often, Tabs would present themselves in exactly the wrong order. Yet I’d still be tempted to go through them.4

People keep tabs around. They use their tab system as a to-do, to-read, or to-check-list. They keep tabs around as applications — things like gmail, and Facebook, and Pandora, which you might as well be logged into all the time. But keep too many things around, and you create a massive cognitive load. By hiding my tab list, I lessened that cognitive load. I outsourced prioritization to my computer. But my computer wasn’t any good at it. I switched tasks less often, which allowed me to concentrate, which helped me read, and helped me work. But it probably didn’t direct me towards the best thing I could have been doing. I worked, but, in some sense, on the wrong thing.

Priorities

I explained this dilemma to my friend Joel, breadth-first/depth-first analysis and all. He said that I had a linked list of task, and what I needed, instead, was a priority queue.

Duh.

Okay. So that’s goal one. Tabs are tasks: make it easier for us to prioritize our tabs, and we make it easier to prioritize our work.

Memory

Tabs serve as a kind of memory. We use tabs as a way to remember things we intend to do. But what aging tabs shows us is that we can also use tabs to remember things that we’ve done: specifically, what we’ve recently been looking at. What else might we be able to show?

First, let’s give ourselves a great big space to work. Suppose, say, when we hold down ctrl-tab (quasimodally), a great big transparent tab display appears over our window. We can zoom in or out, and drag the display around. We can even give it momentum: physics, just like the iPhone. We can toss it around. On it, what might we show?

Let us channel Edward Tufte for a moment. Let us maximize information density. Let us minimize data ink.

A Paper Mockup for FoxGlide

A Paper Mockup for FoxGlide

We can show content. We can display a title, with the favicon, atop a thumbnail of the tab. This is much better than just showing the favicon, as screenshots are far more likely to be distinct, and inform the user of the nature of their content. We can also show which bookmark folders that tab has been placed in or which tags the tab has adopted.

We can show order. Lay the tabs out horizontally, in the order in which they were opened. (Just like the current default, in Firefox).

We can show time. Near the top of the screen, make a small ticker, and mark the horizontal axis with small, vertical ticks: by clocktime, at distant regular intervals, and with icons at every change in activity.

We can show link structure. Connect every parent by an arrow to their children.

We can show relatedness. If two tabs share a domain, if they commonly are switched between, or if they share a large amount of semantic content, draw a faint line between them, to be highlighted when the tab is selected. In this way, we can automatically cluster groups of tabs used for a shared task or activity, and remind the user of tabs habitually used together.

We can show history. Turn the ticker at the top into a lifestream. At each tick, denote the activity by a little icon. Suppose you’ve done a bunch of typing: this is a writing tab, we can represent this with a little ‘key’ icon. Suppose instead that you’ve just scrolled and read: show scroll buttons, up and down. Suppose we’re idle: show an ellipsis. Suppose we open a new tab: show a sprout. Suppose we open a tab in the background, creating another branch: show an arrow branching into two.

We can show age. Let the tabs fade with disuse, as in aging tabs.

We can show status. Make the title of as yet unviewed tabs blue, as in unvisited hyperlinks. Make the title of visited tabs purple. And instead of wiping closed tabs off the display, make the titles brown, and let them inhabit a ‘history gutter’ nestled quietly at the bottom of the screen — to view it more fully, pull-up. when a tab is closed, maintain the arrows, connected to parents and children, but fade them so that they do not distract. By keeping closed tabs accessible, we seamlessly integrate browser history with the tab display, enabling browsers to enhance our memory still further.

Finally, we can show priority. Let the vertical axis represent priority. Higher implies more pressing. Priority can be set automatically: Age lowers priority. Activity raises it. Or you can set it manually. You can drag each tab along the vertical axis to set the priority, or you can strike a number, to alter the priority of the tab currently viewed, or otherwise selected. We can even use a hybrid approach: after a while, an algorithm harness the implicit, ambient information from our browser use, to learn which sites, keywords, and activities you consider high priority, and set priority for every other site you visit accordingly.5

To make it easier to set the priority for large groups of tabs, the priority of related tabs, for example, parents and children, influence each other. This implies that, while raising the priority of one tab, the arrows connecting parents and children pull the related tabs up elastically. We can also uncover tab relatedness by whether they share a domain, or if we would like to get really fancy, we could even link tabs by shared usage of ’statistically improbable phrases’, as Amazon might.

Navigation

With the tab display active, we can quickly type to find the tabs we want. Type ‘wiki’, and all the tabs matching ‘wiki’ are shown, filtering the others out, which fade away. Hit left or right to progress through the history of open tabs: in the same order as is currently used by Firefox. Hit up (or enter), and we switch to the the highest priority tab (that isn’t being currently viewed) matching the filter we typed. Hit down, and we instead decrease priority, stepwise.6 It’s like Tabhunter, but using less keystrokes.

The labs team has already come up with a decent navigation system. They suggest that we put navigation in the Awesome Bar. If someone goes to the Awesome Bar and types a pattern that matches an open tab, pressing enter will take them to it, while pressing option enter will open a new tab.

I worry about this a bit. As much as I like the Awesome Bar, keeping focus on it is modal. And you need to select it (ctrl-L, or a mouse click), and then type in the name, making it a bit more complicated.

Also, it messes with important prior behavior. Most people will expect to go to somewhere on the current tab when they hit enter on the awesome bar. Changing this behavior might confuse users, conflicting with previous program models, doing so in a way that does not make evident that program behavior has changed.

Finally, compared to the approach I describe above, it simply doesn’t show as much information. It’s not as useful as it could be (note: see the mozilla labs wikipage on visual navigation for more ideas.)

Status

Where might you be able to download this? You can’t, yet. It is, for now, just a twinkle in my eye. But we’re on the cusp of something. The Mozilla team is just now in the process of deciding what new tabbed browsing features they want to build into Firefox 3.1, and are soliciting ideas. If you like this concept, or think, at least, that some of the ideas are worth considering, or if you have your own ideas, please, get involved in the conversation!

Also, if you’re interested in helping to do some mockups or some Firefox extension coding, please let me know (email me at daniellefong at daniellefong dot com). It will stay a small, open, simple project; this won’t be my fulltime thing. But for me, at least, I cannot ignore the chance we have to deliver ourselves from cognitive overload. To free us to explore the web more fully. To enhance tools for thought, and thus enhance thought itself. As whimsical as that might sound, this hope shines far too bright for me to simply let go.

Notes:

[1] – In other words, without crashing the browser. Opera was much better in this regard than Mozilla.
[2] – This actually works with regular expressions too.
[3]With great power comes great responsibility.
[4] – This was not the only problem. sometimes, I’d see a link, and I’d want to follow it, just a little ways, to see what was there. But if I opened it in the background, I’d have to finish all the other tabs just to get there. Ahh! Instead, I just followed the link without opening a new tab, with the hope that I’d remember to return to the parent page and alt-left backwards. Soon I found the hotkey for flipping to the end of the list (ctrl-9). That helped a bit. I could open a tab, and flip to it in back, though I’d have to use Tabhunter to get back where I started.
[5]It’s *Xobni for Firefox*! The web is the killer app! (I promised myself I’d never say things like this, but I actually kinda like it.)
[6] – Note that under this key scheme, ctrl-tab doesn’t switch to the next tab like it used to. That would now be done with ctrl-tab-right.

Thanks

to Alex Lang, Nick Pilon, and Joseph Perla for reading drafts of this, Sasha and Patrick from BaseShield, for the encouragement, the Mozilla Team, for running this wonderfully open design process, Blacktree and Humanized, for showing me how good navigation could be, Jenny Boriss, for sparking the tabs discussion, and Joel Muzzerall, for the inspiration.

PS

I’ve given some thought to the name. I, personally, like FoxGlide, which also exhibits the wonderful feature of my having parked it.

Please Leave Your Comments!

This post has sparked some interesting on Hacker News. What do you think? Does the design pass the Phone Test? Are the angles I haven’t considered? Are like features you’d like to see?

Heisenthought.

•August 20, 2008 • 1 Comment

heisenthought,

\ˈhī-zən-tht

noun.

1. A thought subject to mental collapse triggered by interruption.

2. A mental event or sequence characterized by coexistence of loosely related, sometimes contradictory, sets of knowledge or tasks. Highly dispersed in knowledge space, though it may support well defined momentum. Apt to tunnel through imposing technical or energetic barriers.

Collapse triggered upon the observation of the state of current knowledge by a third party. During the beginning states of observation, keen thinkers engage in a mad scramble to save mindstate, which may at some later time be partially reloaded. Status reports are heisenthought’s natural predator. Occasionally, members of a team will spontaneously ‘go dark’, in an attempt to think through some difficult problem — sometimes the only strategy which will work to extricate oneself from local extrema in solutionspace.1,2 Caution should be taken when observing said people, as doing so will both collapse their progress into a single, likely suboptimal path, and be likely misinterpreted as disloyalty, hubris or laziness, since they have disengaged from discussion with no apparent progress.

3. A mental event or sequence characterized by firmly specified and demarcated generating bodies of knowledge, and ill discernible forward movement. Induced by overspecification of problem, method, or solution. Collapse upon observation of current direction of progress by a third party. “‘Are we moving forward on this?’ ‘The heisenthought collapsed; we’re moving, just not in the right direction…’”

4. A coherent collection of thoughts sufficiently isolated from random outside influence. Obeys the relation Δknowledge Δmomentum ≥  ħ/2.

5. A tongue-in-cheek term invented to cast off workplace ‘pings’, ‘check-ins’, ’status-reports’ and ‘how’s it goings?’.


Notes:

[1] – For example, read the interviews with Max Levchin in chapter 1 and Blake Ross in chapter 29 of Jessica Livingston’s Founders at Work.

Max discusses how he went dark for a year, and came up with the something that really mattered — a way to fight fraud.

We had this merger with a company called X.com. It was a bit of a tough merger because the companies were really competitive—we were two large competitors in the same market. For a while, Peter took some time off. The guy who ran X.com became the CEO, and I remained the CTO. He was really into Windows, and I was really into Unix. So there was this bad blood for a while between the engineering teams. He was convinced that Windows was where it’s at and that we have to switch to Windows, but the platform that we used was, I thought, built really well and I wanted to keep it. I wanted to stay on Unix. [...]

I had this intern that I hired before the merger, and we thought, “We built all these cool Unix projects, but it’s kind of pointless now because they are going to scrap the platform. We might as well do something else.” So he and I decided we were going to find ourselves fun projects. [...]

It was me acting out, but it was kind of a low time for me because I was not happy with the way we were going. Part of having a CEO is that you can respectfully disagree, but you can resign if you don’t like it that much. But then eventually I became interested in the economics of PayPal and trying to see what’s going on in the back end, because I was getting distracted from code and technology. I realized that we were losing a lot more money in fraud than I thought we were. It was still early 2001. If you looked at the actual loss rates, they were fairly low. You could see that we were losing money, but, given the growth of the system and the growth of the fraud, fraud was not that big of a problem. It was less than 1 percent—it was really low. But then, if you looked at the rate of growth of fraud, you could see that, if you don’t stop it, it would become 5 percent, 10 percent of the system, which would have been prohibitive.

So I started freaking out over it, and this intern and I wrote all sorts of packages— very statistical stuff—to analyze “How did it happen; how do we lose money?” By the end of the summer, we thought, “The world is going to end any minute now.” It was obvious that we were really losing tons of money. By midsummer, it was already on a $10 million range per month and just very scary.

[2] – Blake discusses how, by necessity, they closed up the source and process during the early stages of work on Firefox.

Phoenix was basically a fork of the Mozilla code base that we controlled. We closed off access to the code, because we felt it was impossible to create anything consumer-oriented when you had a thousand Netscape people in search of revenue and a thousand open source geeks who shunned big business trying to reach consensus. We just wanted to close it off and do what we thought was the right thing. We went through a couple name changes, Mozilla offered us more support, and that’s kind of how it all got started.

Thanks to Marc Chung and Joel Muzzerall for reading drafts of this.

Engineers + A Paper Tablecloth = …

•August 20, 2008 • 4 Comments

Our company is going to have graph paper placemats. Jealous? ;-)

Living Things

•August 14, 2008 • 3 Comments

analogies, once preserved as mere notes, find harmony as poetry

organizations
relationships
communities

as living entities

as where no one cell holds the soul of a person
no one person, incarnate
may form all essence of a community
nor live throughout its lifetime
wholly part

as one cell expires
one soul retires
others, some new, fill their place

generations cycle
yet something remains
an living entity unto itself
though it may be true
that some people or ideas
resemble vital organs

the liver, to filter
the stomach, to process
the lung, to supply
the mind, to direct
the heart, to power

too often
transplanted,
perfectly good hearts
from one being to another
from one community to another
raise immune reactions
edicts like allergens
mobs like histamines

without disarmament
no transplant succeeds
no hire, restructuring, or revolution carried through
without somehow disabling
or surviving
societal antibodies

for some, it seems
while becoming part of a great organization
or movement
or organism
they enter through digestive tracts
to supply the raw stuff
the best
carbon, iron, calcium, oxygen
skills, goals, ideas, selves
are prepared
and cooked
chewed
dissolved
filtered
processed
and made, finally, into a part of something new
to fit some other master plan
a genetic blueprint
an ideology
a mission
a politic

for some
no greatness of which they are part
could come to redefine them
their raw stuff
the parts comprising
can find no better molecule
no further local minima
no structure more solid
between surrounding stomach walls
they are indigestible

or too delicate
too beautiful
too unique
to be eaten

their soul to realize
as part only whole
of something profound,
meaningful,
believed in
or as the kernel of a gem
of another body
of an essence of its own
starting anew

Keeping Prediction Honest

•July 20, 2008 • 13 Comments

I base my action upon prediction. Every technologist should. I try to see how the world will be, and then try and see within that future what place I may come to hold.

So prediction is fundamentally at the heart of a technologist’s work. At the highest level, we must predict to find what work focus on, and what future to aim for.

You might then think that prediction, as a skill, is worthy of practice. And practice it gets. In living rooms, in pubs and classrooms and yearbooks and dial-in talkshows and newspapers and blogs and comment threads and slashdot and every polluted corner of our existence, you find evidence: prediction is practiced all the time.

There’s a problem. In most areas of the technologist’s pursuit, it’s easy to see whether you’ve done well. Code should compile. Planes should fly. Cars should go. Bridges should stay up. We have a lot of honesty in our discipline, much of it because we are blessed with tests that we find hard to fool.

A typical test for predictions, on the other hand, is whether the story sounds good at the pub. You make some exclamation. People nod and clap. Everyone forgets.

This would be fine if you’re just looking for some conversation. But if you are, like technologists fundamentally in the business of creating the future, it becomes lot more troublesome. We are left to ignore predictive incompetence until reality slaps us coldly across the face. We are flying blind.

Taking a cue from Trevor Blackwell, I’ve decided to inject some rigor into my life: when I make predictions, instead of casting them abstractly into the air, I’ll post them here: einfall.slinkset.com. And I won’t delete my predictions — if they turn out wrong, I’ll keep them there, as permanent reminders to learn from.

Through accountability, honesty. Through honesty, improvement.

Thanks to Trevor Blackwell for the inspiration, and John and Brett from Slinkset for the List Hosting.

Notes: a friend of mine noted that most of my predictions seem ‘pessimistic’, in the sense that they take the form of ‘X will not Y.’ I would have to agree with him. But this is largely a byproduct of how these predictions were made – they’ve come from studying some field, working in it for a while, and coming to the creeping realization that one or more of the current approaches were doomed. Besides, much of the skill of experts comes from the ability to ignore false trails.

Further Reading: An excellent site for major predictions (often with significant wagers) is Long Bets.

The Choice of Work

•June 26, 2008 • 25 Comments

During the back and forth of exchange with a technical recruiter, he finally asked me what I was looking for. And so the floodgates opened.

This may sound weird, but I pretty much choose employment based on the promise of quality work. Other factors fade into irrelevance.

When I say quality of work, I mean more than the work environment, more than the magnitude of technical challenges, and more than the IQ of those I’d be working with. I want the opportunity to walk paths with the greatest hope of leading to first-class work. Nobel-prize winning kind of work.1 This force guides me, and so inevitably I tend upstream of technological change. Money and prestige are mere proxies for what really want: to develop and inspire fundamental changes in the way people live. That doesn’t mean I need tackle the greatest problems humanity now faces (yet). What matters is that I, personally, have a reasonable approach. So I must always remind myself to, as Richard Hamming says, ‘plant the little acorns from which the mighty oak trees grow’ — because small projects can, swiftly and strikingly, grow momentum and value.1

I find little value in submitting myself to some company culture. I instead mean to develop my professional values, ambitions, and goals: for example, I would like to develop new methods, make them available by open sourcing them and make them popular by evangelizing them. I’d love to be given the chance to teach what I’ve learned. Excellent people bring ideas and perspective to a communities of makers. Given time, I think this will evolve naturally into a company culture worth having.

Most big companies grow faster than they could build trust, to a size greater than strong values can be supported. Natural culture is the product of alignment of creative philosophies, and in BigCo, this is too often replaced with virtual company nationalism. Fascism even. I find this is more than distasteful. I haven’t really learned how to work within it at all. I could devote my efforts to such an organization only were there deeply meaningful work to be done. And why bother?

Paul Buchheit's first Google Check.

Angling to be upstream of technological change, I bait unusual questions and find surprising answers. Give me the choice between a VP position at a big five media company with oodles of benefits, and, say, work at an early netscape or google for a totally minimal salary, and I’ll choose the latter every time. I’m pulled towards organizations where I can learn about organizing, rather than learning about institutional tradition. It’s not important for me to learn about how to run a large organization: if ever I do, I won’t follow of the paths of current captains of industry. Instead, I intend to help grow large, leaderless, open organizations, and so I’d do almost anything for a chance to work with Caterina Fake, or Linus Torvalds.

I want to work on something I find deep personal meaning in. I strongly believe in supporting open culture. I don’t think I’d work for long in games or entertainment unless it could influence some social change. I worked at MochiMedia because it made possible an income stream for small independent developers where none existed before. This finally opened up professional game development from BigCos. Now, much innovation in gaming emerges from bedroom studios. Independent game developers can now commit to their art in a way they before could not.

Similarly, I’d work at YouTube rather than Hulu, even though one’s a startup and the other isn’t, because they’re more interested in involving everyone in the process. As Clay Shirky says, they’re interested in ‘finding the mouse’.

I want to work somewhere where I can truly make a difference. Why am I working in technology at all? Archimedes once said, ‘If you give me a lever and a place to stand, I can move the world.’ Technology is my lever. I need only find place to stand. This makes me wary of startups that try to do good, but aren’t particularly focused on doing it efficiently. I wouldn’t work for most charities. There’s too little pressure on them to focus — the tempering influence of market competition is replaced by government demands for ‘accountability’, which arn’t nearly so powerful.

There are numerous ‘ecogreen’ websites out there that try to promote simple, green ways of living. These may be virtuous, however, in terms of minimizing environmental impact I think they’re somewhat irrelevant.2 Saving plastic bags won’t lift a toe on our carbon footprint unless we find ways to either cut down on air and automobile travel, or do it more efficiently. And on carbon footprints — global warming is, I think, a red herring — there are thousands of nasty effects of pollution from, say, coal-fired power plants that will hit even if global warming doesn’t occur (though I think, probably, it will). Too much of China now wears breathing masks.3

I can see myself dedicating myself to the right company, so long as our goal, philosophies, and ambitions align. Yet these are stringent requirements. So far, then, I’ve found it necessary to reserve some energy and time for my own projects. So I must be open with companies: with most, I want only consulting work, to help them with some particular project, idea or problem. And I want to be completely, totally honest with everyone about it, because so far, the high road has never let me down.

Sincerely,
Danielle

———

Notes:

[1] – From the classic talk by Richard Hamming, You and Your Research. I don’t, particularly, apologize for my ambition here. Why shouldn’t I try to do first class work? The Nobel prize winning part is purely incidental. But this is the kind of work I mean — a significant contribution, one that people can build upon.

[2] – The free, online book Sustainability – Without the Hot Air is an excellent read. It is the first thing I’d suggest to someone interested in seriously starting into environmental matters. I shouldn’t claim that small contributions to green living are completely irrelevant — each does have some small effect. Perhaps raising the issue of green living in our collective consciousness will have an effect greater at second order than I imagined. But so many of our behaviors are misplaced. Many people, for example, go out of their way to buy ’sustainable’ products at Whole Foods, say, when in reality, longer vehicle trips do more damage than almost anything you could buy. Many things are sold in a way to make you feel good about buying them. They don’t have any real effect!

[3] – This is worded provocatively, but pollution in China is a growing catastrophe. See ‘As China Roars, Pollution Reaches Deadly Extremes’ and ‘Where Breathing is Deadly’.

———


Thanks to Alex Lang, Ma’ayan Bresler, Nick Pilon, Colin Percival, Michael Nielsen, and Joel Muzzerall for reading drafts of this, and Charles Beatty, for sparking it.

PS: Certain misconceptions have been raised. Some feel that this is one demand of an over privileged generation. I reply to this here. Additionally, I am not, in fact, abandoning my startup. But I do need money, and a visa, so I am looking into either employment or seed funding.

Cosmology in Ten Minutes

•June 8, 2008 • 15 Comments

Recently, unusual features of the cosmic microwave background, a ’snapshot’ of the early universe, have raised issues with our understanding of the Big Bang. A Caltech team has shown how we might fix our theories. They suggest that there might have been an asymmetry in the energy that once powered the big bang. If this is correct, anomalies in the CMB may be traces of structure from a time before our explosive beginnings.

True to form, when a discussion appeared on Hacker News I rushed to comment, and this article erupted from that attempt. The current scientific understanding of our cosmic origins is a mystery to the public at large, but it was only after I noticed the bewilderment of my fellow hackers that I realized how poor a job we scientists have done in conveying the motivation behind our discoveries.

This article represents an attempt to replace that sense of bewilderment with that of wonder. I want more than to explain what cosmologists believe. I want give people a deep sense of why we believe it, of how we’ve come to our current understanding, and of why we care.

Look close, and it seems the universe is lopsided.

The cosmic microwave background (CMB) is like a snapshot of the early universe. It was once all hot plasma, gas so hot that the atoms inside it were broken up. Because it was hot, it emitted light. Because it was dense, it was opaque: the light emitted couldn’t just pass through, instead it had to bounce around. But once cool enough, the universe became transparent: all the light could now travel freely. It was as if the photographic shutter of the universe was lifted.

The Cosmic Microwave Background Radiation (from WMAP). False Color/ The light from this moment became the cosmic microwave background radiation. Because the universe seemed to have cooled at almost exactly the same time everywhere, the CMB is, unlike almost everything else in astronomy1, a picture of the entire universe at almost exactly the same moment in time. It is the best picture we have of the structure of the early universe.

The universe appears to have expanded evenly since then. We know it’s expanding now. Light is like a wave. Since the speed of light is constant, an illuminated object moving towards us has its wave crests squish together, turning bluer, and an object moving away from us has the distance between crests expand, turning more red. This is called a red shift. Since he knew the colors of certain celestial objects, Edwin Hubble was able to observe that the further something is from us, the more red-shifted its light, and therefore the faster it is speeding away.

Since we know that the early universe was hot, dense and small, and we know now that it’s cooler, sparse, big, and expanding, we can reasonably deduce that, long ago, there was a Big Bang. The universe exploded.

The Crab Nebula Strikingly, the CMB is almost the same everywhere you look. There are minor fluctuations, but even they seem to have the same distribution everywhere. The CMB, our best picture of the early universe, is extraordinarily smooth. It is one of the smoothest things ever observed in nature. This might not seem like a mystery. You might imagine that anything expanding, hot and dense would look roughly the same in all directions. It needn’t. Nebulae are formed by exploding stars, and they aren’t particularly smooth. In fact, in nature, it would seem, more often than not, that explosions are messy.

In 1981, Alan Guth suggested what might be called a ‘recipe for a universe’: inflation theory. Until then, nobody had come up with any good ideas for why the universe was so smooth and even. It is as if God2 had pressed the entire universe with a cosmic clothes iron.

Guth said, suppose you started with pretty much any initial universe. Suppose you also had an extremely strong, extremely smooth field of energy. If this field started dumping energy into the rest of the universe, it would also evenly expand space itself.3 The universe would undergo a period of exponential expansion — inflation — having the effect of flattening and smoothing the rest of the universe. Inflation is God’s clothing iron.

A flat, smooth universe isn’t the only thing that inflation predicted. For example, at small physical scales, quantum mechanical fluctuations persist. During inflation these fluctuations are blown up as well, and these would seed, almost entirely, the cosmological structure of the universe. We see these fluctuations in the CMB. According to inflation, they are tiny quantum fluctuations blown up to a cosmic scale. They are, quite literally, the ancestors of our galaxies.

It wasn’t just that there were fluctuations. Inflation theory predicted a very specific distribution and type4. When people finally had the technological capability to check, that’s just what they found. The universe appeared, at a cosmic scale, astonishingly consistent with this simple theory. Yet recently our observational capacities have improved. A CMB survey called WMAP has uncovered several surprising and unexplained features, not all of which fit well with the our previous inflation theories.

If you divided the sky in half by tracing the orbit of the earth around the sun5, and compared, in each half, the size of big fluctuations, those between 3 and 5 degrees wide, you would come to the conclusion that one side has fluctuations outweighing the other by an alarmingly large amount. One side of the universe is bumpier than the other. Moreover, the difference is larger than would be accounted for by randomness, at least 99 times out of 100.6

This asymmetry looks real. It has been checked against every known experimental error and background effect astrophysicists have been able to think of. And if it is real, our previous inflation theories, with one field of energy to inflate the early universe, won’t work. They can’t account for this anomaly.

The authors Erickcek, Kamionkowski, and Carroll don’t merely point out this problem. They posit a solution. They describe another inflation model, consistent with our new observations. They suggest the universe had not one, but many fields of universe inflating energy. There’s just one problem. At least one of these fields needs to be asymmetric.

Where could such an asymmetry come from? It is possible that we’ll never know. Cosmology offers us the hope of uncovering consistent, compelling stories of our origins. Thousands of independent observations fit neatly in cosmology’s book. But while we may discover a few lost pages from our first chapters, we may never know all reasons why our book was written in the first place.7

Nevertheless, the authors make an exciting point. Wherever the asymmetry in the inflation field came from, it must have existed before inflation. It must have existed before the big bang. We had once imagined that time before our explosive beginnings would forever remain a mystery. Yet hidden in the CMB are hints of times earlier still. In this wonderful piece of work, the authors carefully consider what anomalies in the CMB could mean. And in the process, they may have discovered a way to look farther into the past than ever before.

Notes:

[1] – Since light moves at a finite speed, when we see something far away, we’re seeing light emitted in the past. What we see of something a light-year away is (at least) one year old.

[2] – I mean ‘God’ here as in a figure of speech. Feel free to substitute ‘Mother Nature’, ‘Allah’, or the ‘Flying Spaghetti Monster’ while reading.

[3] – What does it mean, exactly, for energy to expand space itself? It’s roughly analogous to blowing up a balloon. We know that the gravity of the universe, just like the elastic outside walls of a balloon, pull its contents inward. In a balloon, air pressure pushes against that inward force of the walls. During cosmic inflation, the inflationary force pushes against gravity. There’s one important difference though. We don’t actually know what the inflationary force is. Air blows up our balloons, but we have few clues as to what blew up the universe.

[4] – The quantum fluctuations predicted by inflation follow a nearly-scale-invariant random Gaussian distribution. These fluctuations show up in the CMB, and for the most part follow these predictions pretty closely.

[5] – The line dividing the two halves of the sky here is called the ecliptic.

[6] – Formally, this statement is true at at least the 99% confidence level.

[7] – There are some questions forever beyond our grasp. Even if we knew from where the Big Bang had come, we could always probe further, and ask where that came from.

Advice to the Bright and Young

•May 15, 2008 • 10 Comments

Max.

An article on one bright young man, Moshe, recently appeared on Hacker News. For a long time I’ve been meaning to write about the subject, and what was to be a simple comment morphed into this essay.

The story of educational acceleration is an old one. Curious, bright children learn and explore rapidly on their own, and interactively with their parents. The world is like a playground for the growing mind. The child takes in everything. Eventually, these children find themselves mired in school’s morass. There are new adventures: more kids, older kids, a new environment. Yet kept in one place, individual attention of parents replaced by lectures from often overtaxed and uninterested teachers, their minds are left to go fallow. While some of school is new, and quite enjoyable, boredom and obedience, for the curious child, is torturous, a fact which lucky children and mindful parents come to confront.

Alternatives appear: skipping grades, dropping out, home-schooling, gifted programs, science fairs, participating in the popularity game, sports, focusing on musical or athletic achievement, playing hookie, becoming jaded.

After entering junior high I pretty much stopped responding to the world at large. Life rapidly degenerated. I quickly dropped out, and luckily my parents didn’t make me go back. At that time both of my parents were very busy with work, and so homeschooling couldn’t work for long. We discovered that college was much cheaper than private school, which didn’t seem very good anyway. We argued my way in.

Continue reading ‘Advice to the Bright and Young’

One Response to Rejection

•April 11, 2008 • 19 Comments

Those who’ve spent time with me over the past few months know both how absorbed I’ve been in the catalysis of our startup, and how poor I am at concealing my admiration for YCombinator. We had poured startling effort into building our product, honing our idea, refining our pitch. But our focus was, perhaps embarrassingly, almost entirely toward a single goal. Getting into YCombinator. It was constantly in our minds. Ample encouragement followed months of work. On occasion, I could be found exclaiming my certainty to the universe. We’ll get in. We’ll make sure of it.

The letter arrived silently. I’m embarrassed to admit that my body read like a chapter on the stages of grief. Shock. My stomach churned as I turned inward. Admonishments ‘not to take rejection personally’ meekly confronted universal doubts. Egos struggled against a rethinking of everything. Hours of discussion lapsed. Plans of what to do were floated, accepted, rejected, forgotten. Night passed to sunrise before sleep. Denial. I woke up recalling a story of one rejection mailed out accidentally, to a startup later to succeed. Thoughts strayed from their success – all I could register was the possibility of a mix-up. Anger. I stewed. ‘It doesn’t matter what they think. I know where they’re coming from, I know what they must think of us. They’re wrong! And we don’t need them anyway.’

Finally, acceptance.

Continue reading ‘One Response to Rejection’

On Naming Startups (with Ruby)

•March 1, 2008 • 7 Comments

The name problem has been with our band of hackers for a while. At least we were not alone: judging by the perennial popularity of the topic on Hacker News, it would seem to stump many.

On such matters, an appeal to a higher power is appropriate. My friends use a variety of divination techniques, such as flipping a coin, tarot, or peyote. I, however, found myself reading an infrequently referenced blog post by Paul Graham (an orphan of the collapse of infogami).

“…as happened with lofts, the features that initially repelled people, like rough concrete walls, have now become a badge of coolness. Weird names are now cool, if they’re the right kind of weird. Nothing could be less cool, at this point, than calling a startup “cool.com.” A company with a name like that could not have arisen organically. “Cool.com” smells of a media conglomerate trying to create a web spinoff.”

and

“My favorite recent startup name is probably Writely. It looks so natural that even though it isn’t a word, you feel it should be. Anyone thoughtful enough to come up with a name like that is probably going to have good software.”

Even ordinary people have an extraordinary ability to glark meaning for a word newly encountered. A word that feels natural enough to exist in speech (‘I’m feeling ever so writely’) is quite a goal to aim for. People are sure to remember that.

I threw together a ruby script to create domain names from some simple rules and then check whois. I multithreaded it for throughput. (Ruby threads are easily invoked but apparently the threading system is not so powerful.)

Continue reading ‘On Naming Startups (with Ruby)’

Blue Eyed Islanders. (A Logic Puzzle)

•February 10, 2008 • 5 Comments

Terry Tao’s recent post on a classic logical puzzle has seeded a bloom of activity in the nerdsphere. A friend of mine introduced it to me over mugs of steamed milk in the graduate college coffee house; I was telling him of recent work I’d been doing on the soundness of emergence of Nash equilibria, and where sufficient conditions arise in real life.1 It was an enjoyable conversation, though I wasn’t afforded the luxury of working it out for myself. If you’d like to struggle through the problem on your own, read Terry’s post, and only return after the break when you think you’ve solved it. (why am I writing this? It started as a comment on news.ycombinator.com, and I couldn’t help myself)

This problem is subtle, and wording is important, so I’ve reproduced the statement from Terry’s blog (emphasis mine) [editor's note: Terry didn't intend for a particular subtlety, and so has reworded his main post. He says that it had an 'unexpectedly interesting subtlety in its formulation, but was not the puzzle I had actually intended to write'. He's posted the original here]:

There is an island upon which a tribe resides. The tribe consists of 1000 people, 100 of which are blue-eyed and 900 of which are brown-eyed. Yet, their religion forbids them to know their own eye color, or even to discuss the topic; thus, each resident can (and does) see the eye colors of all other residents, but has no way of discovering his or her own (there are no reflective surfaces). If a tribesperson does discover his or her own eye color, then their religion compels them to commit ritual suicide at noon the following day in the village square for all to witness. All the tribespeople are highly logical and highly devout, and they all know that each other is also highly logical and highly devout.

One day, a blue-eyed foreigner visits to the island and wins the complete trust of the tribe.

One evening, he addresses the entire tribe to thank them for their hospitality.

However, not knowing the customs, the foreigner makes the mistake of mentioning eye color in his address, remarking “how unusual it is to see another blue-eyed person like myself in this region of the world”.

What effect, if anything, does this faux pas have on the tribe?

Continue reading ‘Blue Eyed Islanders. (A Logic Puzzle)’

Incompleteness and Halting. Gödel and Turing.

•January 28, 2008 • 5 Comments

The following occurred to me on a run about two years ago:

It’s not given much press, but the the Halting Problem is intimately related to Gödel’s First Incompleteness Theorem. Indeed it produces it as a correllary. Historically, Gödel’s incompleteness results were proved by hacking arithmetic into a Turing complete system, and this is still how they’re explained today.

There’s a one-to-one bijection between computability of a function and provability of a statement. Hence, the short, and generally accessible proof that the Halting problem is not in general computable for an arbitrary input is also a proof of the ‘most important, surprising result in logic’, namely, that some results, which have may have a perfectly valid truth-value outside a system, cannot be proven within it. One only needs the notion of a computer to follow this line of thinking, which is, in essence, what Gödel did. But the Halting problem is much easier to grasp. I’ve had children understand it, though it does take some walking through!

The interesting thing about the Halting problem is that it’s unsolvable in full generality, independent of whatever special capabilities the system has available. To see this clearly, consider the proof.

Question: Does there exist a (halting) program H which, given any program P, figure out if it would halt, for any input I?

Assume there exists such a program H. Construct a program T as follows.

(Program P, Input I) => (Boolean Halts):
if H(P,I) is true run forever
otherwise halt

Now, call T on itself, with itself as an input. Our assumption presupposes that H always halts. If T would halt on input T, then T will run forever. And if T would run forever on input T, then T would halt. This is a contradiction, so no such program H would exist.

Continue reading ‘Incompleteness and Halting. Gödel and Turing.’

On Outliers: What they represent, and why the Central Limit Theorem is Typically Off.

•January 28, 2008 • 4 Comments

A Bell Curve

The central limit theorem states that if you have many small, independent, random variables, then their sum is distributed approximately as a bell curve. Strikingly, almost everything is made up of many small parts, and these parts don’t tend to influence each other very much.

So much of what can measure seems to fit a bell curve. This is why the normal distribution works. Because this assumption tends to work well, it is usually taken as a matter of course. Students are taught it, lecturers preach it, researchers apply it, and startlingly few stop to question it.

Suppose the variables are not small, or suppose they’re not independent. Suppose, under certain conditions, the value of one variable would seriously effect another. Suppose we’re talking about the buildup of snow on a mountain slope. Most of the time, snowflakes can gradually build, without significant effect. But once enough builds, you don’t find snowflakes resting calmly upon a drift. What you find is an avalanche.

Violent nonlinearities...

The sum total of snowflake movement isn’t what we might expect. The snowflakes on the top used to be lightly packed by the new, gradually coming down. The snowflakes on the bottom used to just sit there. But they’re not just sitting there. They’re moving fast, and they’re moving down.

Continue reading ‘On Outliers: What they represent, and why the Central Limit Theorem is Typically Off.’

General Problem Solving Strategies (for programmers)

•January 28, 2008 • 4 Comments

Konstantin Lopyrev writes (on the TopCoder forums):

“Everyone always discusses algorithms and such on this forum. However, there is one trait that is perhaps equally important as knowledge of algorithms – good problem solving strategy. By that I mean not jumping into solutions too quickly, thinking through everything before writing any code and such. Can anyone share what they’ve figured out that helps them? I’ve noticed that I have several flaws when I solve problems in general. I am too quick to start writing code. Also, I don’t think through my algorithms thoroughly. Also, there are many other things. Does anyone have similar flaws that they’ve already passed. If you have, please share your strategies for getting better at problem solving in general. Anything would be helpful, since I have the TCHS tournament coming up and I want to go home with some sort of prize.”

I wrote back, on May 10th ‘07:

One thing I want to start doing is to make estimates of the implementation time of parts of my solution, how difficult each will be to code correctly/debug, and how sure I am that my method will work in the first place.

I’ve gotten very good at making estimates of time and space constraints. Now they are second nature. But knowing how best to solve a problem practically is still another area which I need to improve.

For example, for the last hard problem on TopCoder (circa May 9th, sorry, you’ll have to register to see it). I broke the problem down. Six steps toward victory! Five of those steps were really quite simple, though a few bugs crept in. The sixth was brutal — I could solve the problem by implementing a complicated comparison function object which recursively called itself on sorted sets of children. It was really crazy. To get it all working properly took nearly eight hours after the match.

Each bug, if not isolated, tends to increase debugging time by something like the square of the number of interacting components: the rate at which the number of dependencies would grow. Perhaps it would be simpler to either independently verify each component (by having pre-written tested functions or methods), or to organize the program, inasmuch as possible, as a linear pipeline or other simple acyclic graph. In fact, one might take a few seconds just to sketch the program idea out — how strongly coupled is it? What can I be sure of? How much of a pain will it be to debug? Can I make it less so? If not, can I anticipate debugging requirements and build in information gathering tools?

Continue reading ‘General Problem Solving Strategies (for programmers)’

Quantum Field… Finance?

•January 28, 2008 • Leave a Comment

One morning around the graduate college dining hall, there was a gathering of physicists, finance students, and economists. The physicists are always quite amazed by those people who decide to forgo the life of the ivory tower, and choose to strike out into the real world, and so could not be kept from asking what the economists actually did. Furthermore, we could not be kept from wondering aloud what type of mathematical models they built and polished, and whether any of them had a physical interpretation.

One of the economists scratched his head, drew a sip of black coffee from his porcelain cup, and mumbled something about how a large proportion of the physics department of Harvard University was hired by a trading company, with the lure of riches beyond the pale of the meager imaginings of the physicists (“you mean I can afford a house?!”).

Continue reading ‘Quantum Field… Finance?’

Third Places

•January 28, 2008 • 10 Comments

Caffe Strada

Tonight it’s winter in Berkeley. 53 degrees and raining, and outdoors, warmed by a heat-lamp, sheltered by an awning. I draw spiced apple cider through my lips. Classical music plays. An earbudded minority vote silently with their ears. Old men watch hooded students roll down the hills towards Telegraph Ave, Berkeley’s epicenter of hippiedom. Moist, newspapers ink the hands of activists, busily plotting the victories in the years long struggle to ’save the oaks’. A young man lids a drink and smiles at me. Separated by glass, headphones, and 12 feet, I smile back. We wave.

There’s something magical about this place.

I don’t know anyone here. To arrive I flew four thousand miles from my place of growth. This place isn’t home. Yet there are few places that attract me so strongly. Modern life has been made private. And in doing so, life’s become a little lonely.

Builders of great cities have long understood that life would, but for misfortune, consist of more than work and one’s home. The vibrancy, energy, and community grown in what are sometimes called ‘third places’ played part in much of the world’s social, political and intellectual revolutions. The roles that the Roman forae, French salons, and English learned societies played in scholarship has been tremendous, as has been the influence of American chautauquas, worker’s taverns, and artist’s ghettos in social and political spheres. These public, accessible, talkative, comfortable playful places are magnets for folks of many stripes. Creativity can thrive there. Unconstrained by work’s implied unity of purpose, and decoupled from the tight bonds around one’s family and home, third places give marginal people, ideas, and voices room to grow, people to hear them, perspectives to challenge them, and food to help keep the conversation going.

Continue reading ‘Third Places’