Life sciences vs Computer Sciences - a challenge for the 21st century

Investor Peter Thiel captures the core difference between bio and computer tech in his recent interview to MTR:

This goes back to that famous Bill Gates line, where he said he liked programming computers as a kid because they always did what he told them to. They would never do anything different. A big difference between biology and software is that software does what it is told, and biology doesn’t.

One of the challenges with biotechnology generally is that biology feels too complicated and too random. It feels like there are too many things that can go wrong. You do this one little experiment and you can get a good result. But then there are five other contingencies that have to work the right way as well. I think that creates a world where the researchers, the scientists, and the entrepreneurs that start companies don’t really feel that they have agency.

Unlike computer science, biology doesn't have the equivalent of the Church-Turing thesis that, essentially, guarantees an implementability of a valid algorithm. The success of Silicon Valley is built on top this important discovery of the 20th century. That is, once a "computation" entrepreneur, either in software or hardware, finds a way to express his useful idea in an algorithmic way, he or she can be sure that it will work, provided the computational power, storage, and networking capacity grow exponentially. Most famously, Larry Ellison created his Relational Database business in mid-1970s when people did not understand implications of the Moore's Law yet.

Biology is different. Vernon Vinge, a science fiction writer, aptly calls our future successes in medicine "A Minefield Made in Heaven" because it's hard to predict the specific locations of magical "mines" that we are going to discover and cure various diseases.

Peter Thiel uses word "random" to describe biology; but from a practical perspective it's actually worse than that. If it were random we could use known randomization techniques from computer science and make new biological discoveries by almost brute force. We can't. Therefore, I'd rather use a different term – arbitrary, and there's no algorithm for generating useful arbitrariness yet - only human ingenuity.

The good news is that some of the life sciences fields are compatible with computation. We are going to make a lot of progress in areas where we can hook up analog biological experiments to the exponentially growing computing platforms. Diagnostics and pattern matching for known problems seem to be the most promising field.

tags: biology, innovation, science, technology, silicon valley

Creating a new Silicon Valley is more difficult than the original Silicon Valley

The original Silicon Valley started in the late 1950s. Since then, many countries tried to reproduce Silicon Valley success, but only Israel could sustain high-tech development over two consecutive innovation waves.

One of the major difficulties in creating a new Silicon Valley would be to talent retention. That is, it is easier than ever for a talented entrepreneur, engineer, or scientist to move to the greater Silicon Valley, which today includes San Francisco. A recent infographic from Bloomberg shows the impact immigrants make on high-tech innovation in the Bay Area (click to enlarge).

In 2010, Asian Americans became the majority of the high-tech workforce in the valley. One third of SV startups are founded by Indian Americans.

tags: silicon valley, innovation, demographics, tool, source

Scalability Problem of the Day: Neural Networks

Deep Learning is the next frontier in computer science. After some initial breakthroughs, scientists and engineers are running into a major scalability problem: increasing the number of neurons doesn't improve a neural network's performance.

(MTR, 5/21/14) We found that if you put a lot of GPUs [specialized graphics processors] together we could make a much bigger neural network—10 billion nodes, with 16 machines instead of 1,000.

We used that same benchmark [images from YouTube videos] that the Google team did. But even though we could train a much larger neural net, we didn’t necessarily get a better cat detector. Right now we can run neural networks that are larger than we know what to do with.

This is a typical situation in Silicon Valley (I described in an earlier post). We are at a point where Machine 1 (exponential growth in computing power) is ahead of Machine 2 (Applications). Most likely, the next S-curve, i.e. a new growth cycle, will begin within the next 5-7 years.

tags: problem, scalability, constraint, machine1, machine2, silicon valley,

Lab Notebook: How to explain Silicon Valley to a Martian.

Explaining Silicon Valley to humans is quite satisfying but can be very hard. John Kelley and I spend an entire academic quarter on this task and we manage to cover only the basics. Human students differ significantly in their cognitive, motivational, and emotional backgrounds. Some of them are locals, who know a lot about hi-tech businesses; some are newcomers, who want to understand better the place they happen to be visiting. Some are interested in personal stories of inventors and entrepreneurs; some are eager to learn the core principles behind SV-style innovations. The list of student interests and motivations could go on and on, but inevitably, our explanations of how Silicon Valley works are subjective.

On the other hand, to a Martian, all human individuals, how they live and do business, look pretty much the same. To explain Silicon Valley to a Martian, we would have to rely on objective truths, not our human emotions and motivations. Therefore, universal mathematical principles would be a logical place to start.

In the 1930s, a human mathematician, Alan Turing, came up with an abstract model for an automatic machine that can execute computational instructions. According to the model, the abstract machine is fed with an infinite tape with symbols on it. The machine can move the tape and manipulate symbols: read, write, or erase them. The state of the machine and its follow-up manipulations depend on its earlier interaction with the symbols. A Universal Turing Machine can simulate any Turing Machine with any symbolic input.

Silicon Valley is a place on Earth, where people know how to make physical machines that can perform operations of a Universal Turing Machine. More importantly, they know how to make such machines run exponentially better over time. That is, humans in Silicon Valley can double computational speed of the machines every 1.5 Earth years. We humans call this technology pattern "The Moore's Law." It held true for the last 50 years, making our computations roughly 1,000,000,000,000,000 faster in the process.

In Silicon Valley, humans also learned how to grow data storage capacity even faster than computations. We humans call this "The Kryder's Law." To top it off, in Silicon Valley humans learned how to connect all kinds of exponentially better performing computational and memory devices with exponentially faster networks. We humans sometimes call this pattern the "Nielsen's Law." The interaction of these three laws made Silicon Valley a computational technology powerhouse. To simplify our conversation, I will call this amazing, exponentially growing technological capacity "Silicon Valley Machine 1" or "Machine 1."

Examples of Machine 1 are: the Integrated Circuit, the magnetic hard drive, the Ethernet, the Internet, the Google Spanner, etc.

Note that the Machine 1 by itself is meaningless to humans. Although it can shuffle symbols back and forth exponentially faster, somebody needs to figure out how to make this symbol shuffling useful. Fortunately for humans in Silicon Valley, mathematicians back in the 1930s formulated another principle — the Church-Turing thesis. Based on the thesis, we can explain to an ordinary Martian that any computable logic, no matter how long or complicated, can be executed by the Universal Turing Machine, implemented either in hardware or software or both. Therefore, if somebody (let's call him Steve Jobs) comes up with an idea for a logically consistent, useful computing device or application, it is guaranteed to work. Guess what, we say to a curious Martian, smart humans in Silicon Valley figured out a way to regularly come up with useful applications. To simplify our conversation, I will call this ingenuous creative capability "Silicon Valley Machine 2" or "Machine 2."

Examples of Machine 2: computer gaming, the spreadsheet, digital movies, the world-wide web, the iPhone, Google Search, social networking, etc.

Over the last 50 years, the folks in Silicon Valley learned how to hook up the ingenuous Machine 2 to the exponentially growing Machine 1. As a result, products, services, and experiences produced in Silicon Valley are becoming — year after year — exponentially faster and more useful to other humans. This is it. The success of Silicon Valley is based on fundamental mathematical truths as well as our ability to make abstract concepts objectively real and subjectively useful to other humans.

tags: innovation, book, machine1, machine2, silicon valley, 

Innovation: Silicon Valley vs Wall Street

In preparation for The Greatest Innovations of Silicon Valley (BUS 117), I read a lot of papers about innovation and growth. Here's a good sentence from Bresnahan and Trajtenberg NBER/Journal of Econometrics (1995) paper "General Purpose Technology: 'Engines of Growth?'"

Economists have known for a long time that technical change is the single most important force driving the secular process of growth (Abramovitz, 1956; Solow, 1957).

From a purely financial (risk) perspective, Silicon Valley is an engine of growth because it is really good at developing uncorrelated assets using new technologies. That is, while everybody is happy with proven technologies, the creative crowd in the valley is willing to bet on something totally new. As the result — and in accordance with both the financial and economic growth theories — Silicon Valley innovations stimulate growth and at the same time reduces overall financial risks.

By contrast, Wall Street innovations create a trade-off between growth and risks. Therefore, when the Wall Street creates a lot of innovation the financial system becomes prone to disasters (or as Nassim Taleb calls them "Black Swans").

tags: innovation, theory, technology, silicon valley, course, stanford

Tricking people into greatness.

In preparation for The Greatest Innovations of Silicon Valley, I'm reading up on computer history in the San Francisco Bay area. The video games industry was born in the early 1970s with the emergence of Atari. In 1972, Nolan Bushnell co-founded the company in Sunnyvale, CA. Their main business was servicing coin-operated arcade game machines around the valley.

At the same time, he wanted to develop and sell his own machines. For that purpose he hired engineer Al Alcorn, who ended up designing one of the best arcade game machines ever - the PONG. The machine was a result of a ruse Bushnell played to test Alcorn's engineering abilities. Here's how "The Ultimate History of Video Games," by Steven Kent describes this accident of greatness,

Shortly after hiring Alcorn, Bushnell gave him his first project. Bushnell revealed that he had just signed a contract with General Electric to design a home electronic game based on ping-pong. The game should be very simple to play—“one ball, two paddles, and a score…. Nothing else on the screen.”

Bushnell had made up the entire story. He had not signed a contract or even entered into any discussions with General Electric. In truth, Bushnell wanted to get Alcorn familiar with the process of making games while he designed a more substantial project.
found out later this was simply an exercise that Nolan gave me because it was the simplest game that he could think of. He didn’t think it had any play value. He believed that the next winning game was going to be something more complex than Computer Space, not something simpler.

An excerpt from an interview with Alcorn in the same book,

Nolan didn’t want to tell me that because it wouldn’t motivate me to try hard. He was just going to dispose of it anyway.

Pong proved to be extremely successful and put Atari on the world-wide technology map. One of the low-ranking employees at Atari later co-founded a famous "fruit" company. The guy's name was Steve Jobs.


tags: innovation, silicon valley, stanford, luck, history