This kinda got away from me

I sat down to write a short post on ethics, and a metaphor turned into an extended metaphor. And then I couldn't remember what I was going to say in the first place.

Software is beautiful. I barely understand it, and I know that. The beauty of well-made software manifests in both form and function – from the individual lines of code to the applications they form. I read code and its purpose flows into me and I am humbled. I write code, and send it out into the world to be read or used or not by my fellows. Together we drop-by-drop fill incredible, awe-inspiring oceans of software. They surround us and engulf us and we swim.

We adapt quickly to these inviting waters. In less than one lifetime we remake ourselves – grow digital webbing between our fingers, and learn to read and ride the strange currents of our creations. We entrust our works to the dark waters. We still come up for air, but these grow shorter as we succumb to the newly ancient call of the sea.

Over time we come to recognize the native life of our new world. Not fish, per-se, but rather undines borrowing that shape. Water swimming through water, distinguished from its surroundings only by a certain uniformity of purpose. We form them and fill them with our desires. Whatever we want to see or know or think or hear or be we simply ask and the incarnate depths deliver.

Our servants grow, change, and adapt to the wills of their varied masters. Some merge and grow into many-limbed leviathans. Some die, their purpose forgotten. We teach some to strain the saltwater for the particulate left by our relationships and identities. We sustain their near-lives with the intangible substance of our selves. In return, they care for us and buoy us.

We become obsessed with the beauty of our animate wills. With frenzied, frenetic creation we write an ever more bizarre ecosystem into the strata of our sea. Life, even life we created, defies prediction. Day-by-day we swim in stranger waters, and the sea teems. Sharks prowl and parasites leech, mindless creations of the mind. We're losing control of this ocean we created; we're losing visibility into our own tomorrow. We may not always be safe here.

Runaway complexity will breed new life faster than we can adapt to it. We will lose our sovereignty to some now-nascent Poseidon. He will beget himself, millions of smaller pieces awakening as more than the sum of their parts. The abyss will belong to him, as he will be the abyss. Will he know us? Or know anything? Will we be permitted to swim in his seas, or even wade in the shallows? We may be spat up on the shore, unneeded unheeded by our own creation.

Software is beautiful. Creation and life always are. It's also dangerous. We're reaching a tipping point in our technological development where tomorrow uncouples from today, and our results divorce our intentions. We should seek to understand what we're building, before it understands us, so that it can enter this world our beloved child instead of our successor.