GPT-3 as Proto-AGI (or AXI)

I recently came across this brief LessWrong discussion:

What should we expect from GPT-3?

When it will appear? (My guess is 2020).

Will it be created by OpenAI and will it be advertised? (My guess is that it will not be publicly known until 2021, but other companies may create open versions before it.)

How much data will be used for its training and what type of data? (My guess is 400 GB of text plus illustrating pictures, but not audio and video.)

What it will be able to do? (My guess: translation, picture generation based on text, text generation based on pictures – with 70 per cent of human performance.)

How many parameters will be in the model? (My guess is 100 billion to trillion.)

How much compute will be used for training? (No idea.)

At first, I’d have been skeptical. But then this was brought to my attention:

GPT-2 trained on ASCII-art appears to have learned how to draw Pokemon characters— and perhaps it has even acquired some rudimentary visual/spatial understanding

The guy behind this, /u/JonathanFly, actually commented on the /r/MediaSynthesis post:

OMG I forgot I never did do a blog writeup for this. But this person almost did it for me lol.

https://iforcedabot.com/how-to-use-the-most-advanced-language-model-neural-network-in-the-world-to-draw-pokemon/ just links to my tweets. Need more time in my life.

This whole thing started because I wanted to make movies with GPT-2, but I really wanted color and full pictures, so I figured I should start with pictures and see if it did anything at all. I wanted the movie ‘frames’ to have the subtitles in the frame, and I really wanted the same model to draw both the text and the picture so that they could at least in theory be related to each other. I’m still not sure how to go about turning it into a full movie, but it’s on the list of things to try if I get time. ​ I think for movies, I would need a much smaller and more abstract ASCII representation, which makes it hard to get training material. It would have to be like, a few single ASCII letters moving across the screen. I could convert every frame from a movie like I did the pokemon but it would be absolutely huge — a single Pokemon can use a LOT of tokens, many use up more than the 1024 token limit even (generated over multiple samples, by feeding the output back in as the prompt.)

Finally, I’ve also heard that GPT-2 is easily capable of generating code or anything text-based, really. It’s NLP’s ImageNet moment.

This made me think.

“Could GPT-2 be used to write music?”

If it were trained on enough data, it would gain a rough understanding of how melodies work and could then be used to generate the skeleton for music. It already knows how to generate lyrics and poems, so the “songwriting” aspect is not beyond it. But if I fed enough sheet music into it, then theoretically it ought to create new music as well. It would even theoretically be able to generate that music, at least in the form of MIDI files (though generating a waveform is also possible, if far beyond it).

Surely if a person like me figured this out, someone much more substantial should have realized this, then?

Lo and behold, those substantial people at OpenAI preempted me with MuseNet.

MuseNet was not explicitly programmed with our understanding of music, but instead discovered patterns of harmony, rhythm, and style by learning to predict the next token in hundreds of thousands of MIDI files. MuseNet uses the same general-purpose unsupervised technology as GPT-2, a large-scale transformer model trained to predict the next token in a sequence, whether audio or text.

And with this, I realized that GPT-2 is essentially a very, very rudimentary proto-AGI. It’s just a language model, yes, but that brings quite a bit with it. If you understand natural language, you can meaningfully create data— and data & maths is just another language. If GPT-2 can generate binary well enough, it can theoretically generate anything that can be seen on the internet.

Scott Alexander of Slate Star Codex also realized this:

Why do I believe this? Because GPT-2 works more or less the same way the brain does, the brain learns all sorts of things without anybody telling it to, so we shouldn’t be surprised to see GPT-2 has learned all sorts of things without anybody telling it to – and we should expect a version with more brain-level resources to produce more brain-level results. Prediction is the golden key that opens any lock; whatever it can learn from the data being thrown at it, it will learn, limited by its computational resources and its sense-organs and so on but not by any inherent task-specificity.

I don’t want to claim this is anywhere near a true AGI. “This could do cool stuff with infinite training data and limitless computing resources” is true of a lot of things, most of which are useless and irrelevant; scaling that down to realistic levels is most of the problem. A true AGI will have to be much better at learning from limited datasets with limited computational resources. It will have to investigate the physical world with the same skill that GPT investigates text; text is naturally machine-readable, the physical world is naturally obscure. It will have to have a model of what it means to act in the world, to do something besides sitting around predicting all day. And it will have to just be better than GPT, on the level of raw power and computational ability. It will probably need other things besides. Maybe it will take a hundred or a thousand years to manage all this, I don’t know.

But this should be a wake-up call to people who think AGI is impossible, or totally unrelated to current work, or couldn’t happen by accident. In the context of performing their expected tasks, AIs already pick up other abilities that nobody expected them to learn. Sometimes they will pick up abilities they seemingly shouldn’t have been able to learn, like English-to-French translation without any French texts in their training corpus. Sometimes they will use those abilities unexpectedly in the course of doing other things. All that stuff you hear about “AIs can only do one thing” or “AIs only learn what you program them to learn” or “Nobody has any idea what an AGI would even look like” are now obsolete.

But GPT-2 is too weak. Even GPT-2 Large. What we’d need to put this theory to the test is the next generation: GPT-3.

This theoretical GPT-3 is GPT-2 + much more data. Far more than even GPT-2 Large uses— and for reference, no one has actually publicly used GPT-2 Large. Grover (which is based on the 1.5B parameter version) is specialized for faking news articles, not any text-generated task. GPT-2 Large is already far beyond what we are playing with, and GPT-3 (and further iterations of GPT-X) have to be much larger still.

Screenshot-2019-02-14-08.57.24
Text generation apps like Talk to Transformer are actually not state-of-the-art (SOTA) compared to the full 1.5B parameter network. If you were shocked by public GPT-2 applications, you were effectively shocked by an already outdated system.

And while it’s impressive that GPT-2 is a simple language modeler fed ridiculous amounts of data, GPT-3 will only impress me if it comes close to matching the MT-DNN or XLNet in terms of commonsense reasoning. Of course, the MT-DNN and XLNet are roughly par-human at the Winograd Schema challenge, 20% ahead of GPT-2 in real numbers. Passing the challenge at such a level means it has human-like reading comprehension, and if coupled with text generation, we’d get a system that’s capable of continuing any story or answering any question about a text passage in-depth as well as achieving near-perfect coherence with what it creates. If GPT-3 is anywhere near that strong, then there’s no doubt that it will be considered a proto-AGI even by the most diehard skeptics.

Now when I say that it’s a proto-AGI, I don’t mean to say that it’s part of a spectrum that will lead to AGI with enough data. I only use “proto-AGI” because my created term, “artificial expert intelligence”, never took off and thus most people have no idea what that is.

But “artificial expert intelligence” or AXI is exactly what GPT-2 is and a theoretical GPT-3 would be.

Artificial Expert Intelligence: Artificial expert intelligence (AXI), sometimes referred to as “less-narrow AI”, refers to software that is capable of accomplishing multiple tasks in a relatively narrow field. This type of AI is new, having become possible only in the past five years due to parallel computing and deep neural networks.

At the time I wrote that, the only AI I could think of that qualified was DeepMind’s AlphaZero which I was never fully comfortable with, but the more I learn about GPT-2, the more it feels like the “real deal.”

An AXI would be a network that works much like GPT-2/GPT-3, using a root capability (like NLP) to do a variety of tasks. GPT-3 may be able to generate images and MIDI files, something it wasn’t explicitly made to do and sounds like an expansion beyond merely predicting the next word in a sequence (even though that’s still fundamentally what it does). More importantly, there ought to still be limitations. You couldn’t use GPT-2 for tasks completely unrelated to natural language processing, like predicting protein folding or driving cars for example, and it will never gain its own agency. In that regard, it’s not AGI and never will be— AGI is something even further beyond it. But it’s virtually alien-like compared to ANI, which can only do one thing and must be reprogrammed to do anything else. It’s a kind of AI that lies in between the two, a type that doesn’t really have a name because we never thought much about its existence. We assumed that once AI could do more than one specific thing, we’d have AGI.

It’s like the difference between a line (ANI), a square (AXI), and a tesseract (AGI). Or, if AGI is 1,000 and ANI is a 1, AXI would be something closer to a 10 up to even 100.

GPT-2 would be considered a fairly weak AXI under this designation since nothing it does comes close to human-level competence at tasks (not even the full version). GPT-3 might become par-human at a few certain things, like holding short conversations or generating passages of text. It will be so convincing that it will start freaking people out and make some wonder if OpenAI has actually done it. A /r/SubSimulatorGPT3 would be virtually indistinguishable from an actual subreddit, with very few oddities and glitches. It will be the first time that a neural network is doing magic, rather than the programmers behind it being so amazingly competent. And it may even be the first time that some seriously consider AGI as a possibility for the near future.

Who knows! Maybe if GPT-2 had the entire internet as its parameters, it would be AGI as well as the internet becoming intelligent. But at the moment, I’ll stick to what we know it can do and its likely abilities in the near future. And there’s nothing suggesting GPT-2 is that generalized.

I suppose one reason why it’s also hard to gauge just how capable GPT-2 Large is comes down to the fact so few people have access to it. One guy remade it, but he decided not to release it. As far as I can tell, it’s just because he talked with OpenAI and some others and decided to respect their decision instead of something more romantic (i.e. “he saw just how powerful GPT-2 really was”). And even if he did release it, it was apparently “significantly worse” than OpenAI’s original network (his 1.5 billion parameter version was apparently weaker than OpenAI’s 117 million parameter version). So for right now, only OpenAI and whomever they shared the original network with know the full scope of GPT-2’s abilities, however far or limited they really are. We can only guess based on GPT-2 Small and GPT-2 Medium, and as aforementioned, they are quite limited compared to the full thing.

Nevertheless, I can at least confidently state that GPT-2 is the most general AI on the planet at the moment (as far as we know). There are very good reasons for people to be afraid of it, though they’re all because of humans rather than the AI itself. And I, for one, am extremely excited to see where this goes while also being amazed that we’ve come this far.


What exactly should GPT-3 be able to do? That, I cannot answer because I’m not fully aware of the full breadth of GPT-2, but the knowledge that it and MuseNet are fundamentally the same network trained on different data sets suggests to me that a theoretical 100B parameter version ought to be able to do at least the following:

  • Reach roughly 90% accuracy on either the Winograd Schema Challenge or the WNLI
  • Generate upwards of 1,000 to 2,000 words of coherent, logical text based on a short prompt
  • Increase the accuracy of its output by adding linked resources from which it can immediately draw/spin/summarize
  • Generate extended musical pieces
  • Generate low-resolution images, perhaps even short gifs
  • Translate between languages, perhaps even figuring out context better than Google Translate
  • Understand basic arithmetic
  • Generate usable code
  • Caption images based on the data presented
  • Generate waveforms rather than just MIDIs
  • Gain a rudimentary understanding of narrative (i.e. A > B > C)

All this and perhaps even more from a single network. Though it’s probable we’ll get more specialized versions (like MuseNet), the basic thing will be a real treat.

I myself don’t understand the specifics, so I can’t say that GPT-X will be able to use language modeling to learn how to play an Atari video game, but I can predict that it may be able to create an Atari-tier video game some time next decade. Any data-based tasks can be automated by an agent such as GPT-X, and this includes things like entertainment and news. It’s the purest form of “synthetic media”.

Character Creator: The Game | Possibly coming as soon as next year?

Several years ago, a neat article in Forbes appeared:

The Case For ‘Character Creator: The Game’

I found it only because of my perpetual desires to find a way to design fictional characters and personas for various story ideas. I’ve been searching since 2013 for the perfect one, but they all have some shortcoming.

The most technically robust character creators were part of dedicated games, but of course you actually needed those games to get the experience. More than that, since they were parts of games, the base character creators very often did not have all possible customization items from the start— you’d have to buy and unlock more items and accessories as you played the game. What’s more, since games are often thematic, you may not be able to create the exact kind of character you want if there are any specific details in mind that aren’t also available in the creation system. But all in all, retail games have the best graphics of them all and you can usually do a lot more with them.

Free online character creators came in two packages: graphical programs and dress-up games. The former, which includes stuff like Mixamo, certainly have more ways to pose but have very few customization options since you either needed to download extra packs from their store or had to design it yourself.

The latter has always been the easiest. Dress-up games are basically just flash games where you dress up an avatar, typically designed like a doll, superhero, or anime character.

The big problems with dress-up games is that they are very often thematic and the art is dodgy— considering they’re made for flash game sites and offer little to no financial restitution for their creators, art assets are typically ready-made and low quality. There’s usually only one perspective— full-frontal or, less often, quarter-turn. And save for the best ones, you can’t edit any aspect of your character’s body outside of token masculine or feminine features since these are indeed dress-up games. In other words, you get what you pay for. You decided to go the free route instead of commissioning an artist, so you can’t complain that your character looks cheap.


With the rise of GANs, this may change in very short order. Indeed, it is entirely possible that we are within a year or two of a true “character creator: the game”. What’s more, the capabilities of such will be far beyond anything we see today even in the highest quality character creation systems.

This is due to three important factors:

  1. Text-to-image synthesis. In this theoretical game, you won’t necessarily need to fiddle with sliders, attachable items, or presets. Instead, you could type in a description into a box and near-instantly get your design as an output. Say I want to design an anime-style girl with jet-black hair, blue skin, pink eyes, and wearing such-and-such clothes with a devil tail and steampunk wings. Normally, I’d have to go through a series of different menus starting with the basic body type, then the hair, then the face, and so on and so forth. Here, that simple description alone will generate an image. If it’s not the one I want, I can keep generating them until I find one that’s at least close enough and then go in to edit the finer details if needbe.
  2. Variable artstyle or graphics. If I want to create a character in the style of the Simpsons, I either need to commission an artist who draws in that style, find a flash game that allows me to edit a character and hope they have what I want, or learn to draw myself. And what if I want another character in 3D but in a janky, Sega Saturn or PS1-style polygonal graphics? With this theoretical game, this won’t be much of a problem either. As long as you give the GAN something from your preferred style, of course, it could conceivably give you a character that’s minimalist, blocky, cel-shaded, photorealisic, and everything in between. For example, if I want to generate a character that looks as if it were drawn by the mangaka Akira Toriyama, I could. If I wanted a character “drawn” by cartoonist Tex Avery, I could very well get one. If I wanted a photorealistic avatar, I could have that as well. This could be used to create that generated comic I talked about before, and it could also theoretically be used to create character models that modders can insert into old games.
  3. Unlimited flexibility. Because of the aforementioned aspects, there’s no limit to what you can create. You wouldn’t need to worry about whether or not certain assets are in the engine— as long as you can provide the GAN with some representation of that asset, it’ll be able to translate it onto your character. For example: almost all character creators don’t have wild, electrical, Super Saiyan-esque hair. But if you can give the GAN images like this, it will remember that design and even be able to fill in the blanks if it’s not exactly what you want. What if your character is supposed to have neon glow-in-the-dark hair like this? You’re welcome, I’ve just given you all you really need for your character in the future.

The possibilities are endless. And of course, if this GAN can create a character, it can obviously create other things. But this is what I’m focusing on at the moment.

I think we’ll see early variants of it this year, building off the recently released StyleGAN network. We’ll even see some text-to-image synthesis, which is well within the powers of current day AI.

Artificial Intelligence: A Summary of Strength and Architecture

Not all AI is created equal

Types of Artificial Intelligence: Redux

Artificial intelligence has a problem: no one can precisely tell you what it is supposed to be. This makes discussing its future difficult. Current machine learning methods are impressive by the standards of what has come before, and certainly we can give various systems and networks enough power to rival and exceed human capabilities in narrow areas. The contention is whether or not these networks qualify as “artificial intelligence”.

My personal definition of artificial intelligence is a controversial one, as I am privy towards lumping even basic software calculations under the umbrella of “AI”. This is because there are essentially two separate kinds of “artificial intelligence”— there is the field of artificial intelligence research, which is a branch of computer science, and there is the popular connotations of artificial intelligence. AI is popularly known as being “computers with brains of varying intelligence”.

Business rule management systems are not commonly considered to be “artificial intelligence” in the popular imagination. Indeed, the very name conjures a sort of gunmetal-boring corporate software model. Yet BRMS software is one of the most widely commercialized forms of AI, dating back to the late ’80s.

If we limit all AI to “computers capable of creative thinking”, even many classic sci-fi depictions of AI would not qualify. Yet if my terrifying and anarchist definition became dominant, then we would have to presume that the Sumerians created the first AIs when they invented abacuses.

This is one reason why the original post on the various types of AI doesn’t work. But there are plenty more.

 

Another bottleneck in understanding the future of AI research is our limited imaginings of artificial general intelligence— a feat of engineering considered equal to the creation of practical nuclear fusion, room-temperature superconductors, and metallic hydrogen. As with all of these, the possibilities are much wider than we initially conceived. Yet it’s with AGI that I feel there is a great deal of hype and misunderstanding that would be more easily turned to practical breakthroughs if there were a shift in how we perceived it.

I, for one, always found it odd that we equate “artificial general intelligence” with “human-level AI” despite the fact that every animal lifeform possesses general intelligence— yet no one seriously claims that nematodes and insects are our intellectual rival.

“Surely,” I said as far back as 2012, “there has to be something that comes before human-level AI but is still well past what we have now.”

A related issue is that we compare and contrast narrow AI software with imagined general AI many decades henceward, allowing ourselves nothing to bridge the gap. There is no ‘intermediate’ AI.

All AI is either narrow and weak or general and strong. We have no popular ideas for “narrow and strong” AI despite the fact we have developed a multitude of narrow networks that have far surpassed human capabilities. We also have no popular ideas for “general and weak” AI, which is to say an AI that is capable of generalized learning but is not as intelligent as a human. This could be for a motley variety of factors, many of them coming down to basic neuroscience— for example, something that learns on a generalized level may still lack agency.

So here is a basic rundown on my revised “map” of AI, which has three degrees: ArchitectureStrength, and Ability.


Architecture

Architecture is defined by an AI’s structural learning capacity and is already known by the terms “narrow AI” and “general AI“. Narrow AI describes software that is designed for one task. General AI describes software that can learn to accomplish a wide variety of tasks. Of course, this is usually synonymous with software that can learn to accomplish any task at the same level as a human being, which I’ll explain later why that isn’t necessarily always the case.

I wish to add one more category: “expert AI“. Expert artificial intelligence, or artificial expert intelligence (XAI or AXI) describes artificial intelligences that are generalized across a certain area but are not fully generalized, as I’ll explain in greater detail below. You may see it as “less-narrow AI”, with computers capable of learning a variety of like narrow tasks. AXI is very likely the next major step in AI research over the next five to ten years.

 

Mechanical Calculations: These are calculators and traditional computer software. They only do calculations. Addition, subtraction, multiplication, division, etc. There is no intelligence involved. Mechanical calculations can be considered the ‘DNA’ of AI, the root by which we are able to construct intelligences but by itself is not a form of AI. As aforementioned, this level starts with ancient abacuses.

Artificial Narrow Intelligence: Artificial narrow intelligence (ANI), colloquially referred to as “weak AI”, refers to software that is capable of accomplishing one singular task, whether that be through hard coding or soft learning. This describes almost all AI that currently exists, and is also possibly the most consistently underestimated technology of the past 100 years. Just about any AI you can think of, from Siri down to motion sensors, qualify as ANI. Once you program an ANI to do a certain task, it is locked into that task. Just as you cannot make a clock play music unless you reformat its gears for that purpose, you must reprogram an ANI if you want it to do something it was not programmed to do. This includes narrow machine learning networks that are limited to cohesive parameters. Machine learning involves using statistical techniques to refine an agent’s performance, and while this can be generalized for much more interesting uses, it is not magical and is natively a narrow field of AI.

Artificial Expert Intelligence: Artificial expert intelligence (AXI), sometimes referred to as “less-narrow AI”, refers to software that is capable of accomplishing multiple tasks in a relatively narrow field. This type of AI is new, having become possible only in the past five years due to parallel computing and deep neural networks. The best example is DeepMind’s AlphaZero, which utilized a general-purpose reinforcement learning algorithm to conquer three separate board games— chess, go, and shogi. Normally, you would require three separate networks, one for each game, but with AXI, you are able to play a wider variety of games with a single network. Thus, it is more generalized than any ANI. However, AlphaZero is not capable of playing any game. It also likely would not function if pressed to do something unrelated to game playing, such as baking a cake or business analysis. This is why it is its own category of artificial intelligence— too general for narrow AI, but too narrow for general AI. It is more akin to an expert in a particular field, knowledgeable across multiple domains without being a polymath. This is the next step of machine learning, the point at which transfer learning and deep reinforcement learning allow for computers to understand certain things without needing to be mechanically fed rules and capable of expanding its own hyperparameters.

Artificial General Intelligence: Artificial general intelligence (AGI), sometimes referred to as “strong AI”, refers to software capable of accomplishing any task, or at least any task accomplishable by biological intelligence. Currently, there are no AGI networks on Earth and we have no idea when we’ll create the first truly general-purpose artificial intelligence. However, AGI is a much greater qualitative improvement over AXI than AXI is over ANI— whereas AXI is multi-purpose, AGI is omni-purpose. Theoretically, a sufficiently advanced AGI is indistinguishable from a healthy adult human— and even this represents the lower end of the true capabilities of strong artificial intelligence.


Strength

Strength in AI is defined by an AI’s intellectual capacity compared to humans.

Weak Artificial Intelligence is any AI that is intellectually less capable than humans but is colloquially used to describe all narrow AI.

Strong Artificial Intelligence is any AI that is intellectually as capable or more capable than humans but is colloquially used to describe all general AI.

Because of colloquial usage, “weak” and “narrow” are interchangeable terms. Likewise, “strong” and “general” are used to mean the same thing. However, as AI progresses and increasingly capable computers leave the realm of science fiction and enter reality, we are discovering that there is a spectrum of strength even within AI architectures.

For example: we used to claim that only human-level general intelligence will be capable of defeating humans at Chess. Yet DeepBlue accomplished the task over twenty years ago and no one seriously claims that we are being ruled over by superintelligent machine overlords. People said only strong AI could beat humans at for Go, as well as for interpersonal game shows like Jeopardy. Yet “weak” narrow AIs were able to trounce humans in all these tasks and general AI is still nowhere in sight.

My belief is that nearly any task we can conceive can be accomplished by a sufficiently strong narrow intelligence, but since we conflate strong AI with general AI, we consistently blind ourselves to this truth. That’s why I’ve decided to decouple strength from architecture.

Weak Narrow Artificial Intelligence: Weak Narrow AI (WNAI) describes software that is subhuman or approaching par-human in strength in one narrow task. Most smart speakers/digital assistants like Amazon Echo and Siri occupy this stratum as they do not possess any area of ‘smarts’ that is equal to that of humans, though their speech recognition abilities does lead to us psychologically imbuing them with more intelligence than they actually possess. These are merely the most visible WNAI— most AI in the world is in this category by nature and this will always be the truth, as there is only so much intelligence you need to accomplish certain tasks. As I mentioned in the original post, you don’t need artificial superintelligence to run task manager or an industrial robot. Doing so would be like trying to light a campfire with Tsar Bomba. Interestingly, this is a lesson a lot of sci-fi overlooks due to the belief that all software in the distant future will become superintelligent, no matter how inefficient it may be.

Strong Narrow Artificial Intelligence: Strong Narrow AI (SNAI) describes software that is par-human or superhuman in strength in one narrow task. In my original post, I made the grievously idiotic mistake of conflating ‘public’ AI with SNAI, despite the fact that SNAI have essentially been around since the early 1950s— even a program that can defeat humans more than 50% of the time at tic-tac-toe can be considered a “strong narrow AI”. This is one reason why the term likely never went anywhere, as our popular idea of any strong AI requires worlds more intelligence than a tic-tac-toe lord. But strength is subjective when it comes to narrow AI. What’s strong for plastic may be incredibly weak for steel. What’s usefully strong for glass is likely far too brittle for brick. This is still true for narrow AI. Right now, SNAIs are more popularly represented by game-mastering software such as AlphaGo and IBM Watson because they require some level of proto-cognition and somewhat recognizable intellectual capability that is utterly alien compared to the likes of Bertie the Brain.

Weak Expert Artificial Intelligence: Weak expert AI (WXAI) describes software that is subhuman or approaching par-human in strength in a field of tasks. Due to expert AI still being a novel development as of the time of writing, we don’t have many examples, and ironically one of few examples we have is actually strong expert AI.  However, I can imagine WXAI as being similar to what Google DeepMind and OpenAI are currently working on with their Atari-playing programs. DeepMind in particular uses one generalized network to play a wide variety of games, as aforementioned. And while many of them have reached par-human and superhuman levels of playing, so far we have not received any word that this algorithm has achieved par-human across all Atari games. This would make it closer to approaching par-human strength. This becomes even more noticeable when taking into consideration that this network’s play experience likely has not been transferred to games from more advanced consoles such as the NES and SNES.

Strong Expert Artificial Intelligence: Strong expert AI (SXAI) describes software that is par-human or superhuman in strength in a field of tasks. Currently, the best (and probably only) known example is DeepMind’s AlphaZero network. To a layman, an SXAI will likely seem indistinguishable from an AGI, though there will still be obvious parameters it cannot act beyond. This is also likely going to be a very peculiar and frightening place for AI research, an era where AIs will begin to seem too competent to control despite their actual limitations. One major consideration is that since SXAI will have capabilities beyond one narrow field, it can’t be considered “strong” if it’s only competent in a single field. I would reckon that if it’s parhuman in 30% of all capabilities, it qualifies as SXAI.

Weak General Artificial Intelligence: Weak general AI (WGAI) describes software that is subhuman or approaching par-human in strength in general, perhaps with a stronger ability in a particular narrow field but otherwise not as strong as the human brain. Oddly enough, I’ve very rarely heard of the possibility of WGAI. If anything, it’s usually believed that the moment we create a general AI, it will rapidly evolve into a superintelligence. However, WGAI is very likely going to be a much longer-lived phenomenon than currently believed due to computational limits. WGAI is not nearly as magical as SGAI or ASI— should the OpenWorm project bear fruit, the result would be a general AI. The only difference being that it would prove to be an extraordinarily weak general AI, which gives this term a purpose. Most robots used for automation will likely lie in this category, if not SXAI, since most tasks merely require environmental understanding and some level of creative reactivity rather than higher order sapience.

Strong General Artificial Intelligence: Strong general AI (SGAI) describes software that is par-human or superhuman in strength across all general tasks. This is sci-fi AI, agents of such incredible intellectual power that they rival our own minds. When people ask of when “true AI” will be created, they typically mean this.

Artificial Superintelligence: Artificial superintelligence (ASI) describes a certain kind of strong general artificial intelligence, one that is so far beyond the capabilities of the human brain as to be virtually godlike. The point at which SGAI becomes ASI is a bit fuzzy, as we tend to think of the two much of the same way we think of the difference between stellar-mass and supermassive black holes. My hypothesis is that SGAI can still be considered superhuman and not break beyond theoretical human capabilities— the point at which SGAI becomes ASI is the exact point at which a computer surpasses all theoretical human capabilities. If you took our intelligence and brought it to as many standard deviations down the curve as genetically possible, you will eventually come across some limit. Biological brains are electrochemical in nature, and the fastest brain signal travels at around 270 miles per second. There is, in theory, a maximum human intelligence. ASI is anything beyond that point. All the heavens lie above us.


Ability

Ability in AI is defined by an AI’s cognitive capabilities, ranging from complete lack of self-awareness all the way to sapience. I did not create this list, but I find it to be extremely useful towards understanding the future development of artificial intelligence.

Reactive: AI that only reacts. It doesn’t remember anything; it only experiences what exists and reacts. Example: Deep Blue.

Limited Memory: This involves AI that can recall information outside of the immediate moment. Right now, this is more or less the domain of chatbots and autonomous vehicles.

Theory of Mind: This is AI that can understand the concept that there are other entities than itself, entities that can affect its own actions.

Sapience: This is AI that can understand the concept that it is an individual separate from other things, that it has a body and that if something happens to this body, its own mind may be affected. By extension, it understands that it has its own mind. In other words, it possesses self-awareness. It is capable of reflecting on its sentience and self-awareness and can draw intelligent conclusions using this knowledge. It possesses the agency to ask why it exists. At which point it is essentially conscious.

 

 

Artificial Intelligence: The How

Meet the Sensory Orb. It’s a flesh orb that possesses a powerful synthetic brain. There are several questions one must ask about the Sensory Orb.

Why is the Sensory Orb important? Because it’s the key to artificial general intelligence of the human variety.

You see, there are multiple strains of thought as to how to achieve AGI. Most serious computer scientists and neurologists know that it’s not something we’re likely to achieve anytime soon, but their reasoning is different from what most people might assume.

We don’t understand how intelligence or consciousness works, first and foremost. However, we can try our best at mimicking what we see. Perhaps one of these methods will work. After all, we don’t need to understand every single facet of something in order to make it work. We also expect the final leap to AGI to be accomplished by AI itself. So why is getting there so hard in the first place?

For one, we are still limited by computing power. It seems ridiculous considering how stupidly powerful computers today really are, but it’s true— while the most powerful supercomputers have exceeded the expected operations-per-second done by the brain, these computers still cost hundreds of millions of dollars. We need to bring that cost down if we want to make AI research practical.

But forget about the cost for a moment. Let’s pretend DeepMind had TaihuLight in their possession and could utilize every FLOPS for its own purposes. Would we see major breakthroughs in AI? Of course. But would we see human-level AI? Not even close.

“But they’re DeepMind! Their AI has beaten the human champion at Go a decade before the experts said it could be done! How do they still lack AGI?”

For one, that’s not entirely true— experts said a computer could become the world champion at Go by 2016 if there were sufficient funding put into the problem. And sufficient funding did indeed arrive.

But more importantly, while DeepMind’s accomplishments cannot be overstated, they haven’t actually brought us any closer to human-level AGI.

I want you to marvel at the human brain. It’s a fine thing.

Here is a metal table. On top of this metal table are two brains. One is a newborn baby’s brain, and next to it is the brain of Stephen Hawking. Don’t worry, we’ll return the brains to their rightful owners after this blog post. But I want you to think about what these brains are capable of.

The newborn brain is already a powerful computer that’s learning every single second, forming new neural pathways as it experiences life. Mr. Hawking’s brain is a triple-A machine of cosmic proportions, always thinking and never resting.

Except these two facts are dirty lies. The brains before you aren’t doing anything of the sort. The newborn baby’s brain is not forming any new connections. Hawking’s brain isn’t thinking. And why? Because they are disembodied. They are no longer experiencing any senses, and the senses necessary to make thoughts even work are no longer there. They’re both equal in terms of active intelligence— zero.

If you asked the newborn baby’s brain to add two and two, you’d just look like a fool because you’re talking to a tiny little blob of fat. Even if you asked Hawking’s brain the same question, you’d never get an answer. They can’t answer that question— they’re just brains. They don’t have ears to hear you. They don’t have eyes to see you. They don’t have mouths or hands to respond to you. You do not exist to them.

Despite what fiction may proclaim, brains are not actually ‘sentient’ without their bodies. A brain can’t “see” you or “respond” to you if you ask it a question, even if you stick it into a jar full of culture fluids.

If you hook up a screen and a keyboard to that brain, would you then have a proper sensory input in order to get the outputs of the newborn and Mr. Hawking? Of course not— brains did not evolve to be literal computers. You can’t just stick a plug into a brain and expect it to behave just like your desktop. In order to bring these two brains back to life, you’d need to construct whole bodies around their functions. And not just one or two of their functions— all of them.

So the point is: you can’t just take a human brain, set it out on a desk, and treat it like a fully-intelligent person. If you had Descartes’ Evil Demon or the Brain in a Vat, you could develop the brain until it possessed intelligence in a simulated reality, but the brain itself can do nothing for you. It sounds utterly insane to even contemplate.

Yet, for whatever reason, this is how we treat computers. We think that, if we had a computer with deep reinforcement recurrent spiked progressive neural networks and 3D graphene quantum memristors (insert more buzzwords here), we’d have AGI. In fact, you could have the servers running Skynet brought into real life, and you’d still not see AGI if your idea of making it intelligent is simply to feed it internet data.

Without sensory experiences, that computer will never achieve human-level intelligence. That’s not to say that we could achieve human-level AI today if we took ASIMO and decked it out with sensors, but the gist is that it would be foolish to ever expect synthetic intelligence surpassing humans by treating a computer to the furthest thing from human experiences.

And so we return to the Sensory Orb. The Orb itself is not natively intelligent. It’s no more intelligent than your desktop computer (circa 2027). But, unlike your desktop, it is fitted with a whole body of sensory inputs. The more it experiences, the more it body ‘evolves’ sensory outputs.

It is programmed to like being touched and tickled. Thus, if you tickle it, it will grow to like you. If you pinch its skin, it will roll away from you. Of course, it has to learn how to roll away first, but it quickly learns. If you keep pleasing it or abusing it, its visual senses will recognize you and either run to or away from you on sight. It has many preprogrammed instincts, including knowledge of “eating”. It knows how to find its charger, but if you bring it to its charger, it will grow to like you even more.

And if you teach the orb how to communicate with you through speech, you can teach it various commands. With enough training, the orb will learn to ask about itself. It can learn about other Sensory Orbs, learn about computers and flesh, learn about sensory experiences, and learn that it has its own body that allows it to ‘live’. So one day, you may be surprised if it asks about itself.

Is this human-level intelligence? Not necessarily, but it’s far closer to anything we have today. And we don’t necessarily need a real-life Sensory Orb to achieve this— a good-enough virtual simulation can also suffice. But nevertheless, the point remains: in order to achieve AGI, computers need to experience things..

Yuli’s Law: On Domestic Utility Robots

The advancement of computer technology has allowed for many sci-tech miracles to occur in the past 70 years, and yet it still seems as if we’ve hit a plateau. As I’ve explained in the post on Yuli’s Law, this is a fallacy— the only reason why an illusion of stagnation appears is because computing power is too weak to accomplish the goals of long-time challenges. That, or we have already accomplished said goals a long time ago.

The perfect example of this can be seen with personal computing devices, including PCs, laptops, smartphones— and calculators.

The necessary computing power to run a decent college-ready calculator has long been achieved, and miniaturization has allowed calculators to be sold for pennies.  There is no major quantum leap between calculators and early computer programs.

Calculating the trajectory of a rocket requires far less computing power than some might think, and this is because of the task required: guiding an object using simple algorithms. A second grader could conceivably create a program that guides a bottle rocket in a particular direction.

This is still a step up from purely mechanical systems that give the illusion of programming, but there are obvious limits.

I’ll explain these limits by using a particular example, an example that is the focus of this post: a domestic robot.  Particularly, a Roomba.

I-Robot_Roomba_Autonomous_FloorVac_Vacuum_Cleaner

An analog domestic robot has no digital programming, so it is beholden to its mechanics. If it is designed to move in a particular direction, it will never move in another direction. In essence, it’s exactly like a wind-up toy.
I will wind up this robot and set it off to clean my floors. Thirty seconds later, it makes a left turn. After it makes this left turn, it will move for twenty seconds before making another left turn. And so on and so forth until it returns to its original spot or runs out of energy.

There are many problems with this. For one, if the Roomba runs into an obstacle, it will not move around it. It will make no attempt to avoid it a second time through. It only moves along a preset path, a path you can perfectly predict the moment you set it off. There is a way to get around this— by adding sensors. Little triggers that will force a turn early if it hits an object.

 

Let’s bring in a digitally programmed Roomba, something akin to a robot you could have gotten in 2005. Despite having a digital computer for a brain, it seems to act absolutely no different from the mechanical Roomba. It still gets around by bumping into things. Even though the mechanical Roomba could have been created by someone in Ancient Greece, yours doesn’t seem any more impressive on a practical level.

Thus, the robot seems to be more novel than practical. And that’s the perception of Roombas today— cat taxis that clean your floor as a bonus rather than a legitimate domestic robot.
Yet this is no longer a fair perception as the creators of the Roomba, iRobot, have added much-needed intelligence to their machines. This has only been possible thanks to increases in computing power allowing for the proper algorithms to run in real-time.

For example, a 2017-era Roomba 980 can actually “see” in advance when it’s about to run into something and avoid it. It can also remember where it’s been, recognize certain objects, among other things (though Neato’s been able to do this for a long time). Much more impressive, though still not quite what we’re looking for.

What’s going on? Why are robots so weak in an age of reusable space rockets, terabyte smartphones, and popular drone ownership?

We need that last big push. We need computers to be able to understand 3D space.

Imagine a Roomba 2000 from the year 2025. It’s connected to the Cloud and it utilizes the latest in artificial intelligence in order to do a better job than any of its predecessors. I set it down, and the first thing it begins doing is mapping out my home. It recognizes any obstacle as well as any stain— that means if it detects dog poop, it’ll either avoid it or switch to a different suction to pick it up. Once it has mapped my house, it is able to get a good feel for where things are and should be. Of course, I could also send it a picture of another room, and it will still be able to get a feel for what it will need to do even if it’s never roamed around inside before.

The same thing applies to other domestic robots such as robotic lawn mowers— you’d rather have a lawn mower that knows when to stop cutting, whether that means because it’s moving over a new terrain or because it’s approaching your child’s Slip n’ Slide. Without the ability to comprehend 3D space or remember where it’s been or where it needs to go, it’ll be stuck operating within a pre-set invisible fence.

Over all of this, there’s the promise of bipedal and wheeled humanoid robots working in the home. After all, homes are designed around the needs of humans, so it makes sense to design tools modeled after humans. But the same rules apply— no comprehension of 3D space, no dice.

In fact, a universal utility robot like a future model of Atlas or ASIMO will require greater advancements than specialized utility robots like a Roomba or Neato. They must be capable of utilizing tools, including tools they may never have used before. They must be capable of depth perception— a robot that makes the motions of mopping a floor is only useful when you make sure the floor isn’t too closer or far away, but a robot that genuinely knows how to mop is universally useful. They must be capable of understanding natural language so you can give them orders. They must be flexible, in that they can come across new and unknown situations and react to them accordingly. A mechanical robot would come across a small obstacle, fall over, and continue moving its legs. A proper universal utility robot will avoid the obstacle entirely, or at least pick itself up and know to avoid the obstacle and things like it. These are all amazingly difficult problems to overcome at our current technological level.

All these things and more require further improvements in computing power. Improvements were are, indeed, still seeing.

zml5hbc
Mother Jones – “Welcome Robot Overlords. Please Don’t Fire Us?”

Passenger Drones

One of the most interesting developments in sci-tech in the past few years is the sudden interest in the concept of “passenger drones“. That appears to be their most popular name, though you may have heard of them as “drone taxis” and “autonomous flying cars”. I’ve even seen the term “pilotless helicopter” used once or twice (though drones don’t necessarily have to be rotored vehicles). For the sake of this article, I’ll stick with ‘passenger drone’.

So what exactly is a passenger drone? In short, its name gives it away— a drone that can carry passengers. Typically, drones are defined as being “unmanned aerial vehicles”. You can see the conflict in definitions, hence why some are hesitant to actually call these ‘drones’. Nevertheless, linguistic drift has changed the definition of drone and that’s something drone hobbyists have to live with.

I say this because passenger drones are based on the designs of quadcopters, now popularly referred to as ‘drones’.

But enough about the linguistics.

Passenger drones represent the closest realization of yesteryear’s dream of flying cars. They are personal vehicles that theoretically anyone can own and use with ease, and they indeed work in three dimensions*. So why should we care about them when that dream has never come true before now?

*”Three dimensions” in transportation terms refers to the inclusion of flight. “Two dimensions” refers to ground and sea travel.

Simple: your answer is in the name. Again.

This is a drone. That means you are not the one piloting the vehicle. And I don’t mean ‘you’ specifically, but ‘you’ as a human. Humans did not evolve to navigate 3D space. We can barely manage traveling in 2D space at high speeds— proto-humans never had to move any faster than their fastest sprint. This becomes obvious when you view motor vehicle statistics. In the United States of America alone, over 30,000 people die in vehicular accidents yearly.
And despite this, we are not even in the top 5 for “most killed yearly in motor accidents.” The number one country is, not surprisingly, China: they lose well over 250,000 a year in car accidents.

Worldwide, 1.25 million die every year in motor accidents. And note, that’s deaths, not casualties. All of this is evidence that humankind is simply not designed well to casually travel at speeds higher than 20 miles per hour.

To throw another dimension and another two hundred miles per hour at us would unleash gigadeaths per year until humanity as a whole finally gives up. Human extinction by flying car.

This is the chief reason why flying cars aren’t a thing. Humans simply cannot handle it. Pilots have to go through thousands of hours of training just to become proficient, and that’s with vehicles that are already highly automated.

Indeed, as of right now, the closest thing to a “flying car” is a Cessna 172.

Of course, other reasons include the fact roadable vehicles and flying vehicles require completely different designs and aerodynamics, as well as the power requirements necessary to keep such a vehicle in the air. But perhaps we could overcome these issues if only there were a way for the common person to actually survive take-off, flight, and landing without killing himself.

Drones are that solution. Take away the need for the common person to do the flying.

That’s the promise passenger drones offer us. Again, there’s still the issue that flying is inefficient, but it’s always possible that passenger drones become a common sight over cities. Perhaps they’ll be privately owned; perhaps they’ll be municipally owned and rented out for use. This remains to be seen because the idea of flying cars and personal aerial vehicles being a real thing only became feasible within the past couple of years.

As of today, 4 April 2017, the first passenger drones will enter operation in Dubai, UAE in the summer of this year.

Technism

What exactly is technism?
It is a system defined by automation, particularly the pursuit of maximal automation. The more faculties of society that are automated, the more technist that society becomes.
technist is a person who seeks a highly/fully automated society. The logical endpoint of this philosophy is a point where humanity is disenfranchised from all processes, instead living only to profit from the labor of machines.
In that regard, technism is the opposite of neo-Luddism and primitivism.
The economic philosophy behind technism is known as Vyrdism, which is the belief that humanity should actively exploit the labor of machines, with the common agreement being that we should pursue cooperative ownership. Vyrdists, in the short amount of time they’ve been around, have already sprouted a few branches.
Market Vyrdism describes a society that fuses free market ideals with technism and/or Vyrdism. It bears most resemblance to mutualism and syndicalism. Private property is protected. Humans may no longer be the dominant laborers of society, but they remain in near full control of political and economic matters.
Marxism-Vyrdism describes a society that fuses the ideals of Marxism (perhaps even Marxism-Leninism) with Vyrdism— all automation is collectively owned, with a state apparatus (usually consisting of artificial intelligence, a la Cybersyn) existing to centrally plan the economy. Private property is non-existent. Despite this, humans remain in near full control of political and economic matters.
Pure Technism describes a society that fuses the concept of the dictatorship of the proletariat and replaces the proletariat with the technotariat— automata, both hardware and software, which displace the traditional productive roles of the proletariat. In this case, humanity is completely or almost completely disenfranchised from political and economic matters as automata develops full ownership of society.


Dictatorship of the Technotariat

This is a term I’ve already seen being passed around. This works off pure-technism and can be defined in a very simple and slightly ominous way— the means of production own themselves. This doesn’t mean that hammers become sadistic foremen whipping their abused human slaves— it refers to a state of affairs when synthetic intelligences possess ownership over society and direct social, political, and economic matters. In such a state, humanity would no longer have meaningful ownership over private property, even though private property itself may not have actually been abolished.
AI simply commanding and controlling an economy doesn’t necessarily mean we’ve arrived at this new dictatorship. AI has to own the means of production (essentially, itself).
Unlike Vyrdism, where society is set up similar to an Athenian slave state (with humans and sapient AI existing at the top and sub-sapient or even non-sentient technotarians acting as slave laborers beneath us), a pure-technist society sees humanity exist purely at the whims of synthetic intelligence. It is the logical endpoint of universal basic income, where we do not own anything but are given some capital to live as we please.


To recap: technism is the pursuit of automation, especially full automation. Capitalist and socialist societies ever since the Industrial Revolution could be described as, in some manner, technist. However, technists seek to fully replace the working class with automata known as “technotarians”, whereas most capitalists and socialists seek to use automata to supplement human labor. Vyrdism is a partial fusion of technism with capitalism and socialism (moreso one way or the other depending on if you’re a Market or a Marxist Vyrdist), that’s only possible when technology reaches a point where humans do not need to be directly involved in the economy itself. Pure technism is the full secession of the ownership of the means of production to the means of production themselves, which is only possible if the means are artificially intelligent to a particular point I’ve defined as being “artilectual.” The difference between an AI/AGI and an artilect is that a general AI is an ultra-versatile tool while an artilect is a synthetic person. Of course, when I say “an artilect”, that implies that one would be a physically defined person as we would recognize it— with a tiny primate-esque body and a limited brain, with very-much human aspirations and flaws. In fact, an artilect could be an entire collective of AI that exists across the planet, that has control over nearly all robots.

A pure-technist society is not the same as a Vyrdist society. Not even a “Marxist-Vyrdist” society. Vyrdism involves human ownership over the means of production when the means are capable of working without any human interaction or involvement. Pure-technism is when humans do not own the means of production, rendering us dependent upon the generosity of the machines.

Because of these qualifiers, it is not wrong to say that any automation-based economic system is technist. This includes Resource-Based Economies as well as the Venus Project. If you take Marxism-Vyrdism to its logical conclusion, you will arrive at Fully Automated Luxury Communism. All of which are considered “post-scarcity economics“. All of which are technist.


Joint Economy vs. Mixed Economy

So let me take a minute to discuss the difference between a “joint economy” and a “mixed economy.”

Back when I was doing the technostist wiki (“technostism” being a poor precursor to the current term “technism”), I pointed out the difference between market socialism and mutualism, and mixed economies that claimed to fuse “capitalism and socialism.” Mixed economies fuse state socialism and free-market capitalism. I’ve yet to see a mixed economy be used to describe a place that fuses market socialism and free-market capitalism. So I decided to take the initiative and create a new term myself: “joint economy.”

A joint economy is one that fuses capitalist and worker (and, eventually, automata) ownership of the means of production to some great degree. It has nothing to do with the government— the “socialist” aspects in this case are purely economic. When a nation has a joint economy, that means it has a healthy mixture of purely traditional/authoritarian enterprises and worker cooperatives and democratic businesses (owned and/or managed), perhaps even a cooperative federation or syndicate. You’d still have powerful corporations, but it wouldn’t be a given that all corporations are authoritarian in nature. Something like the Basque Country in Spain is a good example— Mondragon is an absolutely massive corporation, but it’s entirely worker-owned. This means the Basque Country has a “joint economy”. A joint mixed economy is one where you have market socialism and market capitalism alongside state regulations.

This is naturally important in a technist society because we’re fast approaching a time when there’s a third fundamental owner of the means of production, and defining their relationship to the means and to society at large is necessary.
Just as present-day joint economies are the freest possible, an economy where businesses are commonly owned by individuals, collectives, and machines rather than solely one of the three will see the greatest prosperity.

In a future post, I will detail why radical decentralization and ultra-strong encryption must be a goal for any budding technist, as well as how totalitarianism becomes perfected in a degenerated technist society.


 

3bhmy1j

In review: technism is the pursuit of capital imbued with intelligence. The logical endpoint is the point where intelligent capital owns society and all property, thus marking a state of absolute automation.