If our socialism is to solve humanity’s problems, it can’t be merely a set of virtuous opinions. Socialists must be able to govern, and to do so they must understand economics, society and technology, and this understanding must be infused with humanity.
So in this essay and following ones, I will examine artificial intelligence as both a technology and as a social and economic phenomenon. Not to give away the plot, but once you understand the history and technology of artificial intelligence, the social and economic implications are clear.
I will rely here not only on research, but also my own experience. As a computer scientist of a certain age, I might claim to have been there at the founding of Artificial Intelligence, but that wouldn’t be true, although I thought so at the time. A peculiarity of AI research is that every year since at least 1980 people have believed that AI was just beginning at that moment, despite vast amounts of previous research.
In 1981 I was a graduate student in computer science at a large university which claimed to have a “top 20” CS department, and that might have been true. Most of my professors knew their stuff, at least.
One professor I particularly liked taught Operating System Theory, and one day in class he was asked about Artificial Intelligence—was there anything to it? Would it someday work?
He said, “the fundamental assumption of Artificial Intelligence is that the human brain is basically a meat machine, that it’s very similar to a computer in function and structure, only it’s much slower. And so computer jocks can just write some code and the software will do anything human beings can do, but much faster.”
The term “computer jock” meant a programmer of average ability and boundless confidence, and his use of the term indicated he thought that developing AI software might be a significant challenge.
He looked around the room. “So that implies that computers will replace human beings for practically every cognitive task. And it won’t take that long, it’s just a matter of writing some code. The idea is that computers will replace the human brain the way machines replaced human muscle during the Industrial Revolution.”
One grad student asked, “Is Artificial Intelligence worth studying?”
The professor was a young man, lean and bearded. He smiled slightly and shrugged his shoulders. “If you accept that the human brain is basically just like a computer, but slower, then yes, you might decide to study Artificial Intelligence. Just be aware that if your assumption is correct, then you yourself will shortly be replaced by AI, studying itself.”
This was a serious prediction, often repeated at the time, but the audience still chuckled. I wondered why they were laughing.
However, since we were already using computers to calculate statistical and trigonometric functions, matrix operations and graphics, the basic assumption of AI didn’t seem too implausible. But I was aware (and the professor probably was too) that AI researchers had made a sweeping assertion about not just the brain, but about human nature itself.
There is nothing like spending time with computer scientists to see their limitations. Although hard-working and intelligent, they had no breadth beyond their field. They rarely had any interest in philosophy, and the few that did followed Ayn Rand. Their musical tastes were likewise limited: later I met women engineers who liked Enya, and I knew a computer scientist once who loved Gilbert and Sullivan. But easily 80% of computer scientists and engineers had no interest whatsoever in music. Painting and dance were never mentioned.
Their leisure reading was limited to science fiction.
I mention their lack of breadth not to criticize—engineering programs neither allow nor require much exposure to the arts and humanities—but to illustrate their lack of knowledge outside their discipline. No reasonable person would ask a computer scientist about anything other than computer science. And yet all the funding poured into AI since the Fifties has depended on a questionable assertion about the nature of human intelligence, and by extension about the psyche itself.
In what follows I will discuss major areas of AI research since the ‘50s—their goals, their challenges and their accomplishments. But first I have to point out a major difficulty in discussing Artificial Intelligence—-what is its definition? Resolving that problem is beyond the scope of this essay, but I’d like to point out that often software “displays intelligence” that is merely the intelligence of the programmer. Internet search engines, like Google, are sometimes described as examples of AI, but what is Google, after all? It’s a vast, special purpose database management system that includes descriptors and addresses of almost everything on the internet, along with methods for accessing that data and modifying it if needed. It’s a DBMS, a database management system, a nearly ubiquitous technology, but custom made by Google employees. A DBMS is not AI; if we assert otherwise, then AI has been defined so broadly as to be meaningless. The intelligence in a Google search is mostly the intelligence of the humans who designed the database.
You might say, what about the search heuristics? Aren’t they AI? Let’s think about that: does Google use heuristics in its searches? No doubt. Does the software itself choose between a set of heuristics using statistics of recent searches? Possibly.
Does it create its own heuristics? It almost certainly does not. And the reason is that putting all this data into a DBMS—that is, into a defined format—means that searches are constrained. There is a vast amount of data, but it can only be searched in a few ways, and most searches are quickly accomplished using indices or similar tools, like hash tables. If this weren’t true, Google searches would take a long time. And this also implies that any search heuristics are fairly simple. Not dead simple, because Google queries can be complex, but as simple as the programmers can make them.
You might suppose that such a vast amount of data would need extremely complex search methods just to find anything, but it doesn’t, because the database design reduces the complexity of the search space. It’s the same principle as finding a book in a library; sophisticated search methods aren’t needed.
So the smarts in Google are in the database design and to a lesser extent in the search heuristics. It may also do pre-fetching and caching of content to improve performance, and some of this vast database may be compressed.
What Google does is complex and extremely effective, but is it artificial intelligence? Most or all of the intelligence it displays came directly from the database architects and the programmers. And we shouldn’t forget the intelligence of the users, who learn how to frame Google searches to get relevant results. This learning may be half-conscious, but it’s a factor.
The people who do AI research sometimes say that any software that displays intelligence is AI, but that would mean that the statistical package on my laptop is doing AI when it calculates a standard deviation. Most people would say that’s not AI, because they know it’s just calculations, and I agree. The programmer was intelligent, but not the software. Therefore, unless there’s some independence of AI software from its creators, we don’t have AI. But defining and measuring that independence is difficult.
So for this discussion, a general definition of AI doesn’t yet exist, and we’ll set that problem aside. Let’s turn instead to specific examples of AI research.
Machine Translation
Most of the early work in MT was shaped by the exigencies of the Cold War, particularly the need to translate scientific and technical documents from Russian to English.
Initially, there was hope that cryptographic methods developed during World War II could be applied to machine translation. After all, a language you don’t understand is similar (in some sense) to an encrypted version of a language you do understand. However, when you decrypt a message what you ordinarily get is a numeric mapping of letters to letters. The encryption is simpler if semantic units—words and sentences—are ignored.
But machine translation cannot so easily ignore semantics. And the cryptographers themselves had encountered difficult semantic problems during World War II; even after the Allies captured the German Enigma machine from a damaged U-boat, they still sometimes had trouble making sense of the unencrypted German. There were often too many abbreviations, jargon and acronyms for a German speaker without current military experience to understand. Context mattered.
https://en.wikipedia.org/wiki/History_of_machine_translation
Machine translation projects began in earnest around 1951, at Georgetown and MIT. By 1954, the Georgetown Project was able to translate 49 Russian sentences, carefully selected, into English, and they predicted that machine translation would be solved soon:
“Well publicized by journalists and perceived as a success, the experiment did encourage governments to invest in computational linguistics. The authors claimed that within three or five years, machine translation could well be a solved problem.”
https://en.wikipedia.org/wiki/Georgetown%E2%80%93IBM_experiment#Reception
By a “solved problem” they clearly meant that human translators would join candlestick makers and tinsmiths in the history books. The timeline may seem absurdly aggressive to us, but compared to the progress of the Industrial Revolution in displacing traditional craftsmen maybe it wasn’t. And that was the model of change they had.
The Georgetown Project was focused on papers on organic chemistry, a highly denotative subject, so there was little extraneous context to complicate the translation.
The project at MIT confronted the problem of context more directly, by pointing out the difficulty of “semantic ambiguity.” Yehoshua Ben-Hillel created the following test:
“Little John was looking for his toy box. Finally he found it. The box was in the pen.”
The word pen may have two meanings: the first meaning, something used to write in ink with; the second meaning, a container of some kind. To a human, the meaning is obvious, but Bar-Hillel claimed that without a “universal encyclopedia” a machine would never be able to deal with this problem. At the time, this type of semantic ambiguity could only be solved by writing source texts for machine translation in a controlled language that uses a vocabulary in which each word has exactly one meaning.
https://en.wikipedia.org/wiki/History_of_machine_translation#The_early_years
Let’s examine Ben-Hillel’s test case. This isn’t a matter of inherent ambiguity because no native English speaker could mistake the meaning. Little John is clearly a child because he owns a toy box. Adults put toys in boxes all the time—as presents, or in factories—but the term “toy box” is limited to storage in the home, usually in a child’s room.
And because he is a child, he isn’t Robin Hood’s sidekick. Also, children are usually not allowed ink pens, and a toy box wouldn’t fit in an ink pen anyway. But small children have “play pens,” and understanding that “pen” is short for “play pen” would be a difficult challenge for the translation software—which may not even understand that Little John is a child.
Even today, sixty years or more later, if you feed these three sentences into Google Translate, with German as the target language, “pen” is translated as “Stift,” which means an ink pen or a pencil.
Ben-Hillel was and still is quite right that machine translation software would find these three sentences difficult or impossible, and many more like them. But the problem isn’t ambiguity; it’s that the translation software doesn’t understand the world. This was the problem that Ben-Hillel was trying to address with the “universal encyclopedia,” which is the context that people need to fully understand their native language. It approximates the sum of human knowledge, or at least that which can be expressed in words.
As far as I can tell, Ben-Hillel never described how a “universal encyclopedia” would be implemented, and his skepticism about machine translation may have contributed to the US government setting up ALPAC (the Automatic Language Processing Advisory Committee) to evaluate the progress of machine translation research and of computational linguistics in general. The committee concluded in 1966 that machine translation research since 1951 hadn’t generally been useful. Priorities for future research included:
- practical methods for evaluation of translations;
- means for speeding up the human translation process;
- evaluation of quality and cost of various sources of translations;
- investigation of the utilization of translations, to guard against production of translations that are never read;
- study of delays in the over-all translation process, and means for eliminating them, both in journals and in individual items;
- evaluation of the relative speed and cost of various sorts of machine-aided translation;
- adaptation of existing mechanized editing and production processes in translation;
- the over-all translation process; and
- production of adequate reference works for the translator, including the adaptation of glossaries that now exist primarily for automatic dictionary look-up in machine translation
These priorities are revealing. After 15 years of research, “evaluation of translations” was apparently still an open topic. I wonder how experiments were designed and the results analyzed without standardized evaluation.
Several of the recommendations—2, 5, 6, 8, 9—seem to concede that human translators had a future, even if aided somehow by software.
Efficiency and cost were raised in recommendations 2, 3, 5, 6, 7, 8, in fact the majority of the nine. Apparently those issues were discussed a lot. Did anyone explicitly compare the cost of training a human translator with the cost of machine translation research?
Recommendation 9 might imply that future research should focus on developing tools to aid human translators.
ALPAC wasn’t just questioning the theoretical potential of machine translation, as Ben-Hillel had done. They were questioning the experimental rigor and focus of past research as well. It had after all been fifteen years and who knows how much money.
Government funding for MT research fell dramatically as a result of the ALPAC report. This is sometimes regarded by AI researchers as an inexplicable tragedy.
https://en.wikipedia.org/wiki/ALPAC
https://en.wikipedia.org/wiki/AI_winter#Underlying_causes_behind_AI_winters
The reader may have guessed already that AI research largely depends on government research money, and this was especially the case in the 20th century. Naturally AI researchers developed marketing and PR skills early.
Even after ALPAC, computational linguistics continued, because studying the nature of language using computers as a tool was still a valid undertaking. And machine translation did make a comeback, using statistical methods and expansive lexicons that included common phrases and idiomatic expressions. But as Wikipedia states:
“Current machine translation software often allows for customization by domain or profession (such as weather reports), improving output by limiting the scope of allowable substitutions. This technique is particularly effective in domains where formal or formulaic language is used. It follows that machine translation of government and legal documents more readily produces usable output than machine translation of conversation or less standardised text.
Improved output quality can also be achieved by human intervention: for example, some systems are able to translate more accurately if the user has unambiguously identified which words in the text are proper names. With the assistance of these techniques, MT has proven useful as a tool to assist human translators and, in a very limited number of cases, can even produce output that can be used as is (e.g., weather reports).”
https://en.wikipedia.org/wiki/Machine_translation
Sixty-nine years later, and MT still only produces high-quality translations in tightly defined and denotative domains. The fundamentals haven’t changed much since the Georgetown Project.
We might question how funding for MT ever returned; the ALPAC report might reasonably have put an end to government-funded research in MT for good, and why didn’t it? My theory is that capitalism wasn’t willing to give up on the idea that AI was analogous to the Industrial Revolution, and that it might result in a massive reduction in cognitive labor. After all, great fortunes were made when the Industrial Revolution eliminated so many spinners and weavers, and even greater fortunes might be made when AI replaced translators, lawyers and accountants, no matter what the social cost might be.
In other words, the potential of AI to concentrate wealth and power was too alluring. Without some such powerful motive, why throw good money after bad in MT research?
Everyone has used Google Translate, and it’s useful—and fun–to translate individual foreign words, a few sentences or a technical abstract. And the price is right. But we’ve spent a tremendous amount of money and time to get Google Translate, and it can’t be used in serious situations. If testimony in court must be given in a foreign language, they hire a human translator with training and credentials; the judge doesn’t bring up Google on his smartphone. No one accused of a serious crime would communicate with the court, much less their attorney, using Google Translate. And likewise for international business negotiations—unless the two sides can communicate in English. The situation in diplomacy is the same.
I understand that professional translators sometimes use Google Translate for prose, and that makes sense. Google would save the human time on the “first cut” translation, and the human has Ben-Hillel’s “universal encyclopedia” between her ears. This can result in fast, high-quality translations.
An ironic postscript to this discussion is that Ben-Hillel’s granddaughter is a professional translator who translated the Harry Potter books into Hebrew. Did she ever worry she might be replaced by AI? If the young still listen to their grandfathers, probably not.
The Socialist Investor 