Sound Research WIKINDX

(p.3)

Reading and Writing Ability (RW): Captures all of the declarative knowledge and procedural skills a person uses to consume and produce written language.
• Letter-Word Ability: The ability to recognize letters and decode words.
• Reading Comprehension: The ability to understand connected discourse during reading.
• Writing Ability: The ability to write with clarity of thought, organization, and good sentence structure.
• English Usage Knowledge: Knowledge of writing in the English language with respect to capitalization, punctuation, usage, and spelling. (p.4)

"[...] a fundamental intuition behind AI is that human intelligence is a special form of a general notion of intelligence, which have other forms. Using a well-known metaphor, “to fly” and “to fly as a bird” both can be taken as engineering goals, but they are different goals. The latter is possible (though difficult), but if the former is understood as identical to the latter, many valuable designs will be omitted or even disqualified, simply because they are not similar to the original."

The opening Lewis Carroll quote is saying this same thing. 

"According to my definition, intelligence is still a matter of degree. No system can be “perfectly intelligent”, though one system can be more intelligent than another by being able to acquire knowledge in more forms (e.g., additional sensorimotor channels), to reorganize its beliefs and skills in more complicated ways (e.g., more recognizable patterns), or to adapt more efficiently (e.g., faster responses). Of course, this comparability does not define a total order among the intelligence of systems, as one can be more intelligent than the other in a certain aspect, but less intelligent in another one. One implication is that it is not always meaningful to ask whether an AGI is more or less intelligent than a human, as they may follow the same principles while having distinct sensors and actuators, so cannot be compared directly in concrete capabilities"

"For example, creativity is definitely expected from any truly intelligent system, but since it logically follows from being open (as the ability to handle a novel problem it never saw before), it is not explicitly mentioned in the definition."

"The systematic study of intelligence started in psychology, and there has been a huge literature on this topic (Gottfredson, 1997; Goldstein, Princiotta, and Naglieri, 2015). In general, my definition is compatible with many psychological definitions. For example, Piaget sees intelligence as “the most highly developed form of mental adaptation” (Piaget, 1960), and further stated that “Intelligence in action is, in effect, irreducible to everything that is not itself and, moreover, it appears as a total system of which one cannot conceive one part without bringing in all of it” (Piaget, 1963). Medin and Ross (1992) have even made the statement that “Much of intelligent behavior can be understood in terms of strategies for coping with too little information and too many possibilities.” These citations show that the two major factors in my definition, adaptation and AIKR, are considered as central to intelligence by many psychologists. Even so, on the surface my definition does look different from common definitions given by psychologists. For example, a widely accepted definition of intelligence is proposed in Gottfredson (1997):

“Intelligence is a very general mental capability that, among other things, involves the ability to reason, plan, solve problems, think abstractly, comprehend complex ideas, learn quickly and learn from experience. It is not merely book learning, a narrow academic skill, or test-taking smarts. Rather it reflects a broader and deeper capability for comprehending our surroundings ‘catching on,’ ‘making sense’ of things, or ‘figuring out’ what to do."

"Though it sounds natural to define intelligence as a collection of cognitive functions, such a definition encourages a divide-and-conquer methodology, which is partially responsible for the current fragmentation in AI. Though the techniques developed in this way have great theoretical and practical values, they are not easy to be combined together to form a thinking machine that is comparable to the human mind in general, unless there is a “master algorithm” behind them (Domingos, 2018). However, in that situation the intelligence is arguably in the master algorithm itself, like a general-purpose “hands” that can use the problem-specific “tools”, using the handstools metaphor again. However, an AGI-aspiring system like NARS is not an “algorithm” even in the broad sense of the term, and the tools are not necessarily produced from NARS. Instead, NARS could be thought of as an intelligent operating system that decides when to use which hardware or software for the tasks the whole system is facing."

Stongly agree. It misses the "master algorithm" (intentionality?), and the connection between the various functions only make analogous sense, but their actual influence will be limited because they don't function the same as in humans.

"Even though the human brain indeed produces the best-known form of intelligence, we should not let it limit our imagination when designing an electronic form of intelligence, because intelligence is not fundamentally biological."

"Another major issue is that AIXI assumes infinite computational resources, and Legg and Hutter (2007) explicitly stated that “We consider the addition of resource limitations to the definition of intelligence to be either superfluous, or wrong. . . . Normally we do not judge the intelligence of something relative to the resources it uses.” As far as I know, all tests of intelligence have explicit or implicit time limits, and people usually do not take “to exhaustively evaluate all alternatives and pick the best” as an intelligent way of solving a problem, but that is basically what AIXI does."

If I hear one more "but AI solved a maths problem" - how much money did they throw after the problem? How many people supervised it? Could these same people have accomplished the same result cheaper if given similar time frames and computation power but without LLMs? Resources matter in a finite world.

This logic makes machine intelligence dynamic, because it still has different levels, and only if it is relevant in completing the goals of humans it is actually considered intelligent.

"To take the environment as a computable probability distribution over Turing Machines means that whatever the agent does, the actions can only change the rewards it gets, but cannot change the environment. It is a strong postulation about the relation between an agent and its environment, which is only justified as “in standard physics there is no law of the universe that is not computable in the above sense” (Legg and Hutter, 2007). The problem about this justification is that the descriptions about the world provided in classical physics should not be equated with the world itself, and this reductionist position denies the need for generalization and abstraction that describe the environment at multiple levels, where the descriptions are not exactly the same."

"The model is based on the assumption that the true environment can be described by a computable probability distribution over Turing Machines that is unknown to the agent, while the intelligence of the agent is indicated by its ability to maximize the expected reward according to the observation so far. Among the predictions consistent with the observation, the simplest one is favored as more likely to be true."

What is the point of building an Artificial Intelligence that is not inherently superior to humans? 

"Being finite means the system’s information processing capability (such as how fast it can run
and how much it can remember) is roughly a constant at any period of time. This requirement seems
trivial, as any concrete system is surely finite. However, to acknowledge the finite nature means the
system should manage its own resources, rather than merely spending them. This requirement is
mainly for the theoretical models, since most traditional ones, like the Turing Machine, completely
ignore resource restrictions (Hopcroft, Motwani, and Ullman, 2007).

Being open to new tasks means to make no restriction on the content of a task, as long as it is
expressed in an acceptable form. Every system surely has limitations in the signals its sensorimotor
mechanism can recognize or the languages its linguistic competence can handle, but there cannot
be any restriction on what it can be told or asked to do. Even if a new experience conflicts with its
current beliefs, or a new task is beyond its current capability, the system still should handle them
properly, rather than simply ignore it or even be crushed by it.

For the system to live and work in real-time means that new tasks of various types may show
up at any moment, rather than come only when the system is idly waiting for them. It also means
that every task has a response time restriction, which may be in the form of an absolute deadline, or
as a relatively expressed time pressure, such as “as soon as possible.” In general, we can consider
the utility of a solution to be a decreasing function of time, so even a correct solution may become
worthless when produced too late. A real-time system must respond according to these restrictions,
even when they cannot have been completely satisfied."

- - - 

The whole argument here can be tied to Merleau-Ponty's being-towards-the-world and therefore intentional arc. Let's assume a machine actually cares about climate change, and knows the full extent of the consequences it will bring. How will it care? It won't care on terms that reflect on the human biology, but instead on those that relate to its own ability to thrive. More or less heat might not make much of a difference, so why should it care about heat? Perhaps rising water level prove a challenge, so that could be a motivation.

Either way, how does this robot now set about all of its goals? What are they ultimately related to? Everything humans do can be traced back to biological needs that have evolved into something much different. Why do we work hard? Perhaps to impress other people, or even to impress ourselves. Can a robot impress itself? Can it impress another robot? What is a reward to a robot?

- - - 

"The above three assumptions are collectively called the Assumption of Insufficient Knowledge and Resources (AIKR), which identifies the normal working condition of an intelligent system. Of course, this assumption is about the overall situation, not on every task, as there are surely simple tasks for which the system’s knowledge and resources are relatively sufficient, at least roughly speaking. However, such tasks are not where intelligence is really demanded."

In computer science, “computation” does not mean whatever a computer does, but is accurately
defined as a finite and repeatable process that carries out a predetermined algorithm to realize
a function that maps input data to output data (Hopcroft, Motwani, and Ullman, 2007). As
summarized by Marr (1982), to solve a problem “by computation” means:

1. To define the problem as a mapping from a domain of valid input values to a range of possible
output values;
2. To find an algorithm that carries out this mapping step by step, starting from the given input
and ending with the corresponding output;
3. To implement the algorithm in a computer system so as to use it to solve each instance of the
problem.

However, this approach cannot handle a problem if any of the following is the case:

• The problem is not well-defined as a mapping or function;
• The function is well-defined, but the system has no algorithm to solve it, due to either its
ignorance, or the impossibility of such an algorithm;
• There are implemented algorithms for the problem, but the system cannot afford the resources
(mainly computational time and space) to use any of them on the problem instances to be
processed.

"Driven by motivations including to avoid the impossible missions, to obtain the necessary resources, and to improve its public image, the AI community shifted its aim to more realistic tasks, like solving practical problems and carrying out individual cognitive functions. This shift is praised as “AI becomes a science (1987 – present)” (Russell and Norvig, 2002), which was later changed to “AI adopts the scientific method (1987 – present)” (Russell and Norvig, 2010), because “It is now more common to build on existing theories than to propose brand-new ones, to base claims on rigorous theorems or hard experimental evidence rather than on intuition, and to show relevance to real-world applications rather than toy examples” (Russell and Norvig, 2010). It sounds wonderful, but as it turned out this has come at the price of giving up the initial dream of the field."

Just the opposite of this example, LLMs have greatly clouded work in the AI field. Progress looks different in different fields, but when it comes to creating a general intelligence, perhaps there is no middle-ground, even if we differentiate between human and machine intelligence. Machines can become more capable, but if a separating aspect truly is experience, intentionality and reasoning, then it either is or isn't. In this sense, LLMs are not actually progress in achieving a general intelligence (although they were quite possibly a necessary evil to explore). This is also made clear by reserachers like Sutton, LeCun, etc...

"If it turns out to be the case that the only way to get intelligence is doing exactly what the human
brain does, then AI should be considered as an ill-conceived concept. Instead, we would better talk
about brain modeling or emulation. A related issue is whether “mind” can be completely reduced
into “brain.” If it is not the case, then a good model of the brain and a good model of the mind are
not the same, and the intuitive meaning of intelligence is closer to the latter than to the former."

"AI is conceived as computer systems that are similar to the human mind in a certain sense, though a computer and a human mind cannot be identical in all aspects. Therefore, here the key issue is where the two are similar or even the same. We will see that every working definition of AI corresponds to an abstraction of the human mind that describes the mind from a certain point of view, or at a certain level of abstraction, under the belief that it is what intelligence is really about. This abstraction guides the construction of a computer system that is similar to a human mind in that sense, while neglecting other aspects of the human mind as irrelevant or secondary."

"First, the concept began as an attribute of human beings, and is especially about the mental or intellectual capability displayed by humans. Therefore it is historically anthropocentric, and if a working definition of intelligence could even exclude a normal (average) human being, it would not be acceptable – no matter how good such a definition is in other aspects, it is not about the intelligence as we intuitively understand, but about something else"

"First, a definition can be about either the usage of a word or the content of a concept expressed by a word, and in most situations the debate on the meaning of AI is more about the latter than about the former, though it is the former that is directly mentioned. In the current discussion, if “artificial intelligence” was replaced by “computer intelligence” or “machine intelligence,” then the underlying problem would not change much. The same is true if the concept is expressed not in English, but another human language. After all, the key issue is not in the choice of the words, but in the idea expressed by them, so this discussion is largely language independent"

Chollet, Francois 

Summary: Outlines Wang's paper. Critisises lack of sharp boundaries of Wang's definitions. Conceptually agrees with Wang's definitios of intelligence (ties to psychology), but critiques lack of practical use through formality and implementation in real world contexts.

My thoughts: Overall strong arguments against a lack of formality and specificity from Wang. Mostly agree. However, these critiques are aimed at Wang's definition as a definite one, and not as a working one. Chollet claims it lacks fruitfullness, but some of that is explained by this misunderstanding. Furthermore, as we see today with LLMs (this is pre-LLM), benchmarks and real-life measurements are difficult to make, and doesn't in its own right defeat the purpose of a working definition. As such I find the overall argument very compelling, but feel that it's asking a little more than is possible of Wang, which I believe to be a part of Wang's point in the first place.

Relation to Wang: 4 of 5. 

Quotes: 

p. 27 - Sympathetic to Wang's view of cognition and intelligence. 

Coming from a cognitive developmental robotics background, I find myself highly sympathetic to Wang’s view of cognition. While this vision of intelligence has not been very popular in the field of AI, it is historically ancient within the field of psychology. Wang mentions Jean Piaget, who built his own theory of intelligence development based on a similar vision in the 1940s (Piaget, 1947), but these ideas preexisted Piaget—we note that one of the very first researchers to ever attempt to rigorously characterize and measure intelligence, Alfred Binet, defined intelligence in rather similar terms (Binet and Simon, 1916, pp. 42–43):

“It seems to us that in intelligence there is a fundamental faculty, the alteration or the lack of which, is of the utmost importance for practical life. This faculty is [. . . ] the faculty of adapting one’s self to circumstances.

Istvan S. N. Berkeley

Summary: Dismisses Wang's definition by using ELIZA. Notes that many information-processing systems woudl fail and therefore the proposal is defective. Says perhaps Wang could be aspirational for AI design.

My thoughts: ELIZA example is good, but the rest makes me wonder how thoroughly he read the paper... Wang is very specific about viewing his own contribution as a working definition, which would seem to be the same as the aspirational suggestion of Berkeley. I also agree that many systems falling outside the category makes it defective, on the contrary. Wang is thorough in defining both AI and its relation to his definition - many systems shouldn't be considered intelligent in his view. I find the contribution a little strange.

Wang relation: 3 of 5 - Considers it aspirational for AI design, yet seems negative around it.

Quotes:

p. 24 - ELIZA in relation to Wang's definition

"Although on the face of it, this sounds laudable, it becomes problematic when applied in practice. For instance, Weizenbaum’s classic program ELIZA (Weizenbaum, 1966) appears to trivially satisfy this definition, in so much as, given that the system is nearly fifty years old, it clearly ran with limited resources, at least as compared to modern computational systems. However, as ELIZA had a means of handling a wide range of inputs, often by turning assertions into questions, it clearly had a means of ‘adapting to its environment.’ Yet, most contemporary theorists would be reticent to ascribe intelligence to ELIZA. So, the definition fails by including too many information-processing systems."

Gianluca Baldassarre & Giovanni Granato

Summary: Claims Wang's definitio is too narrow. Their definition relies on multiple intelligences. A first principle in AI design inspired by the brain is flexibility of cognition to deal with incomplete knowledge. Second principle is using internal representations of the world by modifying existing knowledge to fit novel or familiar goals in new conditions (changing environment). Thirdly, stresses sub-symbolism as necessary for generalisation, e.g. deep-learning, neural networks.

My thoughts: It reads like they are disagreeing a fair bit, but align most of the time. The idea of internal representation always seem strange to me, and they liken it to the machine possessing imagination. However, re-creating 'eyes' by using cameras for robots merely display a heuristic to the humans building the robot, it itself has no utility to a robot, because an image as shown by a camera is a representation of the world that makes sense to beings with eyes. (SIDE THOUGHT: When video chatting with someone and a dog comes by that you know, it never seems to recognise you - why?)

Furthermore, in their example they are bruteforcing a system that is mathematically solvable by using reinforcement learning - hardly surprising that it succeeds, but not really a demonstration of what they're describing.

Wang relation: 4 of 5. Overarching concepts seem very similar to me, thought worded quite differently.

Definition of intelligence:

"General-domain intelligence is the capacity of goal-directed agents to flexibly accomplish novel goals in novel conditions/domains by building the knowledge they lack through the manipulation of internal representations and by actively seeking such knowledge in the external environment." (p. 21)

Quotes:

p. 20 - Manipulating internal representations = imagination

"This internal manipulation of representations, at the basis of imagination (Seepanomwan et al., 2015) and problem solving, allows the brain to modify the previously acquired knowledge to produce the lacking knowledge needed to accomplish novel goals or familiar goals in novel conditions/domains."

p. 21 - Flexibility over adaptation

"We think these features do not fully capture what is needed to cope with situations involving partial knowledge (incidentally, we prefer to talk of flexibility rather than ‘adaptation’, a term mainly referring to biological intelligence but less suitable for AI). Our proposal allows the specification of flexibility (adaptation) by stating that in order to flexibly solve goals for which it lacks knowledge, an intelligent agent not only searches information and knowledge in the environment but it also actively builds it internally (manipulation of internal representations).

Partly agree with this - if I am solving a math issue that I do not understand, that represents a problem. If it is a problem that is similar to what I have seen before, my existing knowledge is only sufficient if 'shuffled', i.e. it is flexible. Learning something, however, seems to indicate adaptation, i.e. something fundamentally changes. Like first learning to understand vectors in 3d space rather than 2d - it might be more accurately described as an adaptation, because it fundamentally changes something in the knowledge, and isn't just a flexibility of the current knowledge. They seem to describe slightly different things.

Also, their use of the flexibility term is used to describe the way in which the robot solves problems, not its intelligence per se, so to me it seems that are discussing a different thing altogether.

Joscha Bach

Summary: Draws comparison to biology in defining life - the final definition hinged on knowledge that came over time. A functional definition of intelligence is intimately connected to creating AGI, Wang's working definition gives a direction for this. Considers Wang's definition weak. Questions logic of an agent having liited resources as relevant to its intelligence. Underlines the importance of understanding the nature of human intelligence to build an intelligent system.

Relation to Wang: Strong Disagree. States Wang's definition doesn't depend on the agent, but on the environment (which connects to Thórissons modification of Wang). 

Definition of Intelligence: "Unlike Wang, I don’t think of intelligence is the ability to use “bounded resources for unbounded problems,” but as the ability to deal with complexity by making models, usually in the service of a complex control task (such as the persistent existence of a complex agent in an entropic universe)" (p. 17).

My thoughts: Misses Wang's point in several ways. The idea of insufficient resources is not arbitrary, but rather a reflection of the conditions of intelligence in the real world. It is assumed that the agents 'universe' (as author puts it) doesn't need a relation to the physical world. Essentially hinges on every problem being able to be fully understood by the agent on its own terms (an unembodied agent in this case), which I think is impossible. This is also related to Wang's observation about being able to fully quantify the real world, which is not addressed. 

Connections: 

————————————————————————————————————————————

Quotes:

p. 17 - Understanding human intelligence (or nature) is key

"In other words, the ability to create a generally AI may be a necessary and sufficient condition for general AI. In this sense, the question whether human intelligence qualifies as generally intelligent is still an open one. Our inquiry into how to build a generally intelligent system is in no small part an attempt to understand our own nature."

Summary: Lack of concensus on both intelligence and AGI. Highlights the conundrum of defining a concept as it is being scienfitically pursued—working definitions are essential but can lead research astray, needs frequent updating. ASI/AGI without learning or creativity is an oxymoron—the physical world demands both constantly. Intelligence is what figures out what knowledge is required in a world that can be considered constantly novel (at least to some degree). Specifies that goals and the ability to adapt and alter goals to different situations is a necessity for (A)GI (any general intelligence).

Relation to Wang: Strongly agrees. Doesn't alter, but constrains. Defines intelligence as "Discretionarily constrained adaptation under insufficient knowledge and resources" (p. 11). Her own words: "the adaptation may be arbitrarily constrained at the discretion of the agent itself or someone/something else".

My thoughts: Thórisson emphasises creativity, reasoning, learning as well as goals and adaptation of goals as absolute necessities to any form of general intelligence. This suggests a need for embodiment (or embodied intelligence), as well as self and intentionality.

Connections: 

• Boden - AI: Its nature and future (2016): Both emphasise creativity, learning, reasoning as essential. Thórisson doesn't mention hybridism, but her claims indicate embodiment of some kind.

————————————————————————————————————————————

Quotes:

p. 8 - Absurd amount of definitions of intelligence

"there seem to be so many things relevant to the phenomenon of intelligence that by the end of the last decade AI researchers had come up with over 28 (working) definitions (cf. (Legg and Hutter, 2007; Monett and Lewis, 2018)), a number that undoubtedly has grown since"

p. 8-9 - Specifies creativity/learning as necessary for handling constant novelty the physical world presents. Intelligence figures out which knowledge is needed. 

"The concept of ‘novelty’ is of course a gradient: The scenarios we encounter every day may be in many ways similar to the ones we encountered yesterday, but they are never identical down to every detail. And sometimes they are very different. But due to the impossibility of defining everything up front, and knowing everything beforehand, both natural and artificial intelligences must rely on creativity and learning as a major way to operate in the physical world. Because the world presents this wealth of novelty, we are constantly in a state of lacking knowledge. The purpose of intelligence is to figure out what knowledge is needed, produce that knowledge by any means necessary, and allow us to move on. This is the way both natural and artificial intelligences can handle novel problems, tasks, goals, environments and worlds. No task takes zero time or energy to perform—and neither does thinking. Time and energy present additional constraints on this effort, and cannot be removed. Addressing these challenges is what all intelligent agents must be able to do, as well as any and all other constraints that may come their way. This is why we continue to push our machine’s abilities increasingly towards the ‘general’ end of that spectrum."

p. 10 - There can be no intelligence without goals and adaptive goal-setting, therefore without self or intentionality.

"There is no denying that intelligence requires the ability to create sensible goals and use reasoning to manage them—goals define what is accepted and not accepted when addressing some task, environment, or problem; by specifying their constraints. Goals are thus a kind of temporally-bounded requirement on intelligence, and trying to create a generally intelligent machine that does not have this ability seems tautological." [...] "Based on an agent’s changing needs and wishes relative to its environment, an agent forms multiple (explicit or implicit) sub-goals, which it uses in combination with reasoning to cumulatively build up a collection of reliable and actionable knowledge, to predict, achieve goals, explain, and re-create the phenomena that it models from experience (Bieger, Thorisson, and Steunebrink, 2017; Thórisson et al., 2016)"