Sound Research WIKINDX

Summary: Lack of concensus on both intelligence and AGI. Highlights the conundrum of defining a concept as it is being scienfitically pursued—working definitions are essential but can lead research astray, needs frequent updating. ASI/AGI without learning or creativity is an oxymoron—the physical world demands both constantly. Intelligence is what figures out what knowledge is required in a world that can be considered constantly novel (at least to some degree). Specifies that goals and the ability to adapt and alter goals to different situations is a necessity for (A)GI (any general intelligence).

Relation to Wang: Strongly agrees. Doesn't alter, but constrains. Defines intelligence as "Discretionarily constrained adaptation under insufficient knowledge and resources" (p. 11). Her own words: "the adaptation may be arbitrarily constrained at the discretion of the agent itself or someone/something else".

My thoughts: Thórisson emphasises creativity, reasoning, learning as well as goals and adaptation of goals as absolute necessities to any form of general intelligence. This suggests a need for embodiment (or embodied intelligence), as well as self and intentionality.

Connections: 

• Boden - AI: Its nature and future (2016): Both emphasise creativity, learning, reasoning as essential. Thórisson doesn't mention hybridism, but her claims indicate embodiment of some kind.

————————————————————————————————————————————

Quotes:

p. 8 - Absurd amount of definitions of intelligence

"there seem to be so many things relevant to the phenomenon of intelligence that by the end of the last decade AI researchers had come up with over 28 (working) definitions (cf. (Legg and Hutter, 2007; Monett and Lewis, 2018)), a number that undoubtedly has grown since"

p. 8-9 - Specifies creativity/learning as necessary for handling constant novelty the physical world presents. Intelligence figures out which knowledge is needed. 

"The concept of ‘novelty’ is of course a gradient: The scenarios we encounter every day may be in many ways similar to the ones we encountered yesterday, but they are never identical down to every detail. And sometimes they are very different. But due to the impossibility of defining everything up front, and knowing everything beforehand, both natural and artificial intelligences must rely on creativity and learning as a major way to operate in the physical world. Because the world presents this wealth of novelty, we are constantly in a state of lacking knowledge. The purpose of intelligence is to figure out what knowledge is needed, produce that knowledge by any means necessary, and allow us to move on. This is the way both natural and artificial intelligences can handle novel problems, tasks, goals, environments and worlds. No task takes zero time or energy to perform—and neither does thinking. Time and energy present additional constraints on this effort, and cannot be removed. Addressing these challenges is what all intelligent agents must be able to do, as well as any and all other constraints that may come their way. This is why we continue to push our machine’s abilities increasingly towards the ‘general’ end of that spectrum."

p. 10 - There can be no intelligence without goals and adaptive goal-setting, therefore without self or intentionality.

"There is no denying that intelligence requires the ability to create sensible goals and use reasoning to manage them—goals define what is accepted and not accepted when addressing some task, environment, or problem; by specifying their constraints. Goals are thus a kind of temporally-bounded requirement on intelligence, and trying to create a generally intelligent machine that does not have this ability seems tautological." [...] "Based on an agent’s changing needs and wishes relative to its environment, an agent forms multiple (explicit or implicit) sub-goals, which it uses in combination with reasoning to cumulatively build up a collection of reliable and actionable knowledge, to predict, achieve goals, explain, and re-create the phenomena that it models from experience (Bieger, Thorisson, and Steunebrink, 2017; Thórisson et al., 2016)"

Joscha Bach

Summary: Draws comparison to biology in defining life - the final definition hinged on knowledge that came over time. A functional definition of intelligence is intimately connected to creating AGI, Wang's working definition gives a direction for this. Considers Wang's definition weak. Questions logic of an agent having liited resources as relevant to its intelligence. Underlines the importance of understanding the nature of human intelligence to build an intelligent system.

Relation to Wang: Strong Disagree. States Wang's definition doesn't depend on the agent, but on the environment (which connects to Thórissons modification of Wang). 

Definition of Intelligence: "Unlike Wang, I don’t think of intelligence is the ability to use “bounded resources for unbounded problems,” but as the ability to deal with complexity by making models, usually in the service of a complex control task (such as the persistent existence of a complex agent in an entropic universe)" (p. 17).

My thoughts: Misses Wang's point in several ways. The idea of insufficient resources is not arbitrary, but rather a reflection of the conditions of intelligence in the real world. It is assumed that the agents 'universe' (as author puts it) doesn't need a relation to the physical world. Essentially hinges on every problem being able to be fully understood by the agent on its own terms (an unembodied agent in this case), which I think is impossible. This is also related to Wang's observation about being able to fully quantify the real world, which is not addressed. 

Connections: 

————————————————————————————————————————————

Quotes:

p. 17 - Understanding human intelligence (or nature) is key

"In other words, the ability to create a generally AI may be a necessary and sufficient condition for general AI. In this sense, the question whether human intelligence qualifies as generally intelligent is still an open one. Our inquiry into how to build a generally intelligent system is in no small part an attempt to understand our own nature."

Gianluca Baldassarre & Giovanni Granato

Summary: Claims Wang's definitio is too narrow. Their definition relies on multiple intelligences. A first principle in AI design inspired by the brain is flexibility of cognition to deal with incomplete knowledge. Second principle is using internal representations of the world by modifying existing knowledge to fit novel or familiar goals in new conditions (changing environment). Thirdly, stresses sub-symbolism as necessary for generalisation, e.g. deep-learning, neural networks.

My thoughts: It reads like they are disagreeing a fair bit, but align most of the time. The idea of internal representation always seem strange to me, and they liken it to the machine possessing imagination. However, re-creating 'eyes' by using cameras for robots merely display a heuristic to the humans building the robot, it itself has no utility to a robot, because an image as shown by a camera is a representation of the world that makes sense to beings with eyes. (SIDE THOUGHT: When video chatting with someone and a dog comes by that you know, it never seems to recognise you - why?)

Furthermore, in their example they are bruteforcing a system that is mathematically solvable by using reinforcement learning - hardly surprising that it succeeds, but not really a demonstration of what they're describing.

Wang relation: 4 of 5. Overarching concepts seem very similar to me, thought worded quite differently.

Definition of intelligence:

"General-domain intelligence is the capacity of goal-directed agents to flexibly accomplish novel goals in novel conditions/domains by building the knowledge they lack through the manipulation of internal representations and by actively seeking such knowledge in the external environment." (p. 21)

Quotes:

p. 20 - Manipulating internal representations = imagination

"This internal manipulation of representations, at the basis of imagination (Seepanomwan et al., 2015) and problem solving, allows the brain to modify the previously acquired knowledge to produce the lacking knowledge needed to accomplish novel goals or familiar goals in novel conditions/domains."

p. 21 - Flexibility over adaptation

"We think these features do not fully capture what is needed to cope with situations involving partial knowledge (incidentally, we prefer to talk of flexibility rather than ‘adaptation’, a term mainly referring to biological intelligence but less suitable for AI). Our proposal allows the specification of flexibility (adaptation) by stating that in order to flexibly solve goals for which it lacks knowledge, an intelligent agent not only searches information and knowledge in the environment but it also actively builds it internally (manipulation of internal representations).

Partly agree with this - if I am solving a math issue that I do not understand, that represents a problem. If it is a problem that is similar to what I have seen before, my existing knowledge is only sufficient if 'shuffled', i.e. it is flexible. Learning something, however, seems to indicate adaptation, i.e. something fundamentally changes. Like first learning to understand vectors in 3d space rather than 2d - it might be more accurately described as an adaptation, because it fundamentally changes something in the knowledge, and isn't just a flexibility of the current knowledge. They seem to describe slightly different things.

Also, their use of the flexibility term is used to describe the way in which the robot solves problems, not its intelligence per se, so to me it seems that are discussing a different thing altogether.

Istvan S. N. Berkeley

Summary: Dismisses Wang's definition by using ELIZA. Notes that many information-processing systems woudl fail and therefore the proposal is defective. Says perhaps Wang could be aspirational for AI design.

My thoughts: ELIZA example is good, but the rest makes me wonder how thoroughly he read the paper... Wang is very specific about viewing his own contribution as a working definition, which would seem to be the same as the aspirational suggestion of Berkeley. I also agree that many systems falling outside the category makes it defective, on the contrary. Wang is thorough in defining both AI and its relation to his definition - many systems shouldn't be considered intelligent in his view. I find the contribution a little strange.

Wang relation: 3 of 5 - Considers it aspirational for AI design, yet seems negative around it.

Quotes:

p. 24 - ELIZA in relation to Wang's definition

"Although on the face of it, this sounds laudable, it becomes problematic when applied in practice. For instance, Weizenbaum’s classic program ELIZA (Weizenbaum, 1966) appears to trivially satisfy this definition, in so much as, given that the system is nearly fifty years old, it clearly ran with limited resources, at least as compared to modern computational systems. However, as ELIZA had a means of handling a wide range of inputs, often by turning assertions into questions, it clearly had a means of ‘adapting to its environment.’ Yet, most contemporary theorists would be reticent to ascribe intelligence to ELIZA. So, the definition fails by including too many information-processing systems."

Chollet, Francois 

Summary: Outlines Wang's paper. Critisises lack of sharp boundaries of Wang's definitions. Conceptually agrees with Wang's definitios of intelligence (ties to psychology), but critiques lack of practical use through formality and implementation in real world contexts.

My thoughts: Overall strong arguments against a lack of formality and specificity from Wang. Mostly agree. However, these critiques are aimed at Wang's definition as a definite one, and not as a working one. Chollet claims it lacks fruitfullness, but some of that is explained by this misunderstanding. Furthermore, as we see today with LLMs (this is pre-LLM), benchmarks and real-life measurements are difficult to make, and doesn't in its own right defeat the purpose of a working definition. As such I find the overall argument very compelling, but feel that it's asking a little more than is possible of Wang, which I believe to be a part of Wang's point in the first place.

Relation to Wang: 4 of 5. 

Quotes: 

p. 27 - Sympathetic to Wang's view of cognition and intelligence. 

Coming from a cognitive developmental robotics background, I find myself highly sympathetic to Wang’s view of cognition. While this vision of intelligence has not been very popular in the field of AI, it is historically ancient within the field of psychology. Wang mentions Jean Piaget, who built his own theory of intelligence development based on a similar vision in the 1940s (Piaget, 1947), but these ideas preexisted Piaget—we note that one of the very first researchers to ever attempt to rigorously characterize and measure intelligence, Alfred Binet, defined intelligence in rather similar terms (Binet and Simon, 1916, pp. 42–43):

“It seems to us that in intelligence there is a fundamental faculty, the alteration or the lack of which, is of the utmost importance for practical life. This faculty is [. . . ] the faculty of adapting one’s self to circumstances.