Information compression as a unifying principle in human learning, perception, and cognition, and as a foundation for the SP Theory of Intelligence
In an extended programme of research, Dr Gerry Wolff, with colleagues, is developing the SP System, meaning the SP Theory of Intelligence and its realisation in the SP Computer Model. An important part of this research is exploring the potential applications
of the SP System.
The overarching goal is the simplification and integration of observations and concepts across artificial intelligence, mainstream computing, mathematics, and human learning, perception,
and cognition.
A key idea in this research is that much of human learning, perception, and cognition may be understood as compression of information (more below). The powerful concept of SP-multiple-alignment, borrowed and adapted from the concept of ‘multiple sequence alignment’ in bioinformatics is an important development in this research.
Peer-reviewed papers and other documents about the research are detailed, with download links, on www.cognitionresearch.org/sp.htm.
The SP System
The SP System is conceived as a brain-like system that receives ‘New’ information via its senses and stores some or all of it, in compressed form, as ‘Old’ information, as shown schematically in Figure 1.
There is abundant evidence, described in the first of the 2019 papers under ‘Further reading’, below, that much of human learning, perception, and cognition, may be understood as compression of information. And in many cases, that compression of information may be understood as a process of searching for patterns that match each other and then merging or ‘unifying’ two or more such patterns to make one. The expression “information compression via the matching and unification of patterns” may be shortened to ‘ICMUP’.
Much of human learning, perception, and cognition may be understood as compression of information.
If, when we are looking at something, we close our eyes for a moment and then open them again, we normally see ‘the same’ as what we saw before, as shown schematically in Figure 2. The merging of the ‘before’ and ‘after’ views may be seen as ICMUP.
A well-known technique for compressing information, called ‘chunking-with-codes’, means using a relatively short ‘code’ to stand for and replace each of two or more instances of a relatively large ‘chunk’ of information. So a relatively short name like ‘New York’ can stand for the relatively huge ‘chunk’ of information which is all knowledge of the renowned city itself.1
A little reflection shows that every noun, verb, adjective, or adverb in a natural language may be seen to be a short code for a relatively large chunk of information. Every natural language may be seen to be a means of achieving very high levels of information compression.
SP-multiple-alignment
The previously mentioned concept of SP-multiple-alignment may be seen to provide seamless integration of six main variants of ICMUP, and is thus a powerful means of compressing information.
A large part of the workings of the SP System is a process of creating and selecting ‘good’ SP-multiple-alignments like the one shown in Figure 3, which shows how the system may analyse or ‘parse’ a sentence (in row 0) into words and other grammatical structures.2
Because of the complexity of the search space, heuristic techniques are needed, building SP-multiple-alignments in stages and discarding all but the best partly-developed SP-multiple-alignments at the end of each stage.
An SP-multiple-alignment is ‘good’ if it can yield a relatively large amount of information compression by the merging or ‘unification’ of patterns that match each other within the SP-multiple-alignment.
SP-multiple-alignments can do much more than parse sentences. They are largely responsible for the versatility of the SP System as summarised in the next section.
Information compression is so much embedded in our thinking and seems so natural and obvious that it is easily overlooked.
The SP-multiple-alignment concept has the potential to be as significant for the understanding of intelligence as is DNA in biological sciences. It could prove to be the ‘double helix’ of intelligence.
Strengths and potential of the SP System
Largely because of the versatility of the SP-multiple-alignment construct, the SP System has strengths and potential: in several aspects of intelligence, including several kinds of reasoning; in the representation of diverse kinds of knowledge; and in the seamless integration of diverse aspects of intelligence and diverse kinds of knowledge, in any combination. Some of the strengths and potential of the SP System are shown schematically in Figure 4.
Potential applications of the SP System
The SP System has potential in several areas of application including: helping to solve problems with big data; helping to develop intelligence in autonomous robots; in the processing of natural language; in the development of an intelligent database system; in medical diagnosis; and more. Peer-reviewed papers and other documents about potential applications of the SP System are detailed, with download links, on www.cognitionresearch.org/sp.htm.
SP-Neural
Abstract concepts in the SP System map quite well into SP-Neural, a version of the SP System expressed in terms of neurons and their interconnections, described in the 2016 paper detailed opposite. SP-Neural is quite different from the currently-popular ‘deep neural networks’ (DNNs).
The SP Machine
Dr William J Teahan of Bangor University, Dr Vasile Palade of Coventry University, and Dr Gerry Wolff are aiming to develop the SP Computer Model into an industrial-strength SP Machine, starting with the application of high levels of parallel processing. A paper by Dr Palade and Dr Wolff that describes a roadmap for the development of the SP Machine is detailed under “Further reading”, opposite.
Personal Response
What prompted you to research and develop the SP System?
<>Information compression as an important principle in the workings of brains and nervous systems was the subject of fascinating lectures given by Dr Horace Barlow (now Professor Barlow FRS) when I was a student at Cambridge University. My later research developing computer models of language learning confirmed the importance of information compression in the learning of language. Later experience in commercial software development suggested the potential of a new kind of computing system with a central role for information compression. Experience in developing the SP System, with information compression at centre stage, has progressively expanded and refined the ideas.
What do you envisage as the next development of the SP System?
<> As mentioned earlier, three of us are aiming to create a first version of the SP Machine, derived from the SP Computer Model with the application of high levels of parallel processing. A peer-reviewed paper describes a roadmap for the development of an industrial-strength SP Machine.
Acknowledgements
All the figures are reproduced with permission from Wolff (2019) (in “Further reading”). The landscape in Figure 2 is from Wallpapers Buzz (www.wallpapersbuzz.com), reproduced with permission.
Endnotes
1 Chunking-with-codes may be seen as an example of ICMUP because, in effect, multiple instances of the relatively large chunk of information have been reduced to one.
2 At present, the SP Computer Model works only with one-dimensional patterns, but it is envisaged that it will be generalised to work with 2D patterns. With relatively large basic symbols such as letters or words, the SP System has the flavour of ‘symbolic AI’. With relatively small basic symbols such as pixels in an image, the SP System may be more like ‘non-symbolic AI’.
What’s up to all, the contents existing at this website are genuinely awesome
for people knowledge, well, keep up the good work fellows.