YEE-KING HOME

MATTHEW YEE-KING
EASy MSc 1998-2000
Artificial Intelligence and Creativity Essay
25.04.2000


Preparation and Spontaneity


Introduction


Starting with Perkins [1] concept - using Klondike spaces as a yardstick to measure the quality of a creative system, I will discuss the two phases of the creative process. I call these phases the preparation phase and the spontaneous phase. I believe that preparation is an underrated yet integral part of the creative process. Preparation is often ignored or provided 'for free' in computational models. Artificial creative systems that explore Klondike spaces will be used to illustrate the impressive capabilities of computational models in the spontaneous phase of creativity. The characteristics of humans and other natural systems that contribute to the preparatory phase are discussed. I will then introduce my own project in the context of a human - computer combined system that exploits the strengths of both parties to produce exciting, novel sounds.

The Klondike Yardstick

Perkins[1] discusses creative systems and their movement through a metaphorical 'Klondike space'. A creative system is said to generate novelty in an adaptive fashion. This view of creative systems balances novelty and worth by rewarding adaptations that are both novel and effective. The gold nuggets in the Klondike space are these novel, effective adaptations. The creative system, be it a human or a computer model, explores the Klondike space using various heuristics. The positive development of these heuristics over time show the system is adapting to its Klondike space such as to become a more effective gold digger. There are two levels of adaptation in effect here. Let us say the Klondike space represents the set of possible solutions to a problem such as 'build a car engine that is both efficient and powerful'. An engineer explores the space searching for a novel solution to the problem. As his ideas move through the Klondike space, they can be said to be adapting to the problem space under the direction of the engineer. While this exploration is going on the engineer becomes more experienced at the art of directing his ideas in their adaptive search. As such the engineer himself is adapting to the properties of that particular problem space at the same time as his ideas are adapting to the problem. An example of 'ideas-level adaptation' could be experimenting by varying the volume of a particular part of the engine. An experienced engineer, who is well adapted to the terrain of an engine design Klondike space might see this as a dead end, having tried such things before. The two forms of adaptation have a different feel - the generation of ideas has a spontaneous feel whereas the judgement of these ideas has a longer term, preparatory feel.


The geography of the Klondike space has certain features that make successful movements therein non-trivial:


1. Rarity - gold is rare and therefore hard to find.
2. Isolation - gold is distributed in pockets with little to drive movement from one to the other.
3. Oasis - linked to isolation, this is the tendency to settle in local maxima exploiting the dwindling resources available.
4. Plateau - presented with many selectively neutral paths from one oasis to the next, which one do you take? This is again linked to the isolation feature in that plains separate the oases.

A creative system's ability to travel around the Klondike space, dealing with these difficulties and pitfalls is used as a yardstick to rate its quality. I am proposing an overview of creativity that differentiates two phases. There is a preparation phase that can be viewed as a creative system adapting heuristics with which to explore the Klondike space. Then there is a spontaneous phase where local movement occurs in the space. The first phase plays an important, on-going role in defining a system's path through the Klondike. Johnson-Laird [2] discusses improvisation in modern jazz:


'Musicians ... learn to improvise by improvising; the process takes years to master'


Improvisation is real-time exploration of a Klondike space, an example of the second, spontaneous phase of the creative process. But what are the constraints of this space? The notes available for the improvisation are constrained by the chord sequence and the player's skill, so the musician needs physical ability, knowledge of musical rules and the chord sequence on tap. This knowledge/ skill is gained and thus these constraints are defined over years of playing and study. Many computer systems are provided with this knowledge and skill for free however. Cohen's figure drawing program [3] is provided with an explicit grammar system that it manipulates to produce results - it did not observe human beings in motion and build its own grammar. I think it would be rather more of a challenge to design a system that did build its own grammar and then manipulated it. That is not to say that every musician who composes a piece has invented the entire set of musical rules from scratch. Rather they have built their own interpretation of these rules from what they have learnt and experienced. This process is integral to the way in which they then manipulate the rules when they create a new piece of music. The thing that separates one creative person or system from the next is this personalised interpretation of publicly available knowledge. In this respect, what marks the difference between a truly creative person or system and a person or system that gets stuck in a dwindling oasis to which they followed someone else's directions? This is a separate issue to the question of personal novelty and societal novelty or P-Creative and H-Creative acts [3]. Johnson-Laird [4] describes this judgement of quality as a more distant role for his computational model of creativity and I see this question as beyond the scope of this writing. To address the question of differentiation between oasis dwellers and plateau striders I will first discuss a computer system that I believe has certain elements to separate it from many other systems.

Copycat

Hofstadter [5] states that:

'[the] ability to adapt to a new situation by recognizing its shared essence with an old situation is the crux of learning'

The ability of a creative system to generate low-level interpretations of its substrate is closely linked to this analogical ability. These low-level interpretations are built in the preparatory phase. Artificial creative systems that are provided with these low-level interpretations for free are not participating in this phase. Hofstadter's Copycat system does some work in this area. The system constructs interpretations of its substrate (sequences of letters) from semantic building blocks. It then manipulates these interpretations autonomously in order to map them onto other substrates. Explicitly, Copycat is given a sequence of letters and a transformation of that sequence of letters e.g.


ABC > ABD

It then makes suggestions as to how a new sequence of letters could be transformed in an analogical way, e.g.

XYZ > ?

The preparatory phase involves first building an interpretation of the two initial sequences and second building an interpretation of the transformation from one sequence to the next. The transformational rule is then applied to the new sequence and the results are automatically judged. The rules can 'slip' if the transformation is not satisfactory for the new sequence. This is the spontaneous phase, where pre formed ideas from the preparatory phase are applied to a problem and tweaked if necessary. Going back to the jazz soloist, the spontaneous soloing phase is carried out on the sturdy foundations of years of practice and learning.

The main feature of Copycat that sets it aside from many other computer systems, for example Cope's EMI system [6] and Cohen's grammar based drawing programs is its inclusion of systems for both the preparatory and spontaneous phases. This system is definitely a plateau strider rather than an oasis dweller.

Stick to what you're good at!

Designing the Copycat program with its implementation of both the creative phases was clearly non-trivial in the extreme. It is also a system that is limited by being rather too specialized, with a 'highly idealised' domain to explore[7]. When building useful computer tools rather than models of creativity this is definitely an important issue. Conceiving a flexible artificial system that deals only with the spontaneous phase is arguably easier. In this case the onus is on the user to deal with the issues of P- and H-Creativity and societal view of worth. The increased simplicity of building spontaneous-only systems is especially true in the case of music composition systems where the rules are mathematically based, mathematical rules being a preferred substrate for computer manipulation. Cope[9] provides a thorough coverage of the history of compositional systems from the haunting Aeolian harp through to his EMI system (Experiments in Musical Intelligence). The Aeolian harp makes sounds from its strings when the wind blows on them. The preparatory phase is a given - it consists of the construction of the harp which defines its conceptual space constraints. In the spontaneous phase the harp 'composes' or at least produces melodies. There is a clear lineage from such simple systems that generate melodies stochastically, constrained by the physically available note pitches and timbres through to computational systems that generate sequences of notes constrained by musical rules, resolving choices stochastically. Cope's EMI system generates music in the style of different composers based on statistical analysis (and other analysis) of their work. The results are apparently compelling - given the right constraints the system can produce sonatas in the style of Beethoven or Baroque pieces in the style of Bach.

Humans as Creators - Breaking the Rules

So computer systems can simulate/ perform/ carry out (call it what you will) both creative phases after a fashion. However, building an artificial system that carries out the first phase is difficult, since the system must find meaning in the substrate for itself. When the substrate has emotional content that contributes significantly to its nature, how can an emotion free computer system be expected to interpret? Artificial systems are probably better suited to the second phase as it can be carried out effectively through the rule-based manipulation of pre-defined representational building blocks. What is it about the human approach to the two creative phases that makes the results so compelling?

A human may create in an interesting way because their learning is imperfect - they make mistakes in the preparatory phase. When these mistakes are made, they are not always rejected; sometimes they are integrated into the human's interpretation of the substrate. Can these mistakes be viewed as emergent from the interaction between the musician and thier musical training, producing frozen accidents stabilised by their environment as discussed in [9]? This is the manipulation of the conceptual space discussed in [10] This flexibility helps humans to cross the Klondike plains. Computational systems focused on the spontaneous phase miss out on this vital part of constraint manipulation. For example, if you explicitly represent musical knowledge or a musical style within a computer and the computer generates music from this, you are stuck in a dwindling oasis. Without a flexible interpretation of that musical style you are not going anywhere.

What human characteristics other than imperfect learning are involved in the constraint manipulation and how can these be implemented in an artificial system? Elements of impatience may help. A composer could hardly be satisfied by the reproduction of either peoples' styles (although many modern musicians seem to be!). If the music style generated by an artificial system is too similar to the style it has been taught, maybe it could become impatient and make random mistakes - corrupting the rules by which it manipoulates its building blocks. Competition, the super parallel tool used by nature in its exploration of the Klondike - like fitness landscapes, is another source of effective constraint manipulation. Perkins offers the massive parallelism of natural selection as a tool for Klondike navigation. Competition reduces the tendency to get stuck in oases, or local maxima, by rewarding effective adaptations that allow the adapter to stand out from the crowd. If there is no crowd, how can you stand out from it?

The physical effort of a creative act is also very important. Look at Newton, grinding his lenses by hand. Further, his obsession with the observation of celestial bodies rendered him almost blind for days. There is some sort of extreme mental state associated with creative acts - be it in the form of an obsession or blissful artistic catharsis. This stressful mental state may be the key to the manipulation of constraints, where it is possible to reinterpret or misinterpret these constraints. Copycat's rule generating slipnet is a form of 'artiste'. If it is not satisfied with its analogical output, it shifts the rules around, throwing old schemes in the bin. Systems focused on the spontaneous phase lack this vital flavour. They are stuck in an oasis, following the provided rules.

Joining Forces

There is a class of creative computer systems that allows the human operator to use their hard earned knowledge to provide direction for an iterative artificial creative process. The artificial side of the system generates novelty and the human grades the adaptive qualities of this novelty. The user directs the system such that its output gradually adapts to the human user's idea of what they want it to be. In turn the human user's idea of what they want the system to do is adapting to what the system does. The basics of such a system are this:


1. The user is presented with a selection of novel 'solutions'. These may be sounds [11] or visual patterns [12] or anything else. Each solution is stored by the system as a genotype that can be manipulated.
2. The user uses their skills as a critic to choose their favourite solution. Its genotype is then processed such that several variants are generated
3. The user is presented with the selection of variants of the chosen solution.
4. back to 2.


This system exploits the fact that humans are better critics than they are creators. E.g. you can play a piece of music to just about anyone and be confident that they will know whether they like it or not. This critical ability stands apart from any ability they may or may not have to write a comparable piece of music. As computer software tools, especially creative ones, become more and more inscrutable and require more of a commitment to learn them, a space is appearing for much simpler programs. Why obscure the potential products of a piece of software behind 300 dialogue boxes when you can present the user with end products straight away. Human-selection based evolutionary software allows the user to manipulate many parameters in parallel in a natural, implicit way.


I have written my own program, which I call 'Audiomorph' along these lines. It is an implementation of a human selection driven evolutionary program inspired by Dawkins' biomorph[13] program and Karl Sims' work [12], as described briefly above. In these two examples, the user participates in the evolution of a visual form by selecting candidates from a breed of similar forms generated by the system. In my system the twist is that you evolve raw audio instead of visual forms. The user is presented with a population of sounds, which are generated from sound synthesis circuits. Each sound is made from a unique, randomly generated circuit. The user chooses their favourite sound and its circuit passes into the next generation. The chosen circuit is mutated into twenty variants. The user then chooses one of these to go to the next generation, and so on. Here we have an exploration of a Klondike space. The gold is a sound that the user likes. The route taken is defined by a combination of the stochastic generation of novelty and the 'expert' value assignment on the part of the user. There are different ways to use the system. The user may have a distinct idea of the sound they are looking for or user may just run through a few generations waiting for an inspiring sound. Therefore the value assignment has elements of feedback in it - the novelty generated by the system contributes to the trajectory taken through the Klondike. Part of the inspiration for the project was a desire to use computer power to augment human creativity. In the effect, it has been successful. In terms of audio circuit design, the system has equivalent flexibility to an explicit system where the user hand designs the circuit in that the state space of potential circuits is the same. The patience and critical faculties of the user limit the system. Also the dependence on stochastic principles for the generation of novelty means the user is at the mercy of the random number generator for novel ideas. It is possible to control the system so you 'sniff' around a particular area of the Klondike space for a while waiting for a particular sonic characteristic to arise. As such it is possible to fight the dependence upon random idea generation. Future enhancements may allow the user to hand design the initial circuit, or load in a preset circuit that is known to make a good sound and evolve from there. Alternatively they could present the system with a sound or a set of sounds they like and these could be used as selection targets for the system.


Conclusion


I have shown that creativity can be separated into two distinct phases. Human abilities lend themselves to the first, preparatory phase, the amassment of knowledge and skills. Humans are excellent critics. Computational systems lend themselves to the second, spontaneous phase where the solution to the problem at hand moves around the Klondike space. There have been systems that attempt to perform both phases such as Copycat. This particular system explores a very constrained space however. Other systems have pitched themselves in the spontaneous phase, with considerable success. Cope's EMI generates music from a grammar which describes a particular composer's style. Biles' GenJam[14] improvises jazz melodies in a 'question and response' fashion using a genetic algorithm to generate melodic sequences. My own system[11] exploits the strengths of humans in the preparation phase and the strengths of computational systems in the spontaneous phase in order to develop interesting sounds. This system shows it is possible for a technically unskilled user to manipulate complex circuit parameters in an implicit fashion. As such I feel there is some worth in the principle of designing artistic tools that augment human creativity with an artificial, iterated system to generate novelty.

References

1. David N. Perkins 'Creativity beyond the Darwinian Paradigm' p119-143 in 'Dimensions of Creativity' ed. Margaret Boden. MIT Press 1996
2. Philip Johnson-Laird 'the Computer and the Mind' Second edition p260-265. Fontana press 1993.
3. Margaret Boden p76 in 'What is Creativity?' in 'Dimensions of Creativity' ed. Margaret Boden. MIT Press 1996
4. Philip Johnson-Laird 'the Computer and the Mind' Second edition p256. Fontana press 1993
5. D. Hofstadter 'The Copycat project' publisher not stated. 1991
6. D. Cope 'Computers and Musical style' p18. Oxford University Press 1991.
7. Margaret Boden 'What is Creativity?' in 'Dimensions of Creativity' (p100) ed. Margaret Boden. MIT Press 1996
8. D. Cope 'Computers and Musical style' (chapter 1). Oxford University Press 1991.
9. H. H. Pattee 'Simulations, Realisations and Theories of Life' (p388). In 'The Philosophy of Artificial Life' ed. Margaret Boden Oxford University Press 1996.
10. Margaret Boden 'What is Creativity?' in 'Dimensions of Creativity' (section beginning p79) ed. Margaret Boden. MIT Press 1996
11. Matthew Yee-King Audiomorph - an online program to evolve audio http://www.yeeking.freewire.co.uk/alife/start.htm. 2000
12. K. Sims "Artificial Evolution for Computer Graphics," in Computer Graphics (Siggraph '91 proceedings), p319-328 Vol.25, No.4, July 1991
13. R. Dawkins 'Blind Watchmaker' computer program
14. J. Biles 'GenJam: An Interactive Genetic Algorithm Jazz Improviser' http://www.acoustics.org/134th/biles1.htm