MUSICAL CONTENT

CROSS REFERENCE TO RELATED APPLICATIONS [0001] This application claims all benefit, including priority, of United States Provisional Patent Application Serial No. 61/651 ,429, filed May 24, 2012, entitled SYSTEM AND METHOD FOR GENERATING A PLURALITY OF CUSTOMIZED PERCUSSION PATTERNS, the entire contents of which is incorporated herein by this reference.

FIELD OF THE INVENTION [0002] The present disclosure relates generally to representing musical content by musical content data structures.

BACKGROUND OF THE INVENTION

[0003] Generated percussion accompaniment first appeared in auto-accompaniment technology found on the Rone organs, where static drum patterns were played back at various tempos.

[0004] The next major step in this evolution was the pattern based drum machine, where the standard interface is a step input for each instrument, and a swing processor added to change feel. However, although the pattern based drum machine technology introduced a new genre of electronic music, it was determined to be a failure at producing patterns that had a realistic, human feel. To overcome this difficulty, drum machines moved towards sampling, wherein loops can be triggered to perform human sounding performances from recorded wave files.

[0005] What is needed is an improved system and method that allows customized percussion patterns to be generated in a way that addresses at least some of the above limitations.

SUMMARY OF THE INVENTION

[0006] The present disclosure relates to a system and method for generating a

representation of musical content. In particular, the present disclosure relates to a system and method for generating a plurality of customized percussion patterns. [0007] In an embodiment, the system and method is adapted to generate a plurality of beats in a bar based on three structures. The first structure is a "microform" which is a representation of the metrical structure of a bar that is assumed to be divided into eighth-note-sized metric elements, which are referred to as "beats". Beats in a microform have a "salience", which indicates the perceptual importance of the beat as either strong, medium or weak. A second structure is a "nanoform" which is a representation of the metrical structure within a single 8th note beat within a bar. A third structure is a "molecule", which is a complete description of a bar of music, which consists of the bar's microform, associated nanoforms, voice modulators, and modifiers for relative note volume and swing.

[0008] In an aspect, the system and method generates a beat based on the following general steps: (i) a user choosing a microform for the bar being generated; (ii) a user choosing a vocabulary of nanoforms to be placed within 8-th note sized beats, where nanoforms are "deployed" to beats based on a beat's salience and in accordance with a "deployment function" that sets rules of how nanoforms are distributed across beats that possess the same salience; and (iii) one or more user defined constraints on relative note velocity, note "swing", and note voices for the generation through a set of "surface handlers".

[0009] In another aspect, there are two methods of allowing a user to browse the results of the beat generation: (i) a system that generates a result list created "on the fly"; and (ii) a system that breaks down the result set into three "strata" layers (corresponding to nanoforms deployed to strong, medium and weak beats) and allows users to combine members of each strata manually without having to sift through the entire result list.

[0010] In another aspect, a molecule editor allows the user to make fine-grained changes to a generated molecule during the beat generation process.

[0011 ] In another embodiment, there is proposed a persistent repository of molecules acting as an index of possible patterns, and as a backdrop for analysis of a given corpus. Different types of queries and reports that can be generated through analysis on the repository are described. Such reports may also be used as a basis for highlighting suggested molecule components as guides for user construction and generation in a creation or production context.

[0012] In yet another aspect, there is disclosed a cloud service that manages a molecule repository. [0013] Other aspects and features of the embodiments herein will become apparent to those ordinarily skilled in the art upon review of the following description of specific

embodiments in conjunction with the accompanying figures.

[0014] In this respect, before explaining at least one embodiment of the invention in detail, it is to be understood that the invention is not limited in its application to the details of construction and to the arrangements of the components set forth in the following description or the examples provided therein, or illustrated in the drawings. The invention is capable of other embodiments and of being practiced and carried out in various ways. Also, it is to be understood that the phraseology and terminology employed herein are for the purpose of description and should not be regarded as limiting.

[0015] In accordance with an aspect of the present invention, there is provided a system comprising at least one computing device providing or linking to a musical content data repository, the musical content data repository comprising: at least one musical content microform comprising a first musical content data structure comprising a representation of a metrical structure corresponding to a bar of music, the metrical structure comprising at least one metric element; at least one musical content nanoform comprising a second musical content data structure comprising a representation of a metric element; and at least one musical content molecule comprising a third musical content data structure comprising a selection of the at least one musical content microform and the at least one musical content nanoform, the at least one computing device comprising at least one user interface and is configured to: receive a first user input selecting at least one musical content microform of the musical content data repository; receive a second user input selecting at least one musical content nanoform of the musical content data repository; receive a third user input comprising at least one musical content constraint; arrange the selected at least one musical content nanoform within the at least one metric element of the at least one musical content microform based at least partly on a determined correspondence of nanoform salience, respectively associated with each musical content nanoform, to a salience range associated with the at least one musical content microform; and generate a user-customized musical content molecule based at least partly on the selected at least one musical content microform and the at least one musical content constraint, the user-customized musical content molecule describing a plurality of musical elements of a bar of music.

[0016] In accordance with an aspect of the present invention, there is provided a method, performed by at least one computing device providing or linking to a musical content data repository, the musical content data repository comprising: at least one musical content microform comprising a first musical content data structure comprising a representation of a metrical structure corresponding to a bar of music, the metrical structure comprising at least one metric element; at least one musical content nanoform comprising a second musical content data structure comprising a representation of a metric element; and at least one musical content molecule comprising a third musical content data structure comprising a selection of the at least one musical content microform and the at least one musical content nanoform, the at least one computing device comprising at least one user interface, the method comprising: receiving a first user input selecting at least one musical content microform of the musical content data repository; receiving a second user input selecting at least one musical content nanoform of the musical content data repository; receiving a third user input comprising at least one musical content constraint; arranging the selected at least one musical content nanoform within the at least one metric element of the at least one musical content microform based at least partly on a determined correspondence of nanoform salience, respectively associated with each musical content nanoform, to a predetermined salience range associated with the at least one musical content microform; and generating a user-customized musical content molecule based at least partly on the selected at least one musical content microform and the at least one musical content constraint, the user-customized musical content molecule describing a plurality of musical elements of a bar of music.

[0017] In one aspect, a computer implemented system for enabling at least one user to generate customized percussion patterns is provided comprising:

(a) one or more computers; and

(b) a percussion pattern generator included in the one or more computers , the one or more computers including or being linked to a percussion pattern generator including:

(i) a repository of:

(A) one or more microforms consisting of a representation of a metrical structure of a bar of music, the bar consisting of metric elements or beats;

(B) one or more nanoforms consisting of a representation of the metrical structure within a given beat; and (C) molecules consisting of a description of a bar based on the bar's microform and associated nanoforms, and optionally voice modulators and modifiers for relative note volume and swing;

(ii) one or more user interfaces enabling the at least one user to generate one or more beats by enabling the at least one user to:

(A) select a microform for a bar to be generated;

(B) select a set of nanoforms to be placed within metric elements using a series of rules for placing nanoforms within metric elements based on their salience and so that nanoforms distributed across multiple metric elements possess the same or similar salience; and

(C) apply one or more user constraints selected from a group including:

(I) relative note velocity;

(II) note swing;

(III) note voices (and associated parameters); and

(IV) note durations (for applicable voices); wherein the selection of the mirocroforms and associated nanoforms, and applying the one or more user constraints, enables the generation of user customized percussion patterns embodied in a user generated molecule.

[0018] In another aspect, the system is configured to generate dynamically one or more lists of relevant nanoforms, based on selection of a microform, the one or more lists being dynamically organized by the system as strong, medium, and weak metric elements ("strength characteristics"), thereby enabling users to combine nanoforms having different strength characteristics.

[0019] In another aspect, the note voices are generated using one or more surface handlers, optionally including additional handlers for control of individual voice parameters. [0020] In a still other aspect, the percussion pattern generator includes a molecule editor enabling fine user directed changes to the generated molecule.

[0021] In yet another aspect, a computer implemented method for enabling at least one user to generate customized percussion patterns is provided, comprising the steps of: (A) accessing a computer implemented percussion pattern generator including: a repository of one or more microforms consisting of a representation of a metrical structure of a bar of music, the bar consisting of metric elements or beats; one or more nanoforms consisting of a

representation of the metrical structure of one beat; and molecules consisting of a description of a bar based on the bar's microform and associated nanoforms, and optionally voice modulators and modifiers for relative note volume and swing; (B) using one or more user interfaces to generate one or more beats by: (i) selecting a microform for a bar to be generated; (ii) selecting a set of nanoforms to be placed within metric elements using a series of rules for placing nanoforms within metric elements based on their salience and so that nanoforms distributed across multiple metric elements possess the same or similar salience; and (iii) using one or more user interfaces to selectively, and optionally iteratively apply, one or more user constraints selected from a group including: relative note velocity; note swing; and note voices; thereby generating user customized percussion patterns embodied in a user generated molecule. Steps (ii) and (iii) of this aspect may include whereby analytic statistics based on a given corpus are used to guide the user's selection of microforms and associated nanoforms.

[0022] In accordance with another aspect of the invention, the formal properties of the persistent molecule repository indices may be used as a basis for creating a "remix" of a given arrangement. The initial positions (and associated family assignments) of the microforms in a given multi-track loop may be replaced with new positions within a different family, while maintaining the initial nanoforms along with their assignments to particular salient positions of a bar under a given microform. This may be regarded as a "transformation function" applied to one or more molecules such that certain relationships are maintained, while others are selectively altered, yielding a variation on the initial patterns.

[0023] In this respect, before explaining at least one embodiment of the invention in detail, it is to be understood that the invention is not limited in its application to the details of construction and to the arrangements of the components set forth in the following description or the examples provided therein, or illustrated in the drawings. The invention is capable of other embodiments and of being practiced and carried out in various ways. Also, it is to be understood that the phraseology and terminology employed herein are for the purpose of description and should not be regarded as limiting.

BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Embodiments will now be described, by way of example only, with reference to the attached figures, wherein:

[0025] FIGS. 1 a, 1 b, and 1 c illustrate different aspects of the system architecture of the present invention.

[0026] FIG. 2 is an illustrative example of metric salience.

[0027] FIG. 3 in an illustrative example of codon groups.

[0028] FIG. 4 is an illustrative example of codon hierarchy.

[0029] FIG. 5 is an illustrative example of scaling nodes.

[0030] FIG. 6 is an illustrative example of a 8Ttbt codon hierarchy.

[0031] FIG. 7 is an illustrative example of a 8[B]BbbBbb codon hierarchy.

[0032] FIG. 8 is an illustrative example of a salient ordering of 8[B]BbbBbb codon hierarchy.

[0033] FIG. 9 is an illustrative example of a salient score comparison of activity patterns against various codon hierarchies.

[0034] FIG. 10 is an illustrative example of a salient mask division of 8[B]BbbBbb.

[0035] FIG. 11 shows illustrative examples of time dividing codon hierarchies.

[0036] FIG. 12 is an illustrative example of carrier anatomy.

[0037] FIG. 13 is an illustrative example of modulators mapped to nanoform surface.

[0038] FIG. 14 is an illustrative example of modulators and modifiers mapped to a carrier.

[0039] FIG. 15 is an illustrative example of a complete music molecule.

[0040] FIG. 16 is an illustrative example of a molecule versus a performance element.

[0041] FIG. 17 is an illustrative example of analysis generated molecule providing a seed microform to the bar generator.

[0042] FIG. 18 is an illustrative example of a microform similarity matrix. [0043] FIG. 19 is an illustrative example of manual selection of microform from activity pattern selector.

[0044] FIG. 20 is an illustrative example of a depiction of the nanoform selection wheel in accordance with an embodiment of the system and method.

[0045] FIG. 21 is an illustrative example of nanoforms assigned to deployment bins, setting of functions, and deployment to microform nodes.

[0046] FIG. 22 is an illustrative example of surface handler controls.

[0047] FIG. 23 is an illustrative example of nanoform surface handling.

[0048] FIG. 24 is an illustrative example of microform surface handling.

[0049] FIG. 25 is an illustrative example of the linking of surface handlers.

[0050] FIG. 26 is an illustrative example of weaving together of carrier results from deployment bins and strata.

[0051] FIG. 27 is an illustrative example of inline strata navigation of a result list.

[0052] FIGS. 28A and 28B are an illustrative example of molecule editor and an edited note, respectively.

[0053] FIGS. 29a, 29b, 29c is an illustrative overview of repository technology.

[0054] FIG. 30 is an illustrative overview of a cloud technology implementation of the present invention.

[0055] FIG. 31 is an illustrative example of building a molecule.

[0056] FIG. 32 is an illustrative example of applying a transformation function to one or more molecules.

[0057] FIG. 33 illustrates a representative generic implementation of the invention.

DETAILED DESCRIPTION

[0058] In various aspects, the disclosure provides systems, methods, devices, and computer programming products for generating a representation of musical content data based on a repository of musical content data structures. The generated representation of musical content data may comprise a generated musical percussion pattern. The musical content representation generation may include receiving user-inputted selections of individual metric elements, selected from the repository of musical content data structures, that when arranged together with other metric elements, can be formed into a representation of a metrical structure of a bar of music. Each of the metric elements may represent one beat of music in the bar of music. The representation of the bar of music may be further modified by at least one musical content constraint in accordance with user input in order to change various properties or attributes of each of the metric elements or of the metrical structure as a whole. The user- customized representation of the metrical structure generated in this way may be used in a musical content sequencer or stored in the repository, or another repository of musical content data structures.

[0059] Various terms used throughout this specification have been defined in this specification in order to simplify the description of aspects of the invention. For example, at least the terms microform, nanoform, and molecule are defined for the context of this specification and should not be confused with any similar terms used in the fields of chemistry or biology. Instead, each of the terms microform, nanoform, and molecule is used to describe various elements or aspects of representations of musical content data stored by at least one computing device.

[0060] Therefore, in accordance with an aspect of the present invention, there is provided a system comprising at least one computing device providing or linking to a musical content data repository, the musical content data repository comprising: at least one musical content microform comprising a first musical content data structure comprising a representation of a metrical structure corresponding to a bar of music, the metrical structure comprising at least one metric element; at least one musical content nanoform comprising a second musical content data structure comprising a representation of a metric element; and at least one musical content molecule comprising a third musical content data structure comprising a selection of the at least one musical content microform and the at least one musical content nanoform, the at least one computing device comprising at least one user interface and is configured to: receive a first user input selecting at least one musical content microform of the musical content data repository; receive a second user input selecting at least one musical content nanoform of the musical content data repository; receive a third user input comprising at least one musical content constraint; arrange the selected at least one musical content nanoform within the at least one metric element of the at least one musical content microform based at least partly on a determined correspondence of nanoform salience, respectively associated with each musical content nanoform, to a salience range associated with the at least one musical content microform; and generate a user-customized musical content molecule based at least partly on the selected at least one musical content microform and the at least one musical content constraint, the user-customized musical content molecule describing a plurality of musical elements of a bar of music.

[0061 ] The method of creating a new molecule, or modifying an existing molecule may be performed by at least one computing device, such as a personal computer or mobile device, or by a computing device in communication with one or more other computing devices such as a personal computer, workstation, computer server, mobile device, or any other type of computing device. The at least one computing device either provides a musical content data repository, or links to a musical content data repository stored at another computing device.

[0062] In an aspect, the system and method of generating representations of musical content data (e.g. percussion patterns or beats) is based on a theory of a musical meter having a hierarchical representation of metric elements in a bar of music. In this theory, the musical meter is treated as a harmonic structure, analogous to the harmonic structure of sound waves, where metric elements exist at multiple "harmonic layers" and where the metric elements at each layer are grouped based on their "salience", which is a categorization of the perceptual importance of the elements to listeners, who experience them as either strong, medium or weak beats depending on their context. Part of the theory behind the musical meter is described in US Patent No. 7,723,602, the entirely of which is incorporated herein by reference. The parts of the theory that are germane to the current discussion are reviewed here briefly, and extensions, modifications and additions to the existing theory are described in further detail below.

Meaning of the word "note"

[0063] In this discussion, the word "note" refers to any auditory stimuli that are perceived as an atomic event by a listener. For example, in a sequence of MIDI events, there may exist two or more "note on" events that exist in such close temporal proximity that they are perceived as a single event. In this discussion, such a sequence of note activations is treated as a single note.

Metric elements and salience

[0064] FIG. 2 is an illustrative example of metric salience. In this discussion the term "metric element" is used to denote a fractional portion of time in a bar of music that is perceived as participating in the meter or rhythm of the bar by a listener. At the highest harmonic level of a bar's metrical structure, a metric element either contains no notes or a single note. At lower harmonic levels, a metric element may refer to portions of time that contain zero or more notes (i.e., many notes are possible). In the theory, all metric elements, regardless of their harmonic level, are categorized as either strong (S), medium (M) or weak (W), according to the presumed manner in which they are perceived by listeners. This is the metric element's "salience".

[0065] It is important to note that the term "metric element" is not necessarily used to refer to a portion of time that contains actual note data. Rather, the term is used in an abstract way to refer to divisions of time in a bar that can be grouped according to their perceptual strength were they to contain note data or when compared to a similar grouping that does contain note data. When it is desirable to specifically discuss a concrete scenario involving a metric element that contains actual note data, such a metric element is described as "active".

Binary and ternary codon groups and codon hierarchies

[0066] In the presently proposed theory, metric elements of a bar are grouped into either binary or ternary codon groups, where: (i) a binary codon group is a grouping of two adjacent metric elements such that the first element is considered to be "strong" (S), and the second element is considered to be "weak" (W); and (ii) a ternary codon group is a group of three adjacent metric elements such that the first element is considered to be "strong" (S), the second element is considered to be "medium" (M), and the third element is considered to be "weak" (W).

[0067] FIG. 3 shows an illustrative example of codon groups. Codon groups are considered to be hierarchical, such that after an initial grouping of metric elements is performed with each element under consideration participating in a either a binary or ternary grouping, the resulting groups themselves are grouped into a set of binary and ternary codon groups, and so on and so forth. This process stops as soon as the result of grouping items at one level produces a single group, which is either binary or ternary. The resulting structure is called a codon hierarchy. It is important to understand that each level of the grouping corresponds to a different harmonic layer of the bar's meter. Due to its hierarchical nature, a codon hierarchy can be viewed as a tree with leaves at the highest harmonic layer and with a root at the lowest harmonic layer. The leaves of the codon hierarchy tree consist of the original metric elements being grouped. An upper bound on the height of a tree representing a complete codon hierarchy is rounded to the next highest integer, where is the initial number of metric elements being grouped.

[0068] FIG. 4 is an illustrative example of a 3-level codon hierarchy consisting entirely of binary codon groups. It is important to note that while the concept of codon groups and hierarchies has been introduced, nothing has been said yet about how to group metric elements into codon groups and hierarchies. This is described further below. Scaling nodes in codon hierarchies

¾] In composition literature, a generative theory of tonal music (GTTM) presents a basic principle of metric hierarchy: if a metric element is strong at one (harmonic) level, then it creates a metric element in the (harmonic) level below it, which is called the containing layer. Thus, it can be said that metric activity that occurs at an S node "scales" its activity into a containing harmonic layer. Conversely, it can also be said that metric activity that occurs on an M or W node does not scale to the next harmonic layer of the metric hierarchy.

[0070] In the codon hierarchy shown in FIG. 5, the scaling nodes are circled and the metric elements they "cause" at the next (or containing) harmonic level is indicated.

Activity patterns

[0071] A codon hierarchy provides a complete, hierarchical grouping of a set of metric elements. In addition to this hierarchy, it is frequently necessary to discuss a particular instance of metric elements where some of the elements are active (i.e., contain note data) and some are not.

[0072] In an embodiment, an activity pattern is an array of 1s and 0s that has a one-to-one correspondence with the leaf nodes of a codon hierarchy such that an array element with value 1 indicates that the corresponding metric element contains note activity, and an array element with a value of 0 indicates that the corresponding metric element contains no note activity.

[0073] For example, the codon hierarchy in the previous FIG. 5 with 8 leaf nodes may be associated with an activity pattern of "01100101" to indicate a specific instance of metric elements, such that the second, third, sixth and eighth elements are active and contain note data.

Codon hierarchy naming conventions

[0074] It is convenient to provide a concise naming system for codon hierarchies since, later on, a method of determining an appropriate codon hierarchy for a given activity pattern will be provided and it will be convenient to have an easy way of referring to the "family" of all activity patterns that share the same codon hierarchy. The name that is used for this family is the name of the associated codon hierarchy.

[0075] The following is a simple, context-free language over alphabet {[0-9]+, b, t, B, T} for adequately naming the structure of codon hierarchies, where:

• b indicates a binary codon group of leaf-level metric elements in the hierarchy • t indicates a ternary codon grouping of leaf-level metric elements in the hierarchy

• B indicates a binary codon grouping of non-leaf-level metric elements in the hierarchy

• T indicates a ternary codon grouping of non-leaf-level metric elements in the hierarchy [0076] An illustrative BNF grammar for the language is:

<codon_hierarchy_name>

::= <num_metric_elements_at_highest_harmonic_level> <group> <group> ::= <binary_group> | <ternary_group>

<binary_group> ::= B<group><group> | b

<ternary_group> ::= T<group><group><group> 1 1

<num_metric_elements_at_highest_harmonic_level> ::= N

[0077] Intuitively, B and T can be considered binary and 3-ary operators, respectively, that can be used to indicate groups of other codon groups. The operands of B and T may be leaf- level codon groups (such as b or t) or may be more complicated groupings, such as Btb, which is a binary grouping of two leaf-level groups. This allows the language to easily describe very highly nested codon hierarchies.

[0078] In the remainder of this discussion, attention is generally restricted to codon hierarchies with a height of 2 or 3. As shown in FIG. 6, an example of a codon hierarchy name that follows the rules of the above language is: "8Ttbt".

[0079] This describes a codon hierarchy where the lowest harmonic level corresponds to a ternary (T) grouping of 3 metric elements that correspond to ternary (t), binary (b) and ternary (t) codon groups, respectively, at the next highest harmonic layer (the leaf layer, in this case), which has 8 individual metric elements. This codon hierarchy is visualized in FIG. 6.

[0080] Another illustrative example is the hierarchy 8BBbbBbb which is shown in FIG. 8. This describes a codon hierarchy where the lowest harmonic level is a binary (B) grouping of two metric elements that each correspond to a binary (B) codon group at the next highest harmonic layer, each of which in turn correspond to binary (b) codon groups at the highest harmonic layer (the leaf layer), which has 8 metric elements.

A method of determining a total ordering of metric elements in a codon hierarchy

[0081] In one embodiment, it is important to be able to determine a total ordering of the leaf- level metric elements of a codon hierarchy in order to determine the relative "salience" of those metric elements. US Patent No. 7,723,602 describes a method of determining such a total ordering. The method, modified slightly, and described using somewhat different terminology, is reviewed below.

[0082] 1. First, let Mbe the set of all metric elements of a codon hierarchy and | _et pe: → {0, 3,5, 6}

be a function that maps elements of the codon hierarchy to an integer that represents the metric element's "potential energy" at its harmonic layer. The function P " ^ is defined as follows: a. peim) ⁼ 0 , if m is a binary or ternary strong node b. pe( ^m )— 3 if mis a ternary medium node c. peim) ⁼ 5 jf mis a binary weak node d pe(m) = 6 jf _m \ _{s a} ternary weak node

[0083] 2. Generate a "name" string for each leaf element in the codon hierarchy, called the "harmonic state name" (hsn), by following the path from the leaf node back to its root node at the lowest harmonic layer and generating a string of the form P ^e ( ^m n)~i ^>e ( ^m l) where pe(m,), l < i < n , is the potential energy of metric element m at harmonic layer ι and where n is the harmonic level number for the leaf layer of the codon hierarchy.

[0084] 3. Perform a descending order lexicographic sort of the set of the resulting hsn strings for each leaf-level metric element. The resulting sorted list defines the order, in terms of salience, of each of the leaf-level metric elements in the codon hierarchy, with the first element being the least salient, the second element being the second least salient, etc.

[0085] FIG. 7 shows the hsn names for each metric element of the 8BBbbBbb codon hierarchy, and FIG. 8 shows as a salient ordering of 8[B]BbbBbb codon hierarchy.

[0086] The ordered set of "little endian" hsn names is {555, 550, 505, 500, 055, 050, 005, 000}, and the corresponding sorted list of "big endian" hsn names is {555, 055, 505, 005, 550, 050, 500, 000}. Thus the first leaf-level metric element (i.e., the one with hsn name "000") is the most salient, the fifth leaf-level metric element (i.e., the one with hsn name "500" is the second most salient), etc. Method for determining a salience score for an activity pattern with respect to a codon hierarchy

[0087] So far there has been a discussion of different possible ways of grouping metric elements to produce codon hierarchies, and there has been discussion of activity patterns which indicate the activity status of a set of metric elements. What has not been discussed is how to determine, given a set of metric elements and a corresponding activity pattern for them, what is the most desirable codon hierarchy for the activity pattern. That is discussed below.

[0088] Let ^nbe the set of all activity patterns corresponding to nmetric elements (note that 1 ^ 1 ~ 2 ), and let C« e some set of codon hierarchies that each have n metric elements in their leaf layer and are therefore potential codon hierarchies for any activity pattern in A _n _ section describes one possible method of determining how, given an to choose a codon hierarchy ^c ^ C„ t _na † "maximizes" the salience of that activity pattern.

[0089] The first step is to compute a salience score for an activity pattern ^a £ ^nwith respect to a particular codon hierarchy ^c ^ ^η , which is achieved as follows:

• First compute the total ordering of leaf-level metric elements in the codon hierarchy c under consideration (note that this is independent of any activity pattern). This produces an ordered set of metric elements — { ⁱⁿ \r ^m n} _t where ^m lis considered the least salient metric element of codon hierarchy c at the leaf layer and ^m «is considered the most salient metric element of codon hierarchy cat the leaf layer.

• Now define a salience weighting function ^c^c ^→ N that maps metric elements in Mc to a natural number that represents a salience weighting for the element. The weighting function that is used is:

Wcimi) = 2' ^" * 100

For example, the first metric element (the weakest) in the ordered set Mc is assigned the number 2 * 100 = 2 *100 = 200, and the last metric element in the set (the strongest) is assigned the number 2 ^s * — 25,600 • Using the weighting function ^w c defined above, a salience score function ^S c-A _n — > N that maps activity patterns ^α € A _{n 0} a natural number representing its salience with respect to the codon hierarchy c can be defined as:

n

s (a) - ∑ <*[«]* t*7 _t .(m _f )

i = l

[0090] Now that the salience score function with respect to a particular codon hierarchy has been defined, it can be used to determine which codon hierarchy in C« maximizes salience for a given activity pattern.

[0091] Specifically, the codon hierarchy ^c €&n that maximizes the salience of an activity pattern ⁰ £ -^ n is the one where the following holds:

s _c (a)>s _d (a for all d€. C _n , s.t d - -c .

2. If there is a tie in the scoring, the tie is broken based on the following ordering of

hierarchy families (in decreasing order): 8BBbbBbb, 8Tttb, 8Ttbt, 8Tbtt. So, if there is a tie in the score between 8BBbbBbb and 8Tbtt, 8BBbbBbb is chosen as the codon hierarchy.

[0092] FIG. 9 shows an illustrative example of a salient score comparison of activity patterns against various codon hierarchies.

Partitioning a set of activity patterns by codon hierarchy family and ordering members of the same family

[0093] Given a set of activity patterns of length nand a set C _n of codon hierarchies having n metric elements at their leaf layer, the salience score function defined in the previous section can be used to partition the set of activity patterns into groups, such that there is one group per codon hierarchy and the set of activity patterns in the group is the set of activity patterns that is considered more salient for that codon hierarchy than for any of the alternative codon hierarchies in ^n. Activity patterns that belong to the same partition are said to be in the same codon hierarchy family. Within a codon hierarchy family, the activity pattern strings can be ordered according to their salience score as determined by applying the salience score function for the associated codon hierarchy.

Salient filter masks

[0094] Within the ordered set of activity patterns in a codon hierarchy family, the activity patterns can be further classified into categories based on whether activity is present in their scaling nodes.

[0095] A salient filter mask is a string that indicates the activity status of leaf-level scaling nodes in an activity pattern. In the mask, the letter A is used to indicate an active scaling node and the letter N is used to indicate an inactive scaling node.

[0096] For example, for a codon hierarchy with 4 leaf-level scaling nodes, like 8BBbbBbb, AAAA is a salient filter mask that can be applied to the codon hierarchy family to refer to all activity patterns that have activity in all leaf-level scaling nodes. Similarly, ANAN is a salient filter mask that can be applied to refer to all activity patterns that have activity in the first and third scaling nodes only.

[0097] It should be noted that the salient scoring system is constructed in such a way that for two activity patterns ^{a a} j ^ ^ n^ jf 0» has more active scaling nodes than ^a J, then s (d ·) s (a ·)

c\ i ) c\ j / j _{ne rea} son for this is that, in the ordering of metric elements, metric elements corresponding to scaling nodes collate higher than all non-scaling nodes.

[0098] FIG. 10 shows an illustrative example of salient mask division of 8[B]BbbBbb. AAAA grouping. The table of activities within the salient mask is presented in wrapped descending order. Note that the last activity pattern in the grouping indicates the common active nodes for that salient mask grouping.

Time-dividing codon hierarchies

[0099] Up to this point, codon hierarchies only indicated how to hierarchically group metric elements by salience. There is another type of codon hierarchy that not only indicates how to group metric elements but also indicates how to generate a set of metric elements by dividing a given fraction of time. These types of codon hierarchies are called time-dividing codon hierarchies and they are named in the same way as non-time-dividing codon hierarchies. When a codon hierarchy is to be interpreted as a time-dividing codon hierarchy, this is mentioned explicitly. [00100] Some examples of time-dividing codon hierarchies that are frequently used in the remaining discussion are 2b, 3t, 4Bbb, 5Btb, 5Bbt, 6Btt and a special, degenerate codon hierarchy called a "thru", which divides the specified fraction of time into a single unit (itself). FIG. 1 1 shows illustrative examples of time dividing codon hierarchies. These time-dividing codon hierarchies are typically used in the theory to divide up an eighth-note-sized portion of time and then group the resulting metric elements by their salience.

[00101] As an example, a 5Btb time-dividing codon hierarchy indicates that some given unit of time (frequently an 8th note, in this discussion) is to be divided into two equal halves (that's what the "B" indicates), and the first half is then to be divided into 3 equal parts (indicated by "t") and the second half is to be divided into 2 equal parts (indicated by "b"). This produces 5 heterogeneously-sized metric elements, which are then grouped by salience in accordance with the codon hierarchy's name - so the first 3 metric elements form a ternary codon group and the last two metric elements form a binary codon group, and the resulting codon hierarchy is derived from this initial grouping.

[00102] The following Table A shows how the above time-dividing codon hierarchies can be used to divide an 8th note time division, which is how they are typically used in the remaining discussion:

Table A Microforms and Nanoforms

[00103] There are two structures that are fundamental to the remaining discussion— the microform and the nanoform.

Microform

[00104] A microform is a representation of the metrical structure of a bar that assumes that the bar is divided into eighth-note-sized metric elements, which are referred to as beats (note that divisions into sixteenth-note-sized or any other sized metric elements may also be possible).

[00105] The microform structure consists of: · the activity pattern for the bar, indicating which beats are active (i.e., have note activity).

• the codon hierarchy that is chosen by the system as representing the most salient codon hierarchy for the activity pattern based on the salience scoring system outlined above.

[00106] For this exemplary non-limiting discussion, we limit ourselves to microforms that describe musical bars that can be interpreted as being in so-called "4/4 time", and hence have 8 eighth-note-sized "beats". Due to this restriction, the only codon hierarchy families that are relevant to the discussion are:

8BBbbBbb

8Tttb

8Ttbt

8Tbtt

[00107] In the context of microforms, these codon hierarchy families are referred to as microform families. In the remaining discussion, microforms are often identified by combining their microform family name and activity pattern string together into a single string such as:

8BBbbBbb0100101 1 which indicates a microform with codon hierarchy 8BBbbBbb with metric elements 2, 5, 7 and 8 active. [00108] Finally, the metric elements at the leaf layer of the microform's codon hierarchy are referred to as the microform surface.

[00109] Accordingly, a microform is a representation of musical content, and may also be referred to as a musical content microform. The microform may comprise a musical content data structure comprising a representation of a metrical structure corresponding to a bar of music. The metrical structure may comprise at least one metric element or beat.

Nanoform

[001 10] Whereas a microform is a representation of the metrical structure of a bar that is assumed to be divided into eighth-note-sized beats, a nanoform is the representation of the metrical structure of one of those eighth-note-sized beats.

[001 11] A nanoform consists of:

• the activity pattern for the beat, indicating which of the metrical elements of the beat is active (i.e., has note activity).

• a time-dividing codon hierarchy that indicates: a. how time is subdivided within the beat into metric elements (i.e., it provides an indication of where, temporally, in the beat the active note regions are located). b. how the metric elements of the bar are grouped according to salience (either binary or ternary).

[00112] The only codon hierarchy families that will be considered for nanoforms are:

thru

2b

3t

4Bbb

5Btb

5Bbt

6Btt

6Tbbb

[001 13] In the context of nanoforms, these codon hierarchy families are referred to as nanoform families. See the section "Time-dividing codon hierarchies" above for visualizations for some of these families. [00114] As with microforms, nanoforms are often represented as a single string indicating their nanoform family name followed by a corresponding activity pattern. For example:

5Btb101 11 is a nanoform with time-dividing codon hierarchy 5Btb and metric elements 1 , 3, 4 and 5 active.

[00115] Finally, the metric elements at the leaf layer of the nanoform's codon hierarchy are referred to as the nanoform surface.

[001 16] Accordingly, a nanoform is a representation of musical content, and may also be referred to as a musical content nanoform. The nanoform may comprise a musical content data structure comprising a representation of a metric element, or beat. A microform may comprise multiple nanoforms in order to represent a bar of music.

Carrier

[00 17] A carrier is a complete description of the metrical structure of a bar of music consisting of:

1. a microform for the bar 2. a list of nanoforms, one for each active metric element in the microform.

[00118] FIG. 12 shows an illustrative example of a carrier consisting of microform 8Tbtt01010010 (i.e., 3 active beat positions) which has nanoform 5Btb00010 in the first active beat position, nanoform 5Btb00001 in the second active beat position, and nanoform 5Btb10000 in the third active beat position.

Modulators and modifiers

[00119] A carrier has modulators and modifiers that control the different aural aspects of the carrier.

Modulators

[00120] Modulators assign one or more voices to notes and can define rules regarding how voices are to be managed when there is more than a single voice for the carrier.

[00121] For example, to represent a high-hat, two voices could be assigned to a carrier -- one for the open high-hat sound and one for the closed high-hat sound. The modulator could define rules for this composite voice scenario, such as indicating that whenever a closed high- hat note activation is encountered, any currently active open high-hat note is cut off, or perhaps indicating how the two voices can be played simultaneously to create polyphony.

[00122] FIG. 13 shows an illustrative example of modulators mapped to a nanoform surface. FIG. 13 indicates various modulator mappings across binary and ternary nanoform surfaces. The figure also indicates an example polyphonic mapping.

Modifiers

[00123] Modifiers modify some aural property of a note.

[00124] In the present system and method, there are three types of modifiers that may be used:

1. pico offsets - these are modifiers that indicate a note's offset from the beginning of the metric element with which it is associated. The range of pico offsets is [-100, 100] and one pico offset unit represents 1/96 fraction of the associated bar duration.

2. velocities - these are modifiers that correspond, roughly, to the volume of a note.

3. duration - a modifier that indicates the duration of a note.

[00125] Other modifiers that correspond to general MIDI controller attributes are possible as well.

[00126] In this discussion it will be said that modulators and modifiers are "applied" or mapped to a carrier. FIG. 14 shows an illustrative example of modulators and modifiers mapped to a carrier. In a similar scheme to the modulator mapping, velocity and pico offset ranges are applied to binary and ternary nanform surfaces.

Molecule

[00127] The final structural concept to introduce, and one that ties all previous concepts together, is the molecule, which provides a complete description of a bar of music and consists of the following:

1. the carrier for the bar (microform + nanoforms)

2. modulators, to assign and define the behavior of one or more voices for the carrier 3. modifiers, such as pico offset and velocity that control the playback of active metric elements at the nanoform level (i.e., notes) such as their relative volume and whether they anticipate or lag behind the metric element with which they are associated.

[00128] FIG. 15 shows an illustrative example of a complete music molecule.

[00129] For the purposes of the present discussion, the concept of "molecule" replaces the concept of "performance element" in US Patent No. 7,723,602. The following is a list of differences between molecules and performance elements:

In the molecule, the "compositional layer" of the performance element has been separated into an individual carrier and modulator of the molecule. In the molecule, the "mechanical layer" of the performance element has been separated into individual modifiers, consisting of pico offset, velocity, duration, and any other general MIDI controller attributes.

[00130] FIG. 16 shows an illustrative example of a molecule versus a performance element.

[00131] Accordingly, a molecule is a representation of musical content, and may also be referred to as a musical content molecule. The molecule may comprise a musical content data structure comprising a selection of musical content microforms and musical content nanoforms.

A method for generating a plurality of beats

[00132] In an aspect, an object of the present system and method for producing customized percussion patterns is to produce a set of molecules (i.e., a carrier plus modulators plus modifiers for a single bar of music) taking into account three constraints imposed by the user. The user can then audition molecules from this result set and ultimately select one to use in their song.

[00 33] The three different types of constraints imposed by the user are described below:

(1 ) Microform Constraint

[00134] In order for beat generation to take place, a microform must be selected for the bar being generated. This microform acts as a constraint on beat generation by doing two things:

1. it fixes the beats within the bar that are considered active

2. it determines the salience of each beat in the bar (either S, M, or W) from the codon hierarchy associated with the microform. [00135] FIG. 17 shows an illustrative example of analysis generated molecule providing a seed microform to the bar generator. In this Fig. the properties of the source molecule are utilized to provide a starting point for the generator. The Fig. illustrates the microform activity from the source molecule selecting the corresponding activity on the microform similarity matrix. In the present system and method, microform selection can take place in the following ways:

In the first beat generation workflow, the user loads an existing MIDI loop from a loop library into the beat generator which analyzes it and generates a set of molecules representing the bars for the loop. Note that a full method for generating the contents of a molecule is described in US Patent No. 7,723,602. A single molecule is then selected for generation by the user and the molecule already has an associated microform which was deduced from analyzing the activity pattern for the bar and selecting a codon hierarchy based on the salience scoring system outlined above.

In the second beat generation workflow, the user starts from scratch, selects a microform from a microform selection "wheel" (which may be called a microform similarity matrix), which is a circle that groups microforms according to their codon hierarchy family. The wheel is split into four quadrants, one quadrant each for the four possible microform families: 8BBbbBbb, 8Tttb, 8Tbtt, 8Ttbt, and contains icons that represent each of the 255 possible microforms (the case of all inactive metric elements is excluded).

The wheel is arranged so that the microform families 8BBbbBbb and 8Ttbt are opposite each other to reflect the opposite salient ordering structure of their internal nodes, and microform families 8Tbtt and 8Tttb are also positioned opposite each other. Within each quadrant, the activity patterns for a family are further subdivided by salient filter mask, thus grouping microforms with the same pattern of scaling node activity together, and within those subgroups, microforms are ordered according to their salience score. In general as one moves from the outside of the wheel towards the center, the microforms have decreasing salience (i.e., lower salience scores) and decreasing activity (i.e., fewer active beats). A "slice" of a quadrant contains microforms that are considered to be the most "saliently similar".

[00136] FIG. 18 shows an illustrative example of a microform similarity matrix.

In the present system and method, the user can interact with the wheel in several ways: a. The user can click on an entire quadrant (in the "wheel" interface shown in FIG. 18, in one particular implementation of the present invention) and get a list of all the microforms in that family with their codon hierarchies fully visualized, and then select a specific microform from that list. b. The user can click on a single icon from FIG. 19 (highlighted portion of "wheel" interface shown, in one particular implementation of the present invention) in the wheel and this provides a short list of all the most saliently similar microforms, including the selected one. c. The user can click one or more "slices" (as shown on the bolded lines in FIG. 19) of a quadrant and thereby see all the microforms that are considered saliently similar

(shown in FIG. 19). d. The user can click on the center of the wheel which provides a list of all 255 possible microforms.

[00137] In the third beat generation workflow, the user also starts the beat generation process "from scratch" but interactively indicates the activity pattern for the beats in the bar. From this, a codon hierarchy is automatically chosen based on the salience scoring system outlined above. The activity pattern plus the microform family combine to be the microform for subsequent beat generation and the corresponding microform on the wheel is highlighted automatically.

[00138] FIG. 19 shows an illustrative example of manual selection of microform from activity pattern selector. It is important to note that selection of a microform can be done anytime during the beat generation process. If, after generating a set of beats, the user decides to change the underlying microform, he or she can simply choose a new one and leave all their other choices the same and a new set of beats satisfying the new constraints is automatically generated. (2) Nanoform vocabulary constraint

[00139] In order to generate a set of beats, it is necessary for the user to select a set of nanoforms that are considered the nanoform "vocabulary" for beat generation, i.e., beats that are generated can only contain these nanoforms. These nanoforms are selected from a wheel (officially called a nanoform similarity matrix), similar to the microform wheel described above, which is organized by the 7 different nanoform families that are supported: thru, 2b, 3t, 4Bbb, 5Btb, 5Bbt and 6Btt. [00140] Like the microform selection wheel, the nanoforms in a particular family are organized into groups (4 groups for the 2b, 5Btb, 5Bbt and 6Btt nanoform families) that reflect the different activity status of the scaling nodes for the nanoform. Within each sub-group, nanoforms are ordered from most to least salient, according to their salience score. Like the microform wheel, the nanoform wheel is generally organized so that nanoforms with the highest salience scores appear near the outside of the wheel and the salience of nanoforms decreases as the center of the wheel is approached.

[00141 ] FIG. 20 shows an illustrative example of a depiction of the nanoform selection wheel in accordance with an embodiment of the present system and method. In FIG. 21 , the nanoform visualization includes the activity pattern for the nanoforms represented as note "balls" on top of the nanoform surface. Furthermore, for simplicity, the arrows that have been used in this document so far to connect different levels of a codon hierarchy have been removed.

Nanoform deployment bins

[00142] When a nanoform is selected from the wheel (as shown for example in FIG. 21 ), it is placed in one or "all" of the following three "bins", depending on the value of a radio button:

1. Strong bin: a bin that collects nanoforms that can only be placed in strong beats (either binary strong or ternary strong) of the selected microform. Selected nanoforms (one or more) are deployed to this bin only if the radio button is set to "strong" or "all". 2. Medium bin: A bin that collects nanoforms that can only be placed in ternary medium beats of the selected microform. Selected nanoforms (one or more) are deployed to this bin only if the radio button is set to "medium" or "all".

3. Weak bin: A bin that collects nanoforms that can only be placed in weak beats (either binary or ternary weak beats) of the selected microform. Selected nanoforms (one or more) are deployed to this bin only if the radio button is set to "weak" or "all".

Nanoform bin deployment functions

[00143] Each nanoform deployment bin has an associated "deployment" function that controls how nanoforms in the bin are "deployed" across the set of beats in generated molecules that have a matching salience value (S, M or W).

[00144] The three supported nanoform deployment functions are: • repeating - this is a deployment function that indicates that when a nanoform is deployed to beats in a bar, all the beats in the bar that have a salience matching the salience value of the bin must contain the same nanoform, and all such possible patterns are generated.

For example, if the "strong" bin contains 3 nanoforms (call them x, y and z) and the deployment function for the strong bin is set to "repeating", then every molecule that is generated by the system will fall into one of the three categories: o nanoform x is in all S positions o nanoform y is in all S positions o nanoform z is in all S positions and each category will be non-empty.

• contiguous - this is a deployment function that indicates that when a nanoform is

deployed to beats in a bar, the beats in the bar with a salience value matching the bin must contain the nanoforms in the bin in the sequence they appear in the bin. If there are fewer items in the bin than matching beats in the bar with the same salience, then any extra beat positions will use the last nanoform in the bin repeatedly. On the other hand, if there are more nanoforms in the deployment bin than corresponding beat positions with matching salience in the bar, then all molecules with all possible contiguous sequences will be generated.

For example, if the strong bin contains 3 nanoforms (call them x, y and z) and the deployment function for the strong bin is set to "contiguous", and there are only 2 strong beat positions in the selected microform, then all the molecules generated by the system will fall into one of two categories: o nanoform x is in the first strong position and nanoform y is in the second strong position o nanoform y is in the first strong position and nanoform z is in the second strong position and each category will be non-empty. • mash - this is a deployment function that indicates that when a nanoform is deployed to beats in a bar, the beats in the bar with a salience value matching the bin must contain some permutation of nanoforms in the bin and all possible permutations are represented in the final set of generated molecules.

For example, if the weak bin contains 3 nanoforms (call them x, y and z) and the deployment function for the weak bin is set to "mash", and there are only 2 weak beat positions in the selected microform, then all the molecules generated by the system will fall into one of three categories: o nanoform x is in the first weak position and nanoform y is in the second weak position (and vice versa) o nanoform y is in the first weak position and nanoform z is in the second strong position (and vice versa) o nanoform x is in the first weak position and nanoform z is in the second weak position (and vice versa) and each category will be non-empty.

[00145] FIG. 21 shows an illustrative example of nanoforms assigned to deployment bins, setting of functions, and deployment to microform nodes.

[00146] Like microform selection, nanoform deployment bin contents can be changed at any time and whenever a nanoform is added or removed from a bin, the set of generated molecules is recomputed.

[00147] Another point to make regards the contents of the three deployment bins at the beginning of bar generation. There are two cases:

1. A molecule is being generated from scratch, in which case by default each deployment bin contains a single nanoform, a "thru".

2. A molecule that is part of a loop loaded from a loop library is being generated. In this case, all the nanoforms in the molecule are placed in the appropriate deployment bin (S, M or W) in the order they appear in the molecule. The deployment function for each bin is then set to "contiguous."

in Modulator and modifier constraints (aka "surface handlers") [00148] The final constraint on molecule generation is a constraint on the setting of carrier modulators and modifiers, i.e., voices, pico offset and velocity.

[00149] To set the modulator/modifier constraints, the system and method provides three "surface handler" controls:

• A surface handler for controlling elements of a binary codon hierarchy with two surface elements (S and W) and a strong (S) governing node. This handler is used to control matching surface elements at either the microform or nanoform level.

• A surface handler for controlling elements of a binary codon hierarchy with two surface elements (S and W) and a weak (W) governing node. This handler is used to control matching surface elements at either the microform or nanoform level.

• a surface handler for a ternary codon grouping with S, M and W surface nodes. This handler is used to control any type of ternary codon grouping, possessing any type of governing node, at the microform or nanoform level.

[00150] FIG. 22 shows an illustrative example of surface handler controls, in one particular implementation of the present invention.

[00151] The surface handlers allow the specification of the following modulator/modifier properties:

1. Voices (modulator): voices can be assigned to the metric elements based on their salience (e.g., all strong nanoform surface nodes can be assigned a particular voice, or all notes contained within a weak microform metric element can be assigned a particular voice). Each voice is represented by a different colored note widget.

2. Pico offset (modifier): a range of pico offset values can be specified for microform and nanoform surface elements. A range can be specified for each voice and the range consists of note widgets colored to correspond to the matching voice. If a pico offset boundary is 0, the note widget is a circle centered on the beginning of the associated metric element. To indicate a pico offset range with a lower bound below 0, a left- pointing triangle is used. To indicate a pico offset range with a above 0, a right-pointing triangle is used. The system randomly generates values in the specified the range during generation. 3. Velocities (modifier): a range of velocity values can be specified for microform and nanoform surface elements. The system randomly generates values in the specified range during generation. Velocity ranges are specified using a rectangle whose size can be modified. The larger the area of the rectangle, the wider the velocity range. A rectangle per voice is supported.

[00152] There are 2 different modes that control how surface handlers are applied to a carrier:

1 Nanoform surface mode

[00153] In this mode, the modulators and modifiers apply to matching surface elements at the nanoform level. So, for example, any nanoform surface elements that involve a binary codon group with a governing strong binary node, would be controlled by this surface handler. FIG. 23 shows an illustrative example of an interface for utilizing the surface handler controls. In this figure the surface handling strategy is applied to nanoform codons.

[00154] Specifically, the notes in the S surface element would be controlled by the settings on the S surface node in the surface handler widget, and notes in the W surface element would be controlled by the settings on the W surface node in the surface handler widget.

[00155] In this mode, the so-called "thru" nanoforms are controlled by matching elements at the microform level. So, for example, a "thru" nanoform in a "strong" binary beat at the microform surface that has a corresponding "strong" governing node would be controlled by the settings on the S surface node of the first surface handler.

2. Microform surface mode:

[00156] In this mode, the modulators and modifiers apply to matching surface elements at the microform level. So, for example, any microform surface elements that involve a binary codon group with a governing non-strong binary node, would be controlled by this surface handler.

[00157] Specifically, the notes in the S surface element would be controlled by the settings on the S surface node in the surface handler widget, and notes in the W surface element would be controlled by the settings on the W surface node in the surface handler widget. FIG. 24 shows an illustrative example of microform surface handling interface. FIG. 24 shows surface handlers being applied to the microform surface.

Linking surface handlers [00158] To simplify surface handler settings for users, surface handlers can be linked together so that instead having two separate handlers for binary groupings (one governed by strong nodes and one not governed by strong nodes) settings can be set on one surface handler and they are considered to apply to S/W nodes regardless of governing node salience. Similarly, the first two handlers can be linked with the third handler (the ternary handler) which indicates that the ternary S/W nodes are to take their settings from the other handlers as well, leaving only the M node that needs to be set. Here is a picture of the handlers when they are linked: FIG. 25 shows an illustrative example of the linking of surface handlers.

How the set of generated molecules is computed from the constraints

[00159] Whenever a microform is selected or a nanoform is deployed to one of the deployment bins, the set of generated molecules is recomputed and displayed in a list.

[00160] The method for generating the result set of molecules is as follows:

1. First a set of "partial" carriers for each deployment bin is generated, where the partial carriers only have nanoforms assigned to the beat positions that have a salience that corresponds to the deployment bin's salience (S, M or W). This set of partial carriers is generated according to the deployment bin's deployment function (repeating, contiguous or mash) as described in the previous section. The sets will be named as follows:

a Carrier S - partial carriers for strong microform beat positions (includes binary and ternary strong beats) b Carrier M- partial carriers for medium microform beat positions c. Carrier w_ p _ar tj _a | carriers for weak microform beat positions (includes binary and ternary weak beats) Note that these partial carrier sets can be interpreted as the different "dimensions" or strata of the final result set. Essentially, they form a "basis" for the space of result molecules.

2. The next step is to "weave" together the 3 strata of partial carriers from step 1 into a set of complete carriers, which is called, simply, f ^nM *. This is done by computing the Cartesian product of the strata sets as follows:

Carriers _iliai = Carrier _s x Carrier ^ x Carrier

3. After the complete set of carriers is generated in step 2, the surface handlers are applied to set the modulators and modifiers for the carrier. In particular, the surface handlers specify a range of velocity and pico offsets to be applied, as described previously, at the nano surface level, micro surface level, or nano surface level and micro surface level for "thru" nanos. This is the final generated set.

[00161 ] This method may be implemented using the interface and steps illustrated in FIG. 26, in one implementation of the present invention.

Browsing the result list

[00162] The present system and method avoids computing the complete set of result molecules at one time. Instead, it only computes the 3 partial carrier sets and combines them together "on the fly" as a user scrolls through the list, which is only displayed in small, fix-sized chunks. This is a performance enhancement that is beneficial when there are many nanoforms deployed to the deployment bins which can result in a combinatorial explosion of result molecules which make materializing the entire list at once prohibitively expensive.

[00163] The method for the on-the-fly computation of rows in the final set of carriers is notable and is described below.

1. First, assume that \ Carrier _s \ = i _{(i e} Carrier _{s nas} j elements),

Carrier f \ = j and \ Carrier _w \ = k This means that Carriers _{fi al} \ = = i * j * k

Assume further that the elements in each set are numbered starting from 0.

2. Now, if row ^0 < n<i* j * k _s t ₀ be displayed, the carrier associated with this row can be computed as follows (note that the division operator here "/" is used as an integer division operator, i.e., the answer is rounded down to the next highest integer): a. First determine which element of Carrier g participates in the final carrier. The element is determined by the result of the operation n / U * *). i.e.. if the result value is rthen the element rof Carrier _s j _{S tne des} j _rec : element. b. Then determine which element of Carrier ^ participates in the final carrier.

The element is just the remainder from the division in the previous step divided by k. c. Finally, determine which element of Carrier participates in the final carrier.

The element is determined by the remainder from the division calculation in the previous step. d. Once all the individual elements is determined, they can be combined into a single-element in an obvious way: all the nanoforms in the partial carrier chosen from Carrier w _are placed in the W beat positions of the final carrier, all the nanoforms in the partial carrier chosen from C rrier _{Iare p} | _a ced in the M beat positions of the final carrier, and finally all the nanoforms in the partial carrier chosen from C rrier _Sare placed in the S beat positions of the final carrier.

[00164] So for example, if i=3, j=3, k=3 and the 23rd element of the final carrier set are needed, the elements in the three strata sets can be located as follows:

1. 23 / (3 * 3) = 2, r=5. So element 2 of Carrier _{s js used (note tnat 2 is the 3rd} element of the set because counting starts from 0).

2. 5 / 3 = 1 , r = 2. So element 1 of Carrier _{vjs used}

3. Remainder from previous step is 2. So element 2 of Carrier j _{s used}

An alternative method of browsing generated results

[00165] In this section, an alternative method of browsing results is briefly described.

[00166] FIG. 27 shows an illustrative example of inline strata navigation of a result list.

[00167] As mentioned previously, the beat generator internally calculates three sets of "partial" carriers, which constitute the different strata of the beat generation result set. Instead of combining these sets into a final list to be browsed by the user, a method of allowing the user to browse the result space is proposed which involves:

1. Exposing the three strata sets to the user, which are recomputed every time either the microform or nanoform constraints are modified. 2. Allowing the user to select an element from each strata set. The system combines those three elements into a single carrier to show a result carrier, which can be auditioned. a. An alternative is to allow the user to select an element from only one or two of the strata sets and allow the remaining strata sets to be unconstrained. This acts as a kind of "filter" on the result set that ensures that certain patterns appear in certain beat positions while leaving the other to take on a range of possible values.

[00168] There are two benefits to this alternative approach to browsing the result space (as opposed to the list approach):

1. When a large number of nanoforms are deployed to the nanoform deployment bins, it is possible that the number of possible results is so large that navigating the list is expensive (even with the on-the-fly approach) and it may take an unreasonably long time for a user to locate the carrier they desire since its position in the list may not be obvious. With this approach, the user assembles the pieces they want and only one complete carrier is every built at a time. 2. This approach helps to emphasize the approach to generation taken in this proposal, which is to deploy nanoforms to beats based on beat salience. By forcing the user to deal with elements in the individual strata, this approach is reinforced.

A molecule editor for fine-grained control of molecule content

[00169] To add flexibility to the beat generation process, a molecule editor is provided to allow the user to make manual modifications to generated molecules at any time during the generation process.

[00170] The molecule editor offers standard note-editing functionality. For example, add note, delete note and move note functionality is supported, as well as the ability to modify the velocity and pico-offset of notes. The editor also supports the manipulation of notes at many different quantization scales, from half-notes up to 48th notes, and also allows users to turn of quantization and place notes anywhere desired (notes will be fit into one of the supported nanoforms, with appropriate pico offsets, if necessary, to maintain the original location of the note).

[00171 ] What is unique about the molecule editor is that it allows a user to immediately see the effect of note changes on a molecule's codon hierarchy. Whenever a change is applied (i.e., notes are added, deleted or moved), the note data in the bar is re-analyzed and a molecule is re-generated on-the-fly to show the updated nanoforms and microform. The feedback helps the user understand the effect of their changes on the fundamental structural and salience relationships in the bar, which in turn helps guide their subsequent generation and editing choices.

[00172] For example, the following image shows the molecule editor being used to edit a generated molecule. The subsequent image demonstrates how the first molecule changes when a note is added in the molecule editor:

[00173] FIG. 28a shows an illustrative example of a molecule editor and FIG. 28b a molecule responding to editing. It should be noted how, after the indicated note is added, the microform that underlies the bar changes from codon hierarchy family 8BBbbBbb to codon hierarchy family 8Ttbt. Notice further how this changes the salience of the metric element corresponding to beat six of the bar (i.e., the 6th eighth-note time division in the bar). Prior to the addition of the note to beat 4, beat 6 is considered a binary weak beat, whereas after the change, beat 6 is considered a ternary strong beat, indicating a possible change to the way a listener perceives the meter of this bar. With this feedback, the user may decide to, for example, use the molecule editor to increase the velocity of the note in beat 6, in order to emphasize the beginning of the ternary note grouping.

A molecule repository

[00174] US Patent No. 7,723,602 describes a method of analyzing a MIDI file for a song to produce a set of molecules. In the previous patent, the molecules are called "performance elements" and differ from molecules as outlined in the earlier section on molecules. The method in that patent analyzes a file and produces the following analysis artifacts:

1 . the number of unique molecule configurations utilized in the midi instrument track 2. the occurrence count of molecule configurations as they are distributed across the MIDI instrument track 3. the linear bar location of the molecule configurations to the equivalent MIDI data in the MIDI instrument track

[00175] FIG. 29a shows an illustrative overview of repository technology. In the present system and method, this idea is generalized such that the analysis is performed on any number of music files and the results are stored in a molecule repository, which is a relational database with a persistent store of song data, including:

A description of the songs analyzed (e.g., song id (generated by the system), song name, genre, artist, length of song, list of instrument tracks).

A description of each microform and nanoform encountered during the analysis of the songs.

A description of the unique molecules encountered during analysis, including the associated microforms and nanoforms of each molecule as well as the location of each molecule in each song as identified by a <song id, instrument id, bar number> tuple.

[00176] In the present system and method, it is proposed that the molecules be entered into the molecule repository during song analysis and that the reports that can be generated through analysis of the repository be extended to include the following, which can be requested by a user through a query interface:

1. A report of unique microforms: A unique list of microforms for a given unit of analysis (e.g., a song or set of songs, a genre, an instrument type, etc.). The results can be reported by highlighting the corresponding microforms on a "microform similarity matrix", which looks like the microform selection wheel described in the beat generation section, as shown in FIG. 29b in one particular implementation of the present invention.

2. A report of unique nanoforms: A unique list of nanoforms for a given unit of analysis. The results can also be reported by highlighting the corresponding nanoforms on a "nanoform similarity matrix", which looks like the nanoform selection wheel described in the generation section, as shown in FIG. 29c in one particular implementation of the present invention.

3. A report of the co-occurrence of nanoforms with microforms: For a given unit of analysis a list of nanoforms that co-occur with a particular microform is generated. The list of nanoforms is ordered by salience, so that all the nanoforms that are associated with S beats of the corresponding microform appear together, all the nanoforms associated with M microform beats appear together, and finally all nanoforms associated with W microform beats appear together.

4. A microform "heat map": For a given unit of analysis, a count of the number of instances of a unique microform is tracked. The result of this analysis is reported on a microform similarity matrix by highlighting each microform that occurs at least once using a color scheme that maps the frequency value to a color in the color continuum between blue and red, where blue represents the lowest frequencies and red represents the highest frequencies. Alternatively, frequency of use may be indicated by different shades within a given colour associated with a particular instrument type.

5. A nanoform "heat map": For a given unit of analysis, a count of the number of instances of a unique nanoform is tracked. The result of this analysis is reported on a nanoform similarity matrix by highlighting each nanoform that occurs at least once using a color scheme that maps the frequency value to a color in the color continuum between blue and red, where blue represents the lowest frequencies and red represents the highest frequencies.

6. A song framework matrix: This is a matrix that has one row for each instrument track contained in a song, and one column for every bar in the song. An element at of the matrix at position (row, col) is the molecule that represents the music at bar col for the instrument associated with row row. This structure allows one to easily determine the molecules that co-occur in the same bar of music and to calculate the degree to which specific molecules tend to co-occur with other molecules in the same bar of music.

7. Molecule co-occurrence reports: Based on a given unit of analysis, various reports of the co-occurrence of molecules is possible using the data in the molecule repository, including, but not limited to, the following: a. report of molecules that co-occur within a particular instrument track. b. report of molecules that co-occur at the same bar in different instrument tracks.

[00177] Accordingly, the repository, which may also be referred to as a musical content data repository, may comprise at least one musical content microform, at least one musical content nanoform, and at least one musical content molecule. The repository may be updated with new molecules, or provide for modification of existing molecules stored in the repository. Any microforms, nanoforms, or molecules may be selectively retrieved from the repository in accordance with user input selections received at the computing device.

[00178] The repository may act as an index of possible musical content patterns, and as a backdrop for analysis of a given corpus. Different types of queries and reports can be generated through analysis of the repository. These reports may become a basis for highlighting suggested molecule components as guides for user construction and generation in a creation of production context, as shown in Figure 31.

[00179] The formal properties of the persistent molecule repository indices may be used as a basis for creating a "remix" of a given arrangement. The present embodiment may take the initial positions (and associated family assignments) of the microforms in a given multi-track loop and replace these assignments with new positions within a different family, while maintaining the initial nanoforms along with their assignments to particular salient positions of a bar under a given microform. This may be regarded as a "transformation function" applied to one or more molecules such that certain relationships are maintained, while others are selectively altered, yielding a variation on the initial patterns, as shown in Figure 32.

Support for a molecule cloud

[00180] FIG. 30 shows an illustrative overview of a cloud networking implementation of the present invention. In an aspect, the present system and method supports a molecule cloud, which is a centralized server to which users can subscribe. Professional users can offer subscriptions for premium content (i.e., molecules of the song). Regular users of platform of the present invention which is marketed as LIQUID LOOPS™ (may also be referred to as "LL" in this disclosure) can just use loops in the product, and other users preview content from other users providing customized sound design and performances.

Applications and Use Cases

Generate drum patterns

[00181] A first application is to use the present system and method to generate a plurality of drum patterns. In one scenario, an electronic musician generates nuanced percussion loops for export into a digital audio workstation or "DAW".

[00182] In an illustrative "from scratch" method, a user first creates single bars (molecules) through generation process (see "from scratch" method outline above). The user then arranges selected molecules into a multi-bar loop (length determined by user). Next, the user combines additional molecules to collaborate (in 'stack' fashion across different instruments/tracks) in the loop on the arrange page. The user may refine or adjust the resulting loop in a number of different ways: The user may target any bar in the arrange matrix for "in-place" generation (the components of the existing bar become material seeds for the generator; variations can be introduced at any of the 3 generation layers).

[00183] Furthermore, the user may edit any molecule on the arrange matrix using fine control and manual specification in the Molecule Editor. The user may 'drag and drop' molecules to change their order/position in the matrix. The user may selectively access fine variations in performance parameters (pico offset and velocity settings) for a given molecule in the arrangement through calling up in-place 'modifier skins' (variations on the 3rd layer of the generation process that may be set to a range and applied selectively to beats of differing strength).

[00184] Once a user is satisfied with a loop matrix, they can then send the resulting MIDI data to a DAW of their choice through a MIDI application sharing program such as ReWire (Propellorheads), or OpenSoundControl (the user may view the information in a standard MIDI- roll representation prior to transfer).

[00185] In a second scenario, a songwriter builds a guide drum track to provide musical direction for a live drummer. In essence, much would be the same as above, but in this second scenario, the user would limit themselves to conventional kit instruments, and take care to specify performances that are both "mechanically possible" (in the sense of the physical instrument; i.e. no open and closed hi-hats at the same time, etc; see section on "compound voices" above) and "humanly-performable" (not bursting with three-handed drum-acrobatics, etc.), as well as being more deliberately constructed as a full drum performance (rater than as a compelling matrix of singular bars).

[00186] Again, as above, the user may then choose to export to a DAW, from which the drummer may use the constructed loop as a guide for his/her performance of the drum parts, either for live performance situations or recording situations.

[00187] Alternatively, a drummer may 'learn his/her part' through playback in the Liquid Loops software itself, in which case they may be aided in grasping the nuances of the drum part by reflection in the molecule representation (referred to below, or waveDNA™ molecular representation) (which, it may be argued, is vastly superior to other drum scoring methods, such as standard notation). This representation is particularly useful for revealing patterns of fine timing adjustments useful as performance instructions.

Capture live performance patterns

[00188] As another use case, the present system and method may be used to capture performance patterns of a live drummer. In a first scenario, a database of metric elements may be built up to quantify the "style" of a drummer over a plurality of performances. Musicians may input MIDI-captured databases of performances in order to examine and quantify the characteristic content of the material (i.e. they may use the above described 'similarity matrix' to obtain objective metrics about the material distribution of their corpus, including fine 'groove- type' timing adjustments). These characterization metrics may also be used for comparison purposes with other databases (may be realized in shared-content services).

[00189] One possible scenario for this would be for the purposes of learning from an admired source by comparing one's own performances to those of the admired. Another possible scenario would involve a user investigating the content characteristics of genres/styles of their own devising (i.e. investigating the content that comprises his/her favorite new invented genre- hybrid of 'post-rock garage dub' or what have you). Characterization metrics may be applied to particular performers or particular performances in order to capture unique musical behavior signatures (across the 3 levels of the representation). Characterization metrics may also be applied to externally categorized databases (i.e. particular genres/styles, etc., defined by sources other than the user).

[00190] In another scenario, a user seeds the molecule generator from a performed molecule to create a plurality of selectively controllable variations of the original molecule for assembly into alternative loop and stack arrangements. Users may select a MIDI percussion bar (or stack, or loop) from their own collection and import the file to the generator. This content may then be selectively used as initial-states for generator settings. Additional resulting variations can be created by manipulating these original settings in various ways: a) micro-level variation: by changing the positions of the existing nanos across the molecule (i.e. changing the activation pattern and resulting microform while holding nanos and surface information constant) b) nano-level variation: by adding to the material elements (nanos) assigned for deployment to selected strata-bins (through selections on the similarity matrix), or by shuffling the order of the existing nanos across the molecule, or by editing particular nanos on the molecule. c) surface-level (or groove) variation: by selecting alternate ranges on the surface- handler settings of the generator (i.e. the pico-timing and velocity sliders) for selected strata (at the micro or nano level), or by fine-editing particular events, in order to create alternative swing or groove effects.

[00191] The resulting variations may then be substituted for the original molecules in an alternative arrangement. Additionally, aspects of captured behavior signatures (see scenario 1 above) from user selected corpora may be selectively applied to generator results, or user- created instrument tracks. This may be done at different levels of resolution (i.e. independently across the 3 layers of the representation).

Extensions

[00192] The present system and method may be adapted for various extensions, such as:

1. Creating surface and structure handlers to generate tonal music variations, i.e. rhythmic chord generator.

2. Implementing microform maps to support alternate time signatures than 4/4, ( 3/4, 6/8 etc.).

3. Extending the layout schemes to provide a visual framework for the display of the cumulative constituent material for a song (or selected tracks of a song), the display of the sequential path of that material through the course of a song (or across selected tracks), the comparison of constituent material between two (or more) songs(/tracks), and the real-time display of material changes during playback of a song (or selected tracks of a song).

4. Controlling timbral information with surface handlers by differentially accessing instrument-synthesis parameters (must be customized to particular synthesis architectures).

[00193] Thus, alterations, modifications and variations can be effected to the particular embodiments by those of skill in the art without departing from the scope of this disclosure, which is defined solely by the claims appended hereto. [00194] The present system and method may be practiced in various embodiments. A suitably configured computer device, and associated communications networks, devices, software and firmware may provide a platform for enabling one or more embodiments as described above. By way of example, FIG. 33 shows a generic computer device 500 that may include a central processing unit ("CPU") 502 connected to a storage unit 504 and to a random access memory 506. The CPU 502 may process an operating system 501 , application program 503, and data 523. The operating system 501 , application program 503, and data 523 may be stored in storage unit 504 and loaded into memory 506, as may be required. Computer device 500 may further include a graphics processing unit (GPU) 522 which is operatively connected to CPU 502 and to memory 506 to offload intensive image processing calculations from CPU 502 and run these calculations in parallel with CPU 502. An operator 507 may interact with the computer device 500 using a video display 508 connected by a video interface 505, and various input/output devices such as a keyboard 510, mouse 512, and disk drive or solid state drive 514 connected by an I/O interface 509. In known manner, the mouse 512 may be configured to control movement of a cursor in the video display 508, and to operate various graphical user interface (GUI) controls appearing in the video display 508 with a mouse button. The disk drive or solid state drive 514 may be configured to accept computer readable media 516. The computer device 500 may form part of a network via a network interface 51 1 , allowing the computer device 500 to communicate with other suitably configured data processing systems (not shown).

[00195] In further aspects, the disclosure provides systems, devices, methods, and computer programming products, including non-transient machine-readable instruction sets, for use in implementing such methods and enabling the functionality described previously.

[00196] Although the disclosure has been described and illustrated in exemplary forms with a certain degree of particularity, it is noted that the description and illustrations have been made by way of example only. Numerous changes in the details of construction and combination and arrangement of parts and steps may be made. Accordingly, such changes are intended to be included in the invention, the scope of which is defined by the claims.

[00197] Except to the extent explicitly stated or inherent within the processes described, including any optional steps or components thereof, no required order, sequence, or combination is intended or implied. As will be will be understood by those skilled in the relevant arts, with respect to both processes and any systems, devices, etc., described herein, a wide range of variations is possible, and even advantageous, in various circumstances, without departing from the scope of the invention, which is to be limited only by the claims.