Conceptualising and supporting the learning process by conceptual mapping

Within the developing educational system learning becomes a communicative process between the teacher (author/presenter) and the student audience. The presented educational content takes into account the previous knowledge of learners, accommodates their varying objectives, and supports understanding through the language used and the rhetorical/functional structures that indicate the intentions of the teacher. Also with the aid of technology, learning is becoming a more participative and collaborative process (Barron [2006]; Collins and Halverson [2010]; Chang et al. [2007]; Hartley [2010]; Lai et al. [2007]; Saljo [2010]; Selwyn [2010]; Stahl [2005]).

Typically educational content will include concepts (their functions, classifications and properties), the explanation of processes and their enablement through agents and achievements, causes and effects, and conditions and consequences. These elements are likely to lead to the exposition of principles and their interactions, and to the justification of claims. For their part, students, particularly in Higher Education, have experience with these types of educational materials, and have developed expectations of structures which can guide their objectives of comprehension and understanding. Teachers are also aware of the limitations of working memory, and the capabilities of their audience and will section and structure the material, introduce illustrations and examples, and include introductory and summary statements and focusing questions. Hence, comprehending and understanding and conceptualising become constructive processes, as teachers seek to fashion and facilitate student learning through these pedagogies.

However, within the curricula there are certain concepts, procedures and principles which experience has shown present students with learning difficulties, but which require a secure understanding if further problems are to be avoided. In addition, much more educational material is now readily available and accessible through published sources and the Internet, (e.g. MOOCs) directing more research to their effective designs. It also places greater emphasis on students having appropriate skills and learning strategies which enable them to use such resources efficiently especially as technology allows greater opportunities for interchange and collaboration.

This paper suggests that learning objectives of comprehension and understanding can be supported and better secured through a process of conceptual mapping. However, it is necessary that this conceptualising process is transparent, links to the study procedures of learners, is easily accessible and economical in its use of time, and can accommodate the varying levels of detail and depth of students’ learning objectives.

Further sections of this commentary outline the rationale of conceptual mapping for supporting the learning process, and discuss and illustrate the stages of mapping in relation to learning objectives of comprehension, understanding, conceptualising and extending/exploiting knowledge. The paper then establishes conventions and a vocabulary for these stages; comments on training issues and techniques for such mappings, and outlines the type of evaluation studies to reveal its effects. Finally, it also concludes with a summary discussion of computer supports (apps) to better sustain the conceptual mapping approach.

When reading educational materials students usually annotate the texts to show the structure of sections by noting their functions, i.e. what they are trying to achieve through descriptions, explanations and expositions of principles along with the justification of claims. Informal discussions with a sample of twenty undergraduates undertaking such tasks (on- and off-line) noted that important terms and concepts are likely to be underlined, with interpretive comments and references to previous knowledge or other sources. This brings the material into line with their learning objectives of comprehension where the aim is to follow the ‘gist’ of the content and to be aware of its relevance to learning requirements. However, such techniques can become more organised through Text Charting where the idea is “to show the work of each paragraph or section by identifying what it is doing as well as what it is saying, and moving away from summary to analysis”. The methodology finds it useful to distinguish Macro-charting (those parts of the text that work together in descriptions, explanations or arguments) and Micro-charting that identifies the elements which make up these structures. Such methods encourage students to consider more closely the functional or rhetorical structures which identify the ways authors communicate their intentions. There are several sources and resources of Text Charting, e.g. the AVID website with suggestions and examples distinguishing between what authors are saying and what they are doing. A concise summary is provided by Jentien and Hughes (Table 1) and a brief exposition on “Hot Air Balloon and Ballooning” (taken from presentations by two experienced teachers) is used to illustrate the technique (Figure 1).

Macro-charting	Micro-charting
How do we do macro-charting?	How do we do micro-charting?
• Break text down into sections—identify “chunks” or parts of the text that seem to work together to DO something for the overall argument.	• Break down sections of text by paragraph to analyze what each paragraph is doing for the overall argument.
• Draw lines between sections and label each one, annotating them with “doing” verbs: providing context, making a claim, supporting a claim, rebutting counter argument, illustrating with personal anecdote, describing the issue, etc.	• Detail the smaller “moves” and strategies made within paragraphs: note when, where, and how an author makes a claim, cites evidence, and/or supports his/her arguments using a rhetorical strategy.
Why do we do macro-charting?	Why do we do micro-charting?
• Macro-charting helps with under-standing structure of argument, as well as locating claims, supporting evidence, and main argument.	• Micro-charting can serve as a way to thoroughly understand in a detailed way how a text is put together.
• Macro-charting guides students toward identifying relationships between ideas.	• Micro-charting encourages readers to look more carefully and closely at a text and helps us to focus our reading on tasks asked for in prompts.
• Macro-charting brings awareness that behind every sentence there is an author with intent who makes rhetorical choices to achieve his/her aims.	• Micro-charting brings awareness of the specific rhetorical choices made throughout a text (addressing particular audiences by making deliberate moves).

Paragraphs [1] and [5] give introductory priming sentences and a rounding-off paragraph, but the other sections describe the components of the balloon and their properties [2], identify the buoyancy and weight forces with an explanation of how the balloon rises by expanding and losing air [3], and explaining how the balloon falls by opening the vent causing denser air from the atmosphere to increase its weight [4]. These Macro comments note the main functions of the paragraphs, but the Micro sentences detail the descriptive properties of the balloon components, show the explanatory cause-effect-consequence links of the processes which cause the balloon to rise and to fall.

Text Charting can be applied flexibly to meet students’ learning intentions. Priming sections may be only briefly noted or even ignored, descriptive sections may be summarised in a diagram or schematic. In many cases Text Charting with its annotations may be all that is required to comprehend and judge the relevance of the material as it is filed away for future reference. However, certain sections such as those concerned with the explanation of processes, or expositions that describe a principle at a higher level of abstraction, or competing arguments that justify a claim, are likely to require a deeper probing of the functional structure at the micro level to secure an adequate understanding.

Understanding as a learning objective needs more than a comprehension of the gist of the material carried by the progression of pieces of information: it requires an awareness of the connectivity of these propositions and how they are fashioned to achieve their effects. This directs attention to relational meanings within and between propositions, and to the intentions of the author in expression and design to assist the understanding of readers in ways that enable them to utilize the material within their educational objectives. It is worthwhile to achieve such functional analyses in a principled manner using Rhetorical Structure Theory (RST). This was developed from a wide range of sources to explain the coherence of texts by considering the functions propositions aim to achieve through the relations they contain. These establish the structural place of each proposition in the text. A key paper is that of Mann and Thompson ([1988]) and RST systems have become well developed and discussed (Taboada and Mann [2008], [2013]).

Under the Macro structure of Descriptions, Explanations, Expositions and Justifications, the Micro functional structure can be undertaken in a principled manner by applying RST in ways which emphasize the communicative roles of the text structure. Such analyses can reveal the pedagogical intentions of the material and support a more secure understanding of the content by students. A useful distinction (in the adaptation of Rhetorical Structure Theory proposed in this paper) is between concept-objects and their descriptive properties, and agents/agency that bring about changes (achievements, effects, consequences) as they fulfil their functions (e.g. the burners in the Hot Air Balloon). The unit of analysis (microstructure) is the propositions in the document where the sentences (i.e. subject and predicates which may be compound) are sequenced and connected into larger episodes and themes (e.g. paragraphs providing the macro-structure). When concept-objects are treated descriptively the propositions are likely to include classification (is-a member of a higher-order class) has-properties (e.g. size, qualities) has contents/components (which may have their own descriptions), function/purposes and similarity or contrast comparisons. These descriptions form the predicates of the subject context but it is the labelled relations to the predicates which show the structural meaning of the text, as delivered by the author and perceived and interpreted by the student/reader.

For example, in considering the structure of paragraph [2] in the Hot Air Balloon, the first proposition may be represented as Figure 2.

Notes might not be required, or they might be included as click-on information attaching to the nodes as they are not needed to be shown explicitly in the structure. It is also likely that the student will represent the content of paragraph [2] as a labelled diagram.

Paragraph [3] is an explanation of the process by which the balloon rises and the predicates are concerned with enablement. The analysis draws on different predicate relations concerned with agency-achievement, cause-effect, conditions-consequence, and cause-consequence.

Note that Buoyancy is a force Agent so would be color highlighted in the text, or given a surrounding ellipse envelope, but it might also be represented as a Condition (since the balloon displaces air) Consequence (the balloon has buoyancy force). It is a matter of interpretive emphasis. The fact that the balloon rises as it loses weight might be obvious, but it is important to realise that it is the manipulation of the balloon air weight which governs the rising and falling of the balloon. Again the proposition, the burners—heat the air could be considered to be cause-effect, or agency-achievement: it is a matter of interpretative preference and our pilot trials show it is likely there will be such differences between student mappings. The sequence ‘heats the air … consequence … air expands (and leaves the balloon) … consequence … balloon air becomes less dense … consequence … balloon rises’, is perhaps better concluded with ‘air leaves the balloon … consequence … balloon loses weight and rises’. This is implied but not directly stated by the teacher—so there is a possibility that students will consider that the balloon rises because the air is less dense, and that it will stop rising when the density of the air inside and the atmosphere outside the balloon are the same. This is incorrect since it ignores the weight of the balloon and contents. A similar analysis can be undertaken with Paragraph [4] where the balloon weight increase is explicitly noted, but the consequence in relation to the buoyancy force is again implied but not directly stated.

It is worth noting that conditions can be expressed in several ways, e.g. If … When … Since and the Condition-Consequence predicates can be expressed in reverse, e.g. When the burner is fired (Condition) the balloon air is heated (Consequence). Alternatively, the balloon air is heated (Consequence) when the burner is fired (Condition) … it depends on where the author wishes to place the emphasis. In the latter case it is the consequence, and for the consequence to be achieved the condition must hold. In addition, the author could choose to emphasize the Buoyancy-Weight principle by noting or bringing together all the conditions and consequences for the rising and falling of the Balloon. The predicate summary could then be represented as Figure 3.

The predicate sequences representing the processes could then be attached to the consequences.

The explanations of compound processes or principles can be more complex in their representations as shown in the ‘Weather’ example below. Sentences may refer to two contrasting objects/agents (e.g. high and low latitudes) with differing and contrasting properties (e.g. temperature differences, daylight hour differences) which are interconnected. In such cases it is useful to identify objects/agents and their key contrasting properties which are most important or most related to them. In the weather example, daylight hours and temperature differences are noted with the property contrasts at high and low latitudes. A connective consequence arc can show the overarching consequence relation between daylight hours and temperature differences. However, these differences depend on the time of the year, both being greater in winter, with the causal consequence being the daylight hour differences winter to summer.

Example: “High and Low latitudes (of the Earth) show differences in temperature and in daylight hours. These differences in temperature are greater in winter because the daylight hour difference (e.g., polar day/polar night…equatorial day/night) between the latitudes is increased” (Figure 4).

With these illustrations in mind, a vocabulary of types of structural/functional predicates is proposed for use in analysis. More detailed/discriminating predicates could be noted but this becomes cumbersome. Predicates are at a higher level of abstraction and can tolerate interpretations under the label with readers adding a note for additional clarification: agreeing a vocabulary (pseudo-ontology) more readily enables mappings to be compared and interchanged, and also helps the provision of computer supports.

i.e. concepts being described.

i.e. agent concepts capable of actively engaging in a process.

Principle representation

A representation of principles is shown in Figure 5.

Conceptualizing is a process of organizing understanding to establish ownership over a topic so that the knowledge can be used to meet a range of objectives such as describing and explaining or arguing towards a viewpoint. It is distinctive, reflecting the individual’s knowledge, and is structured and open to discussion. This does not imply the conceptualization is entirely consistent and adequate to meet requirements, but the owner believes it to be consistent with current/previous knowledge (else it would be amended) and that it is fit for purpose while recognizing that discussion and problem-solving exercises could show deficiencies. That is part of the ongoing learning process and the conceptual structure if given an adequate representation is a useful framework for such amendment and exploitation.

There are several types of knowledge mapping each directed to its particular objectives and using diagrammatic methods to show the connections between elements of the structure. For example Mindmapping in which the aim is to generate associated ideas around a central concept or theme. The node-arc visual links tend to have a radial organization and images and notes can be attached. The focus is on associated meaning provided by the individual and setting out their mental constructs. Therefore, Mindmapping is useful for generating ideas, creating aide-memoirs and brainstorming. On the other hand, the important contributions made to mapping by Novak and his team takes a different approach in building knowledge systems (Novak [2002]; Novak and Wandersee [1991]; O’Donnell et al. [2002]). This was influenced by Ausubel’s views on meaningful learning (Ausubel [1968]) and identifies propositions that link two or more concepts to form a “unit of meaning” which is identified by the link name. In construction, it is useful if the concepts are arranged in hierarchical fashion within the knowledge context being considered, and cross-links and notes can be added to give clarity and show wider associations. The visual mappings may be large but parts can be sectioned to help the navigation of the maps. Overall, the aim is to place new perspectives on previous knowledge (hence meaningful learning) and the methodology has been, and still is influential, particularly in lesson design and teaching in school science education.

The focus of this paper is the macro–micro study of educational material presented to students, and the ways this can be comprehended and understood through attention to its functional structure. The analysis following Text Charting under typical section episodes of Description, Explanation, Exposition and Argumentation, results in a list of rhetorical predicate links which themselves need to be linked to create a conceptual map. This process needs some reflection to ensure the mapping, through the visual arrangement of its nodes/arcs, assists navigation and also stimulates exploitation, e.g. by considering implications, solving problems, and linking to further work. The conceptual map has to repay, and be seen to repay, the effort of its construction in the achievement and expansion of learning objectives. Before considering examples the schematic (Figure 6) will be a useful guide to the process.

Following this scheme the conceptual map of the Hot Air Balloon can be produced (see Figure 7). The initial mapping (which might in part be constructed in the previous predicate analysis stage) is likely to encourage reflection and result in a more compact version of the content with the visual arrangement giving clearer navigation.

It is often useful when reflecting on a conceptual map and exploiting its implications, to produce a text summary narrative which gives a connectivity to the navigation. For example:

The Hot Air Balloon through the properties of its components (e.g. the volume of the balloon providing buoyancy through the air it displaces; and the envelope, basket and its contents providing weight) experiences an interaction between the Buoyancy and Weight. [This puts the focus on the Principle.] When Weight is greater than Buoyancy the Balloon falls, when Weight is less the Balloon rises, and when equal the Balloon floats. To decrease the Weight the burner heats the air which expands and leaves the balloon: to increase the Weight the vent is opened, hot air leaves the balloon and cooler/denser air enters from the open base of the Balloon. [This puts the focus on the Process.] By use of the burner and the vent, the pilot can control the altitude of the Balloon.

A short exposition taken from a document on Knowledge Management provides a second example (see Figure 8).

In the first stage of Text Charting, the key concepts and predicate elements are underlined or noted and the thematic and functional content of the paragraph episodes can be given as a summary comment. For example, the first paragraph [A] makes introductory distinctions between the terms ‘data’ ‘information’ and ‘knowledge’ as they are used in the text. Paragraph [B] distinguishes between internal (tacit) knowledge and external (tangible) knowledge as assets in the knowledge management of an organisation. Paragraph [C] moves towards documenting interactive tacit knowledge so that it can become a tangible asset within a data depository. Paragraph [D] considers how such knowledge is to be organised within the structure of the organisation, and the final paragraph [E] develops procedures for reusing and retaining knowledge within the system.

Identifying the predicate types and their relations using the rhetorical vocabulary provides the ingredients for the conceptual mapping. It is usual to follow the chosen paragraph episodes in linking the predicates, and noting or indicating also the linkages between the paragraphs themselves. Typically, and in these ways, a preliminary mapping is produced. On reflection, this can be refined and better organised visually to identify the main themes of the material and to clarify the navigation of the map.

A conceptual mapping of the Knowledge Management article is shown (see Figure 9). There are several comments to be made.

The first paragraph [A] was not included in the map, as the distinctions that were made and are to be kept in mind could be placed in note form as a vocabulary node. The organisation and its management of knowledge assets are the main foci and head the mapping. The left and right portions of the map are concerned with the varying properties of tacit and tangible knowledge, resulting in a stored data repository when and where tacit knowledge is given a tangible form. The left part of the map continues with the process of how tacit knowledge requires interaction mechanisms to give it a tangible form suitable for the organization, so that experts and users can communicate.

The right hand portion of the map concerns structured and unstructured knowledge assets and again these require an organisation (so they can be stored and searched) which matches the administrative patterns of the enterprise. Some Click-on Notes are placed in boxes to indicate particular pointers that help the navigation. Since conceptual maps take into account the previous knowledge and the learning intentions of readers they can vary in the level of detail and the notes which are added. However the broad thematic structure of maps on the same content should be similar, and differences should focus discussion.

Not every statement in the presented text warrants a rhetorical predicate representation, and not every rhetorical predicate identified in the text will necessarily appear in the conceptual map. For example, some statements are incidental notes or illustrations; others are well known to the student and/or can be easily inferred or subsumed within larger span predicates. The student/user will make such value judgements in line with learning objectives. Also, thought should be given to the visual structure of the map so that differing stages and functions (e.g. descriptions and processes), and map navigations are clear.

A text summary navigation of the map can be useful. For example:

Organisations have Knowledge Assets that are Tacit (in the heads/experience of employees), or are Tangible (e.g. external documents) which can be placed in an Information/Database. To make use of Tacit Knowledge it should be given a Tangible form, for example, through interactions (synchronous and asynchronous) between employees and knowledge experts. Then this new information can be stored in the Data Base. But for this to be searched (e.g. by keyword metadata) and used by employees, it must be structured in ways that suit the needs of the organisation. Such interactions should accommodate both synchronous and asynchronous interchanges.

Prostate cancer diagnosis and treatment

It has taken the experts the best part of 25 years to decide that the benefits of the prostate specific antigen (PSA) blood test for those with a “low-risk” prostate cancer are so equivocal, the consequences of treatment so adverse, that patients are much better advised to “wait and see” rather than consent to an operation.

To be sure, the logic behind the test treatment seemed, initially, unassailable, since it can detect the presence of cancer cells in the prostate gland ten years or more before they become apparent—allowing time to extirpate them before they can cause any mischief. This does, by definition prevent the cancer from spreading, but the overall impact on the disease has proved to be much less than predicted, as emerged when researchers anticipating that men from Seattle would benefit from a fivefold higher rate of testing than in Connecticut found the mortality rate to be identical in the two states. This would matter less were the gland not located in the worst possible place for its successful removal—within millimetres of the bladder and rectum and in proximity to the nerves that control their function. Hence despite the best efforts of the most accomplished surgeons, the majority of patients are still, a year after surgery, impotent and having to wear adult diapers. This is an inordinately high price to pay, physically and emotionally, for those patients with “low-risk” cancers who (as a study last year revealed) left untreated have a 97 percent chance of being alive 12 years later. The advocates of screening inevitably dispute these figures, but as the developer of the PSA test, Prof. Thomas Stamey of Stanford University, subsequently observed, “Never in the history of medicine has one disease been so over-treated”.

A text summary of the mapping (shown in Figure 10) might be:

The PSA blood test for the early diagnosis of prostate cancer can detect tumour cells ten years before they become apparent, but the operation to remove them is technically difficult (due to the close location of the bladder, rectum and controlling nerves). Consequently patients can experience serious effects a year afterwards. Moreover differential testing rates (High/Low, Seattle/Connecticut) showed no difference in mortality rates, and untreated “low-risk” patients had a 97 percent chance of being alive 12 years later. The advice is that “low-risk” patients should “wait and see”.

However, within three months, evidence from a Cambridge University study qualified the “wait and see” advice. These data showed that some 50% who had been diagnosed having slow-growing cancers had more dangerous tumours, and that the biopsies had not detected that the cancer had spread beyond the prostate. These predicates could attach as qualifiers/counters to the conclusion node.

Developments in technology have made educational resources and tools available to students but, as well as opportunities, this presents challenges to the management of learning, to methods of improving understanding and to ways of learning from others (Facer and Sandford [2010]; Lai et al. [2007]). It is argued that moving towards these goals is assisted by identifying learning objectives of comprehension, understanding and establishing greater ownership over material by conceptual mapping which can encourage reflection and provide a framework for interactions with others (Puntambekar [2006]). Underpinning the approach is the functional/rhetorical analyses which indicate the intentions of authors and the ways sentences are crafted and framed under larger episodes of descriptions, explanations, expositions and justification of claims. The methodologies of Text-Charting, rhetorical analysis and conceptual mapping seek to establish and represent connectivity and structure and give a visual representation of the reader’s interpretive understanding which is open to others. But how will, and how do the procedures work out in practice? And are they likely to meet expectations and lead to continuance by students within the pedagogies of institutions (Lee [2010])?

Our preliminary mapping experiments and discussions with students have identified several issues. The techniques of Text Charting and conceptual mapping are unfamiliar to most students and initially the mapping is time consuming. However, with well-chosen examples and some experience these activities become practiced. Clearly there has to be recognition of a need for the enterprise in terms that warrant and justify attention and effort. At the outset there should be an expectation that the proposed learning supports are capable of achieving their aims, with some evidence from data and student experience. These perceptions are important, influenced by the supports and tools being tractable, and giving a feel of enlarging student capabilities. Specifically the techniques need to repay the time and effort in their application; do they bring the benefits students were looking for? A further issue concerns how the facilities can be exploited, e.g. in learning from others through interaction, which brings in social expectations and the attitudes of fellow students and staff. For the techniques to be grounded they need to be compatible with the pedagogies of institutions and their technological developments. And is the enterprise likely to lead to further insights of learning processes that give a wider scope to educational attainments?

Some research has shown that where learning materials require less analysis multimedia presentations are more efficient, but where more probing of content is needed then “lean media” such as text is more effective (Lui et al. [2009]) Text Charting and functional analysis support these deeper aims with conceptual mapping giving visual representations. However although the techniques are easily done by hand by overwriting and annotating documents, placing the work on the computer gives a better organisation and wider access for cooperation and discussion. The techniques discussed previously have tablet computers and apps in mind. Content can be transferred to the tablet and document/mapping apps are available to highlight and draw round parts of the text, to link these nodes and hand-label by stylus the predicates, which can be chosen from the predicate vocabulary list. From these predicates (automatically printed at the end of the document) a drawing app enables the conceptual map to be constructed, visually arranged and edited, with iCloud and Dropbox facilities used to give wider access for comparison, discussion and small group activities. This requires, though, an acceptance of such technology support which links to design (Selim [2007]) to the ease of understanding and using the system (Venkatesh [2000]; Lui et al. [2009]) and to the quality and value of the interactions which result (Pituch and Y Lee [2006]). Indeed students may stop using virtual learning tools after first practices and continued use is not high (Chui et al. [2005]; Sun et al. [2008]). Such data place the focus on the content and conduct of the training methodology and its materials, and it is through the consistent use of the e-learning methods which brings success. To this end, are the contexts and tasks motivating, and do they encourage students to note their success and continuing improvement (Willoughby et al. [2000])? A further important issue is the social context, the opinions of other students, and the ways the experience can be enhanced and exploited through learning from others (Sun et al. [2008]; Wan [2009]).

Our preliminary studies have shown that although the procedures bring a focus to the learning, the resulting maps representing student understanding show differences. Key concepts and relations are similar but vary in detail and the mapping structures can show varying perspectives which should be useful for discussion. In the Knowledge Management example, distinctions are made between Tacit and Tangible Knowledge, how these are required by the Database, which has to be structured to meet the needs of the Organisation. Differences in student mappings reflect their learning objectives, previous knowledge and interpretations, and it is these factors which inform discussion. While it is a matter of research interest if and how computer ‘intelligence’ can compare and suggest improvement to mappings, and produce navigated summary texts from them, the emphasis here is on the students themselves comparing and discussing their conceptualizations.

In this regard related research has developed interface and supports for continuing dialogue and argumentation. The example of Prostate Cancer and the justification of claims following the mapping is a useful context. Influenced by Toulmin ([1958]) and Drawing on Speech Act Theory to develop a vocabulary suitable for students, and collaborative dialogue analyses, Ravenscroft and his collaborators built the on-line InterLoc system as a framework that maintained focus, showed dialogue function and assisted connectivity in these interchanges (Ravenscroft [2000], [2007]; McAlister et al. [2004]; Ravenscroft and McAlister [2008]). Connectivity was maintained by identifying dialogue targets and Move categories which showed Intention (e.g. Inform, Challenge, and Agreement). Within these categories at a micro-level sentence openers were available to clarify what the text contributions were trying to do, and the rhetorical link labels in conceptual mapping have a similar function. This enterprise is successful in both its conceptions and applications.

Looking forward, the claims of conceptual mappings and their exploitation need to be substantiated and the processes validated, through illuminative evaluation studies. A first issue is to be clear on the question(s) of interest because they relate to the selection of educational materials; the mapping procedures (i.e. Text Charting, Rhetorical Functional Analysis, Conceptual Mapping, and Exploitation); the Evaluation Model and Experimental Design; the data measures and their analysis and interpretation; with commentaries on the technology and other supports of the learning process; and on the range of potential applications. In brief these issues require research programs with authentic training materials.

The experimental design can become complicated because there are interlinked components of Text Charting, Rhetorical Predicate Analysis and Conceptual Mapping. What are the contributions of each, and do they overlap? The progression of these techniques suggests a planned comparison design which is more focused than the usual post-hoc comparisons following an Analysis of Variance [ANOVA]. The planned comparisons have the hypotheses set up before the experiment proceeds. Students are then randomly assigned to the treatment groups, the test measures taken and analysed [e.g. ANOVA]. The difference is that with post-hoc ANOVAs the actual comparisons are not strictly statistically independent, hence the possibility of Type 1 and Type 2 errors. With planned comparisons the hypotheses are orthogonal and the information given is non-redundant.

In addition to the performance test measures it is useful for studies to provide data on the actual processes students were following which gives the evaluation an illuminative component that can be gathered by observation, student self-report questionnaires (fixed-choice and open) and through focus group discussion. Such information guides future work, e.g. on improving computer supports and on exploitation and/or amendment to the materials and methods. These activities enlarge the scope into a more general consideration of how technology can enrich the educational experience, and to student and teacher perceptions of the methodology.

This approach with its large scope in the questions of interest can benefit from an empowerment evaluation (Fetterman [2001]; Cousins [2005]; Fetterman and Wandersman [2007]). This takes the view that evaluation is a responsibility shared by all participants, not merely the external evaluators, as it seeks to determine what has been achieved, what factors and processes have brought them about, and how the enterprise can be improved in its practices. Such an evaluation program typically involves surveys, testing, and focus group interviews combined in ways that enable the scheme to be participatory and collaborative.

There is controversy. How is the evaluation to be controlled? What does it mean to be an evaluator and what self-checks on bias are needed? These have to be taken into account if the evaluation is to serve its multiple purposes with the various internal and external evaluation components operating in partnership. And overall, a key interest is not only to consider how the technology and its developments can support these aspects of deeper learning, but how they can be brought into the pedagogies and practices of institutions.