Today’s webinar consisted of accounts from the Curriculum JISC projects which are finished/finishing on how they managed large scale organisational change. Contributors gave tips on what worked at differnent stages. Some key issues I identified for the AC project at 3 stages are:

• Start of project-resisting pressure to deliver too soon which I don’t think we have a problem with. Also stakeholder engagement which maybe we need to address including IT, registry and QA.
•  Running the project-there was a discussion about formal reports being very dull and alternative such as use of social media, video, stories of problems and how they were solved and use of process mapping. But commitees (and JISC) want formal reports so not sure how we can be more creative. Another approach suggested was the submarine approach where not much is publicised during the project until there are results which are disseminated.
•  Finishing the project-a long way off for us-there was agreement that it is not the system that is the product but the environment and what people do differently. We have discussed this in our meetings that enabliing more discussion across the IOE about the role and purpose of formative feedback would be a useful outcome.

When and where to have more conversations about the project at the IOE will be important for us.

See recording at http://bit.ly/jiscdslschange

Gwyneth gave a presentation about the project at the IOE’s learning and teaching conference. The presentation also involved an update from the institution’s Assessment Working Group. Gwyneth used the opportunity to invite discussion and feedback from staff, which raised several issues for us to think about, including:

• The issues raised seemed representative of the experiences of most – but not all – of the people in the room. If anything, people expected greater use of essays than was reported in relation to formative feedback.
• It was felt that people may feel more able to experiment and innovate with formative feedback than with summative feedback.
• Geography uses a bridging module between PGCE and other Masters-level courses, with a formative presentation and a written summative report.
• Some areas are already considering feedback on earlier work. In this case, feedback sheets are online; markers are encouraged to read prior feedback before marking new work. However, in this programme there’s no face-to-face feedback, which is why the written feedback needs to be careful and detailed.
• The same project also involved a standardisation meeting that generated an exemplar that markers use as a point of reference to guide current practice.
• In some areas, in the final summative work students are asked to reflect on how they’ve taken feedback on board, and also to self-assess the degree to which they’ve done this.
• Peer feedback was discussed. It was felt that this had worked well in specific programmes (high quality, specific, supportive feedback was mentioned), although there were issues, including how important it is to help students understand what a good piece of work is, what the limits of trust and confidence could be in this, etc. It was suggested that working towards peer assessment required students to make progress in relation to understanding, self-assessing, etc and so could be a good way of working through some of these issues, helping people understand their own work (and what needs to be done to it) better. It was suggested that this might be easier in a PGCE group because an environment of trust and discussion was built up across the programme. It could be harder to create this online.
• There were concerns that the flexibility of the current offer – particularly following curriculum review – could raise issues for coherence of feedback and development. Consistent contact with a tutor could help in relation to this, although there may be an issue with tutors supporting students on modules from other programmes where they don’t have expertise. Feedback on structural aspects (e.g. academic literacy) might be possible to support, however.
• Getting assessment integrated across modules is important but is likely to take years to achieve.
• Recognising the time needed for marking, assessment and feedback is important – it might be necessary to spend less time on teaching and more on assessment/feedback.
• It may help to have a conversation about what good feedback looks like at an institutional level.

The University of Hertfordshire presented their Strand A iTEAM project about ‘Integrating Technology-Enhanced Assessment Methods’ as well as the ESCAPE project on 15th March 2012.

Educationally effective and resource effective assessment is the gold standard of assessment methods, which often is difficult to achieve. In the postgraduate arena, summative assessment in the form of 5,000 word essays is often the norm, but how can we be sure this is educationally and resource effective?

The ESCAPE project attempts to answer this question, and they provide a toolkit to ‘calculate’ the time spent on different assessment methods (calculator spreadsheet not yet available publicly at the time of the session). This approach is not too dissimilar from the LDSE project‘s attempt to quantify the learning design process (and would work well as another component of the LDSE toolkit), and it suffers from similar difficulties: The data input is quite often based on guesswork and might not reflect reality adequately enough, though the point was made that the toolkit promotes reflection, which might unearth new insights, and more importantly, it can be used for rough (or highly accurate, depending on the baseline data quality) comparisons of different assessment types.

The online discussion quickly pointed out potential misuse of time calculations, in that they might favour efficiency over quality. However, the presenters insisted that assessment improvements should be about making thoughtful choices and not going for the cheapest option, and the tools were developed with this goal in mind. Indeed, the presenters acknowledge that the real picture is more complex than a set of numbers, and one should not forget the actual purposes of assessment, including the timing of assessment.

The ESCAPE project has therefore developed a series of assessment patterns to visualise how various forms of assessment and the associated feedback is used within modules, and how they interlink. The presenters emphasised that a one-size-fits-all approach is not really desirable – ideally, any assessment item would inform subsequent assessment items, either within one or even across multiple modules. Examples of assessment patterns are available from the Effective Assessment in a Digital Age Workshops site (see Session 7).

It is this final part of the presentation that displayed the highest relevance to our Assessment Careers project, as the ESCAPE team recognises the value of feedback that affects learning (and other assessment items) across many modules. ESCAPE’s timeline visualisation approach is something we might want to adopt – apparently the timelines/patterns have been used at several institutions already with great success, even as a part of validation processes. Interestingly, a team at Greenwich University picked up this idea and is currently developing an online tool to visualise assessment in modules in a similar way.