Author Archives: Tim Neumann

About Tim Neumann

Learning Technology Fellow at the Institute of Education, University of London, based at the London Knowledge Lab.

Notes from online seminar about Assessment Efficiency

The University of Hertfordshire presented their Strand A iTEAM project about ‘Integrating Technology-Enhanced Assessment Methods’ as well as the ESCAPE project on 15th March 2012.

Educationally effective and resource effective assessment is the gold standard of assessment methods, which often is difficult to achieve. In the postgraduate arena, summative assessment in the form of 5,000 word essays is often the norm, but how can we be sure this is educationally and resource effective?

The ESCAPE project attempts to answer this question, and they provide a toolkit to ‘calculate’ the time spent on different assessment methods (calculator spreadsheet not yet available publicly at the time of the session). This approach is not too dissimilar from the LDSE project‘s attempt to quantify the learning design process (and would work well as another component of the LDSE toolkit), and it suffers from similar difficulties: The data input is quite often based on guesswork and might not reflect reality adequately enough, though the point was made that the toolkit promotes reflection, which might unearth new insights, and more importantly, it can be used for rough (or highly accurate, depending on the baseline data quality) comparisons of different assessment types.

The online discussion quickly pointed out potential misuse of time calculations, in that they might favour efficiency over quality. However, the presenters insisted that assessment improvements should be about making thoughtful choices and not going for the cheapest option, and the tools were developed with this goal in mind. Indeed, the presenters acknowledge that the real picture is more complex than a set of numbers, and one should not forget the actual purposes of assessment, including the timing of assessment.

The ESCAPE project has therefore developed a series of assessment patterns to visualise how various forms of assessment and the associated feedback is used within modules, and how they interlink. The presenters emphasised that a one-size-fits-all approach is not really desirable – ideally, any assessment item would inform subsequent assessment items, either within one or even across multiple modules. Examples of assessment patterns are available from the Effective Assessment in a Digital Age Workshops site (see Session 7).

It is this final part of the presentation that displayed the highest relevance to our Assessment Careers project, as the ESCAPE team recognises the value of feedback that affects learning (and other assessment items) across many modules. ESCAPE’s timeline visualisation approach is something we might want to adopt – apparently the timelines/patterns have been used at several institutions already with great success, even as a part of validation processes. Interestingly, a team at Greenwich University picked up this idea and is currently developing an online tool to visualise assessment in modules in a similar way.