Assessment of skills

Definition: The need for graduates, either from schools or further/higher education, to acquire skills beyond those linked with subject disciplines has been accepted for some time. Such skills have been described as ‘soft skills’, ‘essential skills’, ‘functional skills’, ‘employability skills’, ‘entrepreneurial skills’, etc. and relate to the personal qualities other than cognitive abilities needed by learners to be wholly effective outside of the education environment.

Description: In recent years, there has been a growing expectation that one can measure, with precision and accuracy, the many positive personal qualities other than cognitive ability that contribute to student wellbeing and achievement. Four broad approaches to the assessment of generic skills have been identified: holistic judgments by teachers, portfolios created by students, assessment based on work experience, and assessment using purpose-developed instruments, each of them with its advantages and limitations according to the purposes of program evaluation, accountability, individual diagnosis, and practice improvement .

Benefits: Among other things, a sound approach to the assessment of skills will provide:

  • a means of providing feedback to learners on their acquisition of generic skills and a framework for their improvement
  • a mechanism for communicating the scope of generic skills to learners, training providers and employers, providing a rich source of information about individual achievement, with supportive evidence
  • an opportunity to undertake authentic assessments which occur within a work context or one that closely simulates it
  • a summary of the performance of individuals readily accessible by employers
  • a cost-effective means of collecting performance information, individually and at aggregate (institutional and system) levels.

Challenges: Since the inception of key competencies, their assessment has been a challenging issue for practitioners. On the one hand there is a lack of clarity about individual constructs of each generic skill. In addition, the evidence on transfer and the ability to apply a skill in different contexts is difficult to attain. Progression is another big issue and the identification of levels of performance is far from resolved. Each measurement tool has its advantages and limitations, so that the accuracy of data obtained could be questionable. Finally, we still do not know how to act on data regarding the personal qualities of students in various classrooms or schools.

Applied to entrepreneurial education: While a variety of methods has been used in the summative assessment of generic and/or entrepreneurial skills, there would appear to be a role for formative assessment methods that seek to assist student learning and enhance generic skills performance. A combination of different measures including self-report questionnaires, teacher-report questionnaires, performance tasks and portfolio assessment may produce a more accurate picture of progress in students’ ability and disposition to use such skills. In the classroom the teacher must ensure that they signal key elements of entrepreneurial skills and encourage students to analyse their use of the skills and to contemplate alternative situations in which it might be applicable.




Reflective Assessment

Definition: Reflective assessment is when students are included in the assessment process by being asked or required to reflect individually or together with others upon their own learning experience, either in writing or orally.

Description: Reflective assessment is a method that belongs to the formative assessment family of methods, although it can be used summatively. It is about assessment primarily for the purpose of improving learning. It is further an active and empowering assessment method since it engages the student in self-evaluation and self-reflection. It leans theoretically on the crucial role that meta-cognition plays in learning, in that it asks students to think about their own thinking. Gibbs has specified a cycle of six steps in reflective learning that is often applied when teachers design reflective assessment . Some ways to implement reflective assessment include, for example, asking students to write down “I learned” statements, asking for an oral “turn and talk” with the student next to them, asking students to write down what is clear and unclear from the lesson on a piece of paper, and asking students to compose a journal / dairy.

Benefits: Fostering student meta-cognition has been shown to be one of the most powerful ways to improve student achievement . Reflective assessment is a relatively simple and straightforward way to implement formative assessment in the classroom, and also makes students ‘own’ their learning process to a greater extent.

Challenges: Reflective assessment is at odds with the current trend in many education systems towards standardised tests and summative assessment. Some forms of reflective assessment also generate substantial amounts of text that then need to be processed by the teacher to provide feedback back to the student.

Applied to entrepreneurial teaching: Reflective assessment is easy to apply to entrepreneurial teaching that is action-oriented and experiential. Students can be asked to reflect upon their practical entrepreneurial experiences . Since these experiences can often be emotionally strong, reflective assessment will be a natural kind of reflection for entrepreneurial education students. Reflections can be both private for the teacher’s eyes only, and forum based for the entire team or class to reflect upon and discuss together. Reflective assessment can also be easily combined with e-assessment through the use of different kinds of social media and digital discussion forums.




Learning Progressions

Definition: A learning progression is a carefully sequenced set of building blocks that students must master en route to a more distant curricular aim. They map out commonly travelled paths that students take.

Description: Learning progressions outline expected behaviours related to the skill in question, from a novice level to an expert level. Learning progressions will necessarily begin as hypotheses to be tested in real settings. They are refined through rigorous data collection and analysis in order to gain a better understanding of more complex skills and how students demonstrate them at various levels.

Benefits: Progressions can help teachers better understand how students develop and demonstrate knowledge and skills, opening what might otherwise feel like a black box as they teach and reteach. Teachers can use learning progressions as road maps to guide their instruction and assessment practices.

Challenges: Creating high-quality progressions is far from child’s play. There is no single, universally accepted and absolutely correct learning progression. Gain size matters. Some learning progressions turn out to be very complex and off-putting to potential users.

Relevance for entrepreneurial teaching: Not enough is known about the typical path students follow to learn some of the non-cognitive skills commonly ascribed to the EE competence. However, learning progressions could support a more coherent development of entrepreneurial skills by signposting a sequence of expected and observable behaviours that denote a broader, deeper, and more sophisticated understanding and application of the skill at hand. They may help teachers to establish where students are in their learning, to monitor progress over time and to evaluate the effect of interventions on learning progress.

Applied assessment methods: Formative Assessment, Constructive Alignment

Examples from practice: The project AT21CS led by the University of Melbourne has developed a learning progression for “collaborative problem solving”. Likewise, EntreComp, the Entrepreneurial Competence European Framework, suggests a progression for a broad range of skills.




Feedback Assessment

Definition: Feedback assessment is information about how the student is doing towards reaching a set goal and the next feasible steps. Feedback is usually provided by the teacher as part of both formative and often summative assessment. Feedback is a response to the student‘s work to support further learning towards a final goal.

Description: Feedback assessment comprises clear descriptive commentary as to where students are in their learning and what next steps could be taken to improve their work. Feedback supports cognition as it can help learners realise which of the knowledge and skills they hold are strong and which are weak. Sometimes feedback clears misunderstandings and misconceptions. Feedback supports metacognition, that is, students´ awareness of their own thinking. Effective feedback shows students how to consider their work, using criteria from the assignment, and apply self-assessment skills.

Benefits: Effective feedback can enhance students´ cognitive processing, increase their autonomy, foster resilience and provide strategies for next feasible steps. Feedback assessment supports students’ with self-regulation of learning, by helping them understand their learning goal and how close their current work is to it and what could be done next. Students become better at appraising their own work and learn the value of review and revision and reshaping of work for improvement. Where feedback is effecively applied: mistakes are viewed as opportunities for learning and students are encouraged to accept them and learn from them; students are not afraid to ask for help; and assessments build on strengths and practice to overcome weaknesses.

Challenges: Where work is graded, some students will concentrate on that rather than seriously analysing descriptive feedback provided. The optimum time for feedback can be a challenge as the typical sequence of classroom activities and the time to turn in assignments are not supportive to written or discursive feedback. Students often look at grading as evaluation and judgement but the feedback must preferably be experienced as description and information. From a teaching perspective, feedback can be time consuming.

Applied to entrepreneurial education: Feedback assessment can be a good way to facilitate development of generic or transversal skills e.g. creativity. The emphasis is on progress not the final product. In order for feedback assessment to be constructive and effective in EE (or other kinds of education), the following suggestions can help: Positivity, including affirmations and highlighting the good qualities in the student‘s work as well as pointing out what needs amendment; Clarity; Being specific enough to be helpful but not so specific that the work is completed for the student; Being descriptive and focusing on the work and the process the student used to do the work;and, Adopting a constructive, rather than a deminding, tone, where the student is able to respond, e.g. asking the student What could you add? Change? etc. relevant to the assessment criteria.




E-assessment

Definition: The ‘e’ in e-assessment stands for ‘electronic’. E-assessment is thus defined as when computers are used to support assessment for / of learning.

Description: E-assessment ranges from simple computer-based tests with multiple-choice questions to complex and multimedia rich simulations, games, case studies and e-portfolios. It thus encompasses a broad range of assessment methods used in computer-based learning environments, often termed ‘E-learning’. E-assessment can be used both for high-stakes and for low-stakes testing, as well as both for summative and formative assessment. Teachers can distribute questions and tasks as well as collect and analyse student responses. Some marking can be automated.

Benefits: By using e-assessment tools, various administrative tasks can be passed on to computers, freeing up time for teachers to focus on formative assessment and feedback. E-assessment can also be used to let computers adapt questions and tasks to each student’s individual circumstances and replies. The more complex forms of e-assessment can be used to assess higher-order thinking and transferable skills.

Challenges: Many of the less complex forms of e-assessment use simple right-or-wrong questions, and have therefore been criticised for relying on and promoting an outdated model of surface and rote learning, focusing solely on recall of simple facts. While more complex forms of e-assessment avoid such criticism, the resulting problem is instead the high cost and difficulty in designing and validating questions and tasks.

Applied to entrepreneurial education: Assuming that challenges around surface learning are taken into account, the less complex forms of e-assessment can be used in entrepreneurial education to increase the focus on formative assessment. Short quizzes can be administered while a course is still on-going, allowing for adjustments and feedback. The more complex forms of e-assessment often require solutions tailored to the specific entrepreneurial contents and processes for a course or a program. There are a number of entrepreneurship specific simulations and games available. Examples include SimVenture, MarkStrat and SimBrand. Another potential use of e-assessment in entrepreneurial education is to use applications not specifically designed for assessment, such as online word processors, online forums and social media. An interesting avenue for the future is to use computers to facilitate the experiential and action-based nature of many kinds of entrepreneurial education, see for example the e-assessment tool LoopMe. Such approaches have become more affordable as many students today have access to a smartphone where apps can be easily deployed for assessment purposes.




Assessing creativity

Definition: Creativity Assessment evaluates the learner’s ability to provide evidence of novel connections between ideas, artefacts and insights within a given context. The most surprising will be the most creative.

Description: In creativity assessment new connections have to be made in the mind of the learner, and these can be evidenced through iterative stages of development or reflective journals. Creative acts can be recorded and evaluated by asking: how many ideas? how are they interconnected? and how distinctively different are the range of solutions presented? Multiple and not single solutions are presented by the learner – in a response to a project that has no clearly predetermined outcome. Solutions and their connections must be well explained and justified within an appropriate context.

Benefits: A creative and knowledge intensive society can address economic and social issues and enhance growth through new ways of seeing and thinking. A creative mind can spot and respond to opportunities, navigate complex and ever changing scenarios and be resilient in the face of perceived failure, where the creative person views a failed attempt as successful learning. Entrepreneurs not only have new ideas for a business but can constantly adapt because they foresee many alternative ideas when circumstances change. Opportunity recognition is reliant on this type of creativity.

Challenges: Teachers require an awareness of when creativity is inhibited and when it is developed. For example, ‘fixed’ learning outcomes that clearly state what is needed are commonly used in education (Implement what is asked). However creative entrepreneurial people demonstrate divergence and breadth of thought; they find alternative solutions within changing circumstances (Innovation). They grasp opportunities that even teachers may not have seen. Teaching activities such as ‘brainstorming’ are usually short-lived and do not necessarily enhance long-term abilities. The challenge is to continuously develop a flow of creative outputs from learners.

Implementation examples Innovation examples
Can the student write and follow a business plan? Can the student respond positively to short term and ever changing venture environments / do they come up with new ideas in response?
Does the student respond to the problem identified by the educator? Does the student identify new problems and opportunities for themselves?

 

Applied to entrepreneurial teaching: In creativity assessment multiple creative ideas generation (divergent thinking) is the focus, not the testing of ideas (convergent thinking). How well do learners adapt to changing scenarios? How many different ideas can they generate? How well do they adapt their solutions when things change? How diverse and different are their (multiple) solutions. Less creative people will have fewer ideas and demonstrate a tendency to rush to singular solutions, whereas entrepreneurial people see more unusual links and connections, hence more opportunities.




Constructive Alignment

Definition: Constructive Alignment is a principle stating that teachers should try to align (1) what the students need to do, in order to learn, with (2) what is being assessed.

Description: Constructive alignment is based on the relationship among (1) active learning, (2) intended learning outcomes and (3) assessment activities. The learner constructs meaning through the learning activities undertaken. Assessment should then align with these activities, so that the activities that students need to learn from are actually those being assessed. The learner is in a sense ‘trapped’ and finds it difficult to escape without doing those activities that need to be done in order to learn what is intended to be learned. Learning outcomes should be stated with a verb that is then also used for assessment tasks. Examples include: explain why a particular course topic is important, apply a course topic to your own life, reflect on a theory you have learned in the course, evaluate a situation that has gone wrong and apply a solution.

Benefits: Constructive alignment requires teachers to reflect upon what it means to understand a subject on a deep level, and how students can attain such a deeper understanding. By asking teachers to state those verbs that describe necessary learning activities for students, a gap between abstract theoretical knowledge and personal competent practice can be bridged. This increases consistency, improves educational effectiveness and minimises ineffective surface learning.

Challenges: Although the approach is common sense, it requires a different way of thinking about teaching, and in particular about defining levels of understanding. It requires a theory of teaching one’s discipline over and above knowledge of the discipline itself; and knowledge about options for teaching and assessment. The initial stages of setting up an aligned system require careful consideration and possibly also redesign of the curriculum. This can be a challenge in educational settings where planning time is a scarce resource, such as in compulsory education.

Applied to entrepreneurial education: Since entrepreneurial education is often recommended to be based on a learning-by-doing approach, constructive alignment can be used as a method to put more assessment focus on those activities that really foster entrepreneurial learning outcomes. Constructive alignment can be a way to get a more explicit activity-based assessment focus, thus facilitating experiential and entrepreneurial teaching and learning. Activities that have been shown to foster entrepreneurial competencies include interaction with the outside world, value creation for others, teamwork over long periods of time, uncertainty in learning environments, time pressure, presentation for others (preferably unknown), experiencing individual differences in a team and helping others.




Assessment of learning

Definition: Assessment of Learning has been defined as “summative assessment which is a success measure of the outcomes of the end of a unit, programme, year’s study, qualification or educational experience. It is almost always a formal process and can include teacher judgement as well as testing.”

Description: Assessment of learning measures the student’s learning at a set point in time against a set of standards and leads to the award of accredited qualifications such as certificates, diplomas, degrees, etc. It tends, in the main, to utilise a range of marks, grades and classifications rather than just ‘pass’ or ‘fail.’ Assessment of learning is mainly used to record achievement to allow progress to a higher level of study or to a higher position within a workplace or within a profession.

Benefits: The benefits of assessment of learning to the learner are that it requires teachers to use transparent approaches and to interpret the outcomes of learning in a standardised way so that students are able to demonstrate their knowledge, competence and skills level at a particular point in time and to be rewarded for that achievement. The rewards associated with assessment of learning provide defensible public reference points for making judgements about an educational approach or system as a whole and the process of assessment of learning provides strategies for recourse in the event of disagreement about end decisions.

Challenges: Although assessment of learning tends to be continuous and is therefore usually based solely on teacher judgement in vocational education, nationally accredited qualifications are externally set and tend to be based on tests and examinations designed to measure the learner’s achievement against strict criteria and standards. In relation to the latter, as suggested above, assessment of learning is used to monitor the educational achievement of all learners locally, regionally and nationally within all subjects at all levels. This accountability aspect of assessment of learning can result in unintended outcomes such as, for example, where teachers spend more time preparing learners for tests rather than teaching the full breadth and depth of the subject. In most countries, assessment of learning is highly regarded as being reliable and valid, and is controlled by external regulators and awarding bodies, with much less trust being placed on the academic judgement abilities of teachers. This can impact to some extent on the perception and attractiveness of teaching as a profession.

Applied to entrepreneurial education: As with any type of education, there are elements of entrepreneurial education which lend themselves to the assessment of learning approach, allowing learners to gain certificated qualifications that they can use to demonstrate their mastery in entrepreneurial knowledge and skills, supporting them in undertaking further study and self-development, for example, the Entrepreneurial Skills Pass (ESP). In addition, the EntreComp framework is based on 15 competences embedded within an 8-level progression model which encompasses 442 learning outcomes. From an assessment of learning perspective, the framework can be used as a basis for the development of curricula and learning activities fostering entrepreneurship as a competence and to assess learners’ entrepreneurial competences.




Assessment for learning

Definition: Assessment for Learning (AfL) can be defined as a process of gathering evidence on a learner’s progress which is then used to help both the learner and the teac her to recognise where the learner is in their learning, where they need to go and how best to get there.

Description: The term Assessment for Learning was popularised by Black and William (1998) in their work relating to the importance of student and teacher feedback in the learning process. It involves personalising learning through the inclusion of self-assessment by the learner in both formative and summative assessment activities. In classrooms where assessment for learning is practised, students are encouraged to be more active in their learning and associated assessment. The ultimate purpose of assessment for learning is to create self-regulated learners who can leave education able and confident to continue learning throughout their lives.

Benefits: Assessment for Learning (AfL) is an essential part of a learner’s education as it defines whether or not the objectives of teaching are being met. Assessment affects decisions about grades, educational needs of learners and in some cases funding. AfL is a significant way to raise a learner’s academic achievement and is centred on the belief that in order for learners to progress then they must understand the purpose of their learning, where they are in relation to that purpose and how they can achieve their goals. AfL helps a learner reflect on their own development which in turn helps them to recognise and appreciate their own strengths as well as developing an insight into themselves as a learner. If a learner is given the opportunity to discuss their learning either with a teacher or one of their peers then they will develop a deeper understanding of their learning which can build confidence and motivate them as students. Effective AfL identifies individual educational needs as well as informing the learner about their specific performance and achievements, allowing teachers to utilise approaches that are personalised to the needs of learners. AfL can be used not only to measure learning but also to promote learning by teaching learners how to ask questions as well as answering them, by emphasising to a learner that it is acceptable to ‘Have a go’ and that giving the wrong answer it is still learning.

Challenges: The main challenge associated with AfL is that teachers and leaders often need to value the change in philosophy and pedagogy and the effort required to effectively embed the approach. It is a shift in the balance of teachers’ skills and requires planning which does not depend on a fixed scheme of work and standardised testing. The learning has to flex and bend to meet the needs of the learners where there is no ‘one size fits all’. In addition, time must be given for reflection, which means less ‘coverage’ but deeper learning. Learners themselves often find the transition to AfL difficult as the urge to ask “What grade did I get?” is so strong and deeply set in their minds.

Applied to entrepreneurial education: Assessment for learning is suitable in entrepreneurial learning as it recognises and values the development of learning and skills outside of the core subject area. It provides learners with the opportunity to discuss strategy, to define goals, to talk about strengths and weaknesses and then, following the learning experience, to analyse what went well and what went not so well and to be open about the failures in order to learn. These learning conversations help to nudge the learner back on course, to change habits, to think about things differently (metacognitive conflict) and to focus on making the small learning behaviour changes that can have a huge impact.




Performance assessment

Definition: Let students perform meaningful and hands-on real-life tasks and assess them based on their task accomplishment1.

Description: Performance assessment is about assessing students based on their active hands-on demonstration of knowledge and skills. This should preferrably take place in authentic settings. The approach is different from traditional assessment where students talk or write about what they have learned2, see table. Performance assessment is common in arts education (dance, music, arts, crafts, etc) and in vocational education (construction, plumbing, hairdressing, vehicle repairs, etc). Performance assessment should be designed so that the relevant real-life performance is emulated as realistically as possible3. Tasks could be designed in accordance with constructive alignment principles; what do the students need to do in order to reach desired learning outcomes?

Benefits: Performance assessment has been described as a way to assess and foster more complex and higher-order learning. It requires students to be active, and to apply critical thinking and problem-solving skills. This leads to more motivated and engaged students and improved learning. It is also a more inclusive assessment approach for students struggling with written exams.

Challenges: Performance assessment has shown to be difficult to design and time-consuming to deploy. It is also often difficult to discriminate between poor, average and excellent performance. Teachers have also found performance assessment to be subjective, especially compared to the objectivity of determining whether a reply is simply right or wrong.

Applied to entrepreneurial teaching: Entrepreneurial teaching allows for a large number of learning activities that can be formulated as hands-on tasks that students could learn from doing. Examples include developing an idea for a new business through a workshop format, presenting a business idea to an external audience, asking external stakeholders for commitment to take part in developing the idea and making a budget for required resources needed to realize the idea. If the entrepreneurial teaching leans on a broad definition of entrepreneurship viewed as not only starting a business, other tasks could be relied upon. Students could be asked to develop answers to the question “For whom could this knowledge be valuable?”, contact external stakeholders to verify a hypothesis about how valuable something might be, or simply educate the general public on an issue of importance to society. Generic kinds of performance tasks include interacting with external stakeholders, helping others, presenting something to others and working in teams to create something of value to an external stakeholder based on curriculum content.