This project has been funded with support from the European Commission. This publication reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein.

Examiner training in Slovenia

 

From October 19th till October 21st  Biotehniški izobraževalni center Ljubljana, Gimazija in veterinarska šola from Slovenia held examiner training for 9 participants from Slovenia, 2 from Denmark and 1 from Belgium. Training was led by Julie Dugmore and Vicky Hedges from UK, the official trainers for exams according to RCVS OSCE system. We started with short a review of the Practical examiner training pack and solved some problems that occurred mostly due to language barrier. The role of the OSCE examiner was introduced, followed by the introduction of OSCE mark sheets, video and live marking exercise. In the afternoon we had to confront with first practical challenge: 

designed for testing tasks and examining real students. During the day each of participants was examiner of two different tasks. We ended the second day with task evaluation; we discussed which tasks should be improved and how to upgrade the whole examination process.
On the last day we concentrated on setting the pass mark and the essence of task writing. However, most of the time was dedicated to meant for or improving tasks or developing new ones, which were introduced, videoed and discussed.
Julie Dugmore and Vicky Hedges were energetic and patient during the whole examiner training and were able to answer our questions and doubts with practical examples.
We were working hard but still had some time for socializing and fun, so training was really a success.


Laura Spik   

 

Before training all the participants had studied Practical examiner training pack and then completed the questions that were checked by RCVS before the training began.
Training had more than one purpose:
-to become familiar with the role of the OSCE examiner, mark sheets, task stations, examiner and student briefing,
-to try the tasks and marking
-to review RCVS OSCE tasks,
-to develop new tasks.

OSCE task stations had to be set up. We set up 10 different tasks, some of them had been already tested some of them not. After the examiner briefing and demonstration of student briefing half of the participants played the role of students and the other half were official examiners. In that way we tested tasks and had the first practical training that would help us to become reliable examiners.
The second day was 		  

Sustainability of the Pepas project

Last December 2009 I visited a workshop organised by the Leonardo agencies of the Netherlands and Belgium. It was fun to meet people involved in all kind of Leonardo projects and talk about parallels and differences of our projects.
The main aim of this workshop was to learn something about sustainability. This is a very important item and leading in the agenda of our next conference in Mallorca!
What is meant by ‘sustainability’? There have been a lot of ‘technical projects’ sponsored by the Leonardo agency. The agency does not want to spent more money on this kind of projects, but is more interested in projects witch have an outcome for the long-term. Therefore the project members should look for possibilities what can be done with the outcome in the long term.
Sustainability of a project should generate ‘impact’; witch is defined as ‘direct and indirect long-term effects on actors, structures, sectors or systems beyond the original project environment’. Sustainability means that crucial activities and results of the project are maintained and continue to deliver benefits to the target group, structure, sector or system after the end of the EU funding.
So the implementation of European cooperation projects should lead to generating added-value in vocational education and training in Europe. Though sustainability 

may not concern all the aspects of a project, in each project some activities or results should be maintained.
The EU developed a check list based on successful projects. The check list is divided in two parts; one part is about the project level factors and the other part is about the context level factors.
In the project plan is described what the target is of the project. 


   


And there is also described who the target groups may be. In terms of dissemination or sustainability you can look for your target groups on micro level, e.g. project partners, own organisation, students, etc. On meso level you can think of educational umbrella organisations, chambers of commerce, employment agencies. And, last but not least, on macro level you can look for policy makers, European branch organisations, European commission, etc. So maybe for our Pepas project the focus not necessarily should be on our target group (like vet nurse schools or students) but on decision makers (like the aoc-raad

in the Netherlands)?
I also learned about the difference you can make in choosing the dissemination channels. These are also divided in a micro, meso and macro level. Micro channels can be found in your personal network/contacts, via meeting, etc. Meso channels are for instance local or regional media, (direct) mailing, courses, etc. And macro channels can be found in conferences, publications, internet, etc.
When we might discuss the exploitation of results and materials of our project in Mallorca, we can think of the following;
1. We should define the selling points. What are the strong points, how can we show how attractive our product is.
2. Show the transferability of the project results. Are the results/products easily transferable to other sectors/target groups?
3. Sell it! What is the best strategy? Offer it for free? Give guiding training/courses? Is selling possible?
I’m positive we will have fruitful discussions on our next conference in March and that we will find a suitable outcome for our ‘dissemination plan’. So we can make sure our Pepas project will be seen as a successful long-term sustainable project!
Looking forward meeting you all very soon, best wishes,

 Annemarieke de Man 

 

 Examinertraining by RCVS 
The Royal College of Veterinary Surgeons (RCVS) in the United Kingdom became partners in the PEPAS project with the specific remit of delivering training to the other participating partners on how to examine practical skills using an Objectively Scored Clinical Examination (OSCE) format.
The PEPAS project was set up to establish a consistent practical assessment system which will be suitable for use across ACOVENE accredited schools. The aim was to develop a bank of a minimum of 100 practical tasks testing the ACOVENE Dossier of European Competencies for Veterinary Nurses. During the planning phase of the project a number of the European schools compared their existing veterinary nursing practical assessment methods. In the UK, the RCVS had delivered practical examinations for over 20 years and, since 2005, these examinations have been held using the OSCE method. It was agreed that the assessment system used in the UK was reliable, objective and transparent. Because OSCE examinations have been used across a wide variety of professional examinations for the last 40 years it was evident that the format could easily be adapted to the Dossier of European Competencies. In addition, the RCVS had already developed a robust examiner training programme and had trained over 100 examiners, including veterinary nursing tutors from overseas. This examiner training was used as a model for the PEPAS examiner training.
All participating delegates completed a pre-training home study pack to familiarise themselves with the OSCE examination system 		and the role of the examiner. The training was delivered in each school over three days. An extra fourth day was added in Denmark to cover the quality assurance of the examinations. Some of the delegates would also be responsible for training staff in their schools so additional sessions were held to assist them with this task. In total 32 veterinary nurse educators were trained. 
Day one of the training consisted of an overview of the examiner role in the OSCE examination. The focus was on room management, test management, irregularities in the examination and examiner conduct. Delegates had practice at marking a pre-filmed DVD and a live station.
During day two, examiners examined real students supplied by the schools. Each examiner gained experience in examining at least two different OSCE stations for a minimum of 16 students. A total of 50 students were actively involved with the training. There was the opportunity to discuss the tasks and suggest adaptations. It was quickly realised that veterinary nursing practice differs between each country. Even the different equipment available would mean a rewrite of the task was required.
Day three was OSCE station writing day. The examiners had the opportunity to start developing new tasks to add to the PEPAS task bank. The biggest challenge was being able to find reference material to demonstrate best practice and also to make each task generic and not country-specific. This proved a bit of a challenge because there are 

many acceptable methods to achieve some of the tasks. This raised the question that, if all schools were going to use the examination, would it be acceptable to have many tasks examining the same outcome? This remains a challenge.
A selection of new tasks was videoed. At the end of this final day the delegates were instructed to test and refine the tasks prior to them being published.
In January, the training began in Denmark hosted by Hansenberg where the training was delivered to Danish, Italian, Norwegian and Swedish veterinary nurse tutors. At this stage there were no tested OSCE tasks in the PEPAS bank and with a target of 100 by the end of the project this seemed an enormous task. For this first

 group of training examiners the existing RCVS OSCE tasks were used. Some of the tasks did not readily lend themselves to the equipment used in the different counties and therefore some tasks were adapted so that they could be used. All of the delegates came to the training with ideas of tasks which could be developed and the final day of the training was spent writing and developing new tasks. These included:

  • Intravenous cannulation
  • Ear swab
  • Faecal flotation
  • Maintaining the operating theatre 

 

In April it was the turn of the University College Dublin to host the training in Ireland, with delegates from their own school and Groenhorst College Barneveld in the Netherlands. On this occasion we were able to use some of the tasks previously developed in Denmark as well as other tasks adapted from the practical examinations already held at UCD. By the end of this training session there were a number tasks tried and tested with a number of students and the task bank began to grow. 
All the partners met in Holland for a mid-term conference in June 2009 to discuss progress so far. We were to hear Maurizio Albano from Abivet in Rome talk about the training he had delivered to his staff. Maurizio reported that he had used the OSCE examination method for his last veterinary nurse final examinations. This had been quality assured by an external examiner who reported back that the examinations had gone well. During this conference we were able to demonstrate through video the development of new tasks and the importance of detailed examiner instructions, equipments and appropriate candidate scenarios. We were pleased to see that many new tasks had been written and that the target of 50 tasks by September was within reach.
The final examiner training was hosted by Biotehniski Izobrazevalni Center Ljubljana, Slovenia were the local tutors were joined by further Danish delegates and one delegate from Belgium. All tasks used on

this occasion had been developed or adapted for the PEPAS project. No RCVS tasks were used. 
A major challenge was that many of the tasks had little or no examiner instructions and the delegates looked to us for information on how they should  be used. As we had not written the tasks it was very difficult to guide them. Some delegates were frustrated that candidates had made dangerous errors but could still score 100% in the test. Two examples included a student who, when intubating the patient, placed the ET tube the wrong way round. Another example was a candidate who dropped the cap of the milk sampling tube onto the floor and put it back on the tube therefore potentially contaminating the sample. This highlights the importance of piloting and refining the tasks with real students who perform naturally rather than experienced

 

 tutors who generally do not make mistakes. Further adaptations were made to all twelve tasks in order that they could be reliably used in the examinations.
Bogdan Zdovc and his team excelled themselves by developing realistic usable models including udders, and a dog which could be 

used for endotracheal placement. If you looked closely you found the detergent bottle and sticky-backed plastics. 
On day two of the Slovenian

 

 training we were visited by a contingent from the Ministry of Education along with senior staff from the school. They spent considerable time observing the examination system and spoke to both examiners and students. The OSCE system was well received and it was reported that they will consider approving this examination style as part of the final examination for the veterinary nurse programme at this school.
The project has achieved a lot during the practical examiner training phase, including the development of OSCE examination tasks. At least one school has used the system for its final examinations and a pool of trained examiners has been established.
The project now enters its development phase where further tasks are being written, translated and a standard examination system piloted. We wish luck to all the schools during this phase and hope that all the hard work pays off.


Victoria Hedges & Julie Dugmore