Change search
Refine search result
1 - 20 of 20
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Rows per page
  • 5
  • 10
  • 20
  • 50
  • 100
  • 250
Sort
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
Select
The maximal number of hits you can export is 250. When you want to export more records please use the Create feeds function.
  • 1.
    Alger, Susanne
    Umeå University, Faculty of Social Sciences, Department of applied educational science.
    Is This Reliable Enough?: Examining Classification Consistency and Accuracy in a Criterion-Referenced Test2016In: International journal of assessment tools in education, ISSN 2148-7456, Vol. 3, no 2, p. 137-150Article in journal (Refereed)
    Abstract [en]

    One important step for assessing the quality of a test is to examine the reliability of test score interpretation. Which aspect of reliability is the most relevant depends on what type of test it is and how the scores are to be used. For criterion-referenced tests, and in particular certification tests, where students are classified into performance categories, primary focus need not be on the size of error but on the impact of this error on classification. This impact can be described in terms of classification consistency and classification accuracy. In this article selected methods from classical test theory for estimating classification consistency and classification accuracy were applied to the theory part of the Swedish driving licence test, a high-stakes criterion-referenced test which is rarely studied in terms of reliability of classification. The results for this particular test indicated a level of classification consistency that falls slightly short of the recommended level which is why lengthening the test should be considered. More evidence should also be gathered as to whether the placement of the cut-off score is appropriate since this has implications for the validity of classifications.

  • 2.
    Alger, Susanne
    Umeå University, Faculty of Social Sciences, Department of applied educational science, Departement of Educational Measurement.
    Licence to drive: the importance of reliability for the validity of the Swedish driving licence test2019Licentiate thesis, comprehensive summary (Other academic)
    Abstract [en]

    Background: The Swedish driving licence test is a criterion-referenced test resulting in a pass or fail. It currently consists of two parts - a theory test with 65 multiple-choice items and a practical driving test where at least 25 minutes are spent driving in traffic. It is a high-stakes test in the sense that the results are used to determine whether the test-taker should be allowed to drive a car without supervision. As the only other requirements for obtaining a licence is a few hours of hazard education (and a short introduction if you intend to drive with a lay instructor) it is important that the test result, in terms of pass or fail, is reliable and valid. If this is not the case it could have detrimental effects on traffic safety. Examining all relevant aspects is beyond the scope of this licentiate thesis so I have focused on reliability.

    Methods Reliability for both the theoretical and practical test results was examined. As these are very different types of tests the types of reliability examined also differed. In order to examine inter-rater reliability of the driving test 83 examiners were accompanied by one of five selected supervising examiners for a day of tests. All in all 535 tests were conducted with two examiners assessing the same performance. At the end of the day the examiners compared notes and tried to determine the reason for any inconsistencies. Both examiners and students also filled in questionnaires with questions about background and preparation. As for studying decision consistency and decision accuracy of the theory test, three test versions (a total of around 12,000 tests) were examined with the help of methods devised by Subkoviak (Subkoviak, 1976, 1988) and Hanson & Brennan (Brennan, 2004; Hanson & Brennan, 1990).

    Results The results from two research studies concerning reliability were presented. Study I focused on inter-rater reliability in the driving test and in 93 per cent of cases the examiners made the same assessment. For the tests where their opinions differed there was no correlation to any of the background variables or other variables examined except for three, which had logical explanations and did not constitute a problem. Although there were cases where the differences were due to different stances on matters of interpretation the most common suggested cause was the placement in the car (back seat vs. front seat). Although the supervising examiners gave both praise and criticism as to how the test was carried out the study does not answer the question whether the tests were equal in terms of composition and difficulty.

    In Study II the focus was on decision consistency and decision accuracy in the theory test. Three versions of the theory tests were examined and, on the whole, found to be fairly similar in terms of item difficulty and score distribution, but the mean was so close to the cut-score (i.e. the score required to pass) that the pass rate differed somewhat between versions. Agreement coefficients were around .80 for all test versions (between .79 and .82 depending on method). Classification accuracy indicated an .87 probability of a correct classification.

    Conclusion It is important to examine the reliability and validity of the driving licence test since a misclassification can have serious consequences in terms of traffic safety. In the studies included here the rate of agreement between examiners is deemed as satisfactory. It would be preferable if the classification consistency and classification accuracy, as estimated by the methods used, were higher for the theory test, given its importance.

    While reliability in terms of agreement between raters/examiners or consistency and accuracy of classification are routinely examined in other contexts, such as large-scale educational testing, this is not often done for the driving licence tests. At the same time, the methods used here can be transferred to contexts where such properties are generally not examined. Collecting information about test-takers and examiners, like in Study I, can provide evidence concerning possible bias.

    Examining to what extent decisions are consistent is one important aspect of collecting evidence that shows that test results can be used to draw conclusions about driver competence. Still, regardless of outcome, validation is a process that never ends. There is always reason to examine various aspects and make further improvements. There are also many other relevant aspects to examine. A prerequisite for the validity of the score interpretation of a criterion-referenced test like this one is that the cut-score is appropriate and the content relevant. This should therefore be the subject of further research as the validation process continues.

  • 3.
    Alger, Susanne
    Umeå University, Faculty of Social Sciences, Department of applied educational science, Departement of Educational Measurement.
    Prov för förarbevis för moped klass II: Enkätstudie avseende provkonstruktion och provgenomförande2012Report (Other academic)
    Abstract [en]

    In 2009 a requirement to obtain a driver’s certificate was introduced for riding a class II moped. In order to obtain a driver’s certificate test-takers are required to complete a compulsory course and pass a test. In this report the second part of an evaluation of testing practices for the driver’s certificate for moped class II is presented. In the first part of the project the content and quality of a number of tests was examined.

    This second part of the project focuses on the development or selection of tests and the administration of the tests. After an interview guide and a web questionnaire were designed 33 test administrators were asked to fill in a questionnaire or take part in a telephone interview. 22 test administrators filled in the questionnaire. Another two test constructors were interviewed.

    All  the test administrators we reached still use the test enclosed with their application for administering tests. Fifteen of them use the test constructed by the Swedish National Association of Driving Schools (STR) and are pretty happy with that. The items in STR’ s test are constructed by a group who meet up a couple of times per year to develop and review items which are then placed in a data base, marked with the appropriate driving licence category. Later on, the items are reviewed further. Since class II was an additional category there was no such marking in the data base so an employee had to manually search for and pick out suitable items for the tests produced in 2009.

    Most test administrators who used self-constructed tests stated that they had chosen to do so since there were no other tests available at the time. Some stated that it suited them best to design the test themselves. The descriptions of the test development process are not particularly detailed, but often mention using the requirements from the Swedish Transport Agency as a starting point.

    STR’s tests are, with one exception, administered via computer, while the other tests are pen-and-paper tests. Often the tests are administered to one test-taker at a time, and nobody administers them to groups of more than 16 people. Test administration seems to adhere to the rules, but four test administrators claim that there is only one version of the test (should be at least 2).

    Some of the test administrators examine what items the test-takers have problems with in order to adapt their teaching or the test. It is common to require failed test-takers to study more at home before retaking the test, but most test-takers pass at their first attempt.

  • 4.
    Alger, Susanne
    et al.
    Umeå University, Faculty of Social Sciences, Department of applied educational science, Departement of Educational Measurement.
    Eklöf, Hanna
    Umeå University, Faculty of Social Sciences, Department of applied educational science, Departement of Educational Measurement.
    Det praktiska körprovet 2009-2011: analys av datas tillförlitlighet samt provresultat och trender2013Report (Other academic)
    Abstract [en]

    The aim of this report is to examine the reliability of data from the practical driving test for category B 2009-2011 and study the composition of the group of test-takers and results and changes in pass rates over time during the period. Based on the data checks done it can be concluded that data is reliable. Data from 2011 does not comprise the entire year, but when data for the equivalent period previous years is compared with the year as a whole the differences are negligible.

    There are certain differences between groups as to the proportion of passed tests. A larger proportion of test-takers below the age of 20 pass the test compared to older test-takers. A larger proportion of test-takers who were registered by a driving school passed than those who registered themselves. If you divide the test-takers by gender and type of registration a somewhat larger percentage of the men passed the driving test. As for average number of mistakes there are no major differences between the sexes except for manoeuvring.

    As for the content of the test, the most common areas to test are street crossing, driving towards a goal and reversing. Motorway, lanes, roundabouts and signalled crossings are more commonly tested at the main offices, while narrow winding roads, railway crossing and turning off a main road are more common in the other places. Street crossing, lanes and motorway are the situations where the largest percentage of test-takers fails. The percentage of fails for each curricular module are fairly similar for main offices and other offices, but the proportion of fails is higher for roundabouts, driving towards a goal and railway crossings at the main offices. The requirement for attention and identifying risks is the hardest for test-takers to fulfil.

    When comparing the results with previous years we find that the group differences are fairly similar, but the percentage of tests registered via a driving school continues to decrease. It is more common to include reversing in the test. Even though test-takers who has failed the theory tests are now allowed to take the practical driving test the percentage of passed tests remains on the same level for tests registered via driving schools and has only decreased by a couple of percentage points for the remaining tests.

  • 5.
    Alger, Susanne
    et al.
    Umeå University, Faculty of Social Sciences, Department of applied educational science, Departement of Educational Measurement.
    Eklöf, Hanna
    Umeå University, Faculty of Social Sciences, Department of applied educational science, Departement of Educational Measurement.
    Jämförelser mellan provorter: en studie av körprov för behörighet B2016Report (Other academic)
    Abstract [en]

    There are more than 130 places in Sweden where driving tests can be carried out, with their own particular traffic environments, examiners and test-takers. The test-takers' ages and their preferred method of registering for the test vary between places. In some places only a few tests are carried out and in some a large number of tests. In some places certain aspects of the traffic environment are absent, like motorways, railway crossings or roundabouts. The intensity of traffic also varies between different locations. In short, there are many aspects that can vary between individual tests. In this report various ways of presenting such differences and some ways of categorizing places are explored.

    Does this have consequences for results? It is difficult to analyse as several variables probably interact and are hard to untangle and due to the nature of the data. The result is binary – pass/fail – which limits what analyses can be made – and there are many categories in the variables place and examiner. In order to achieve fewer categories some can be collapsed into fewer and larger categories. The question is then how many categories and what criteria should be used to form them. When analyses are made we can see that the content of tests differ both within a particular location and between different places. It is, however, unclear if the same or different skills and abilities are measured in different places, but nevertheless not as simple as to claim that it is "easier" to obtain a licence in certain places than others based on simple comparisons of success rates. Test-takers' age and method of registration for the test, aspects that we know are connected to the percentage of passed tests, also differ between places.

  • 6.
    Alger, Susanne
    et al.
    Umeå University, Faculty of Social Sciences, Department of applied educational science, Departement of Educational Measurement.
    Eklöf, Hanna
    Umeå University, Faculty of Social Sciences, Department of applied educational science, Departement of Educational Measurement.
    Samma prov - andra provtagare: en studie av förändringar i körprovsresultat och provtagargruppens sammansättning2017Report (Other academic)
    Abstract [en]

    The purpose of this study is to describe the trend regarding the pass rate for the driving test and examine to what degree changes of the pass rate can be linked to changes in the composition of the group of test-takers. Previously there have been no trend studies exploring changes with regard to the language of the theory test, i.e. whether the test-takers take the test in Swedish or another language so special attention will be given to this aspect.

    The composition of the group of test-takers has changed quite a lot over the last 20 years. Test-takers can either register for the test via a driving school or on their own. The percentage of tests registered via a driving school has decreased, and in later years the number of driving tests not registered via a driving school following theory tests in other languages than Swedish has increased. The pass rate among the tests that were not registered via a driving school has decreased more than for those registered via a driving school. If the results are weighted to reflect the distribution of groups as it was in 1998 or 2010 one may conclude that if the total group of test-takers had had the same composition in later years the pass rate would probably not have decreased as much.

    If theory tests are categorised according to whether they were carried out in Swedish or another language (translated or interpreted) results show that the percentage of tests in other languages has increased significantly since 2013. The percentage of driving tests following theory tests in other languages has also increased. Most of the driving tests after theory tests in other languages are carried out by test-takers who are 25 years or older and who have not registered for the test via a driving school. Such tests are also failed to a greater degree.

    Many more tests are administered today than 10 or 20 years ago. The number of test-takers increased by 87 per cent between 1998 and 2016 while the number of tests increased by 137 per cent. A contributing factor is the change in 2008 that all were allowed to take the driving test regardless of result on the theory test (previously only those who passed could). If results on the driving test are divided by result on the preceding theory test (pass/fail) it turns out that the pass rate has decreased at roughly the same rate since 2012 for both those who failed and those who passed the theory test. However, a larger proportion of those who have passed the theory test also pass the driving test.

    The changes in conditions for taking the driving test does not constitute the whole explanation. If comparing the number of driving tests in 2010 (when all were part of the new system) and 2016 the increase is 22 and 33 per cent respectively.

    When performing analyses based on test-takers rather than tests it is also clear that the number of test-takers who are 25 or older and not registered via a driving school is increasing. At least from 2013 this increase is almost entirely due to test-takers who have taken the theory test in other languages than Swedish.  In 1998 24 per cent of the test-takers only took tests for which they had not registered via a driving school. In 2010 that percentage was 42 per cent an in 2016 53 per cent. Students who do not register for the test via a driving school also carry out more tests on average within the year than those who do.

    Test-takers who carry out many driving tests have, to a greater extent than others, had the examiner intervene (e.g. by braking) during their first test attempt, which perhaps is a consequence of not being properly prepared.

    To summarize: the trend with decreasing pass rate, an increasing proportion of older test-takers and of tests not registered via a driving school that has been visible over the last 20 years or so seems to be continuing. When looking at trends in a shorter perspective, the most notable change is the increase in theory tests that are taken in languages other than Swedish. These tests are, to a great extent, taken by older test-takers, not registered via a driving school, factors that are associated with a lower pass rate. Findings may inform policy makers and other stake-holders when it comes to understanding changes in the composition of test-takers and how possible efforts to increase pass rates could be targeted.

  • 7.
    Alger, Susanne
    et al.
    Umeå University, Faculty of Social Sciences, Department of applied educational science, Departement of Educational Measurement.
    Eklöf, Hanna
    Umeå University, Faculty of Social Sciences, Department of applied educational science, Departement of Educational Measurement.
    Sämre  år för år?: En studie av förändringen av andelen godkända på körprov för körkortsbehörighet B2012Report (Other academic)
    Abstract [sv]

    Vägverket har samlat in en mängd data i samband med körproven under många år. Ett av de mått som presenterats är andel godkända prov och denna andel har minskat under perioden 1998-2008. I denna rapport diskuteras faktorer som kan påverka resultatet och hur detta kan tolkas. Som bakgrund presenteras några generella frågeställningar som gäller mätningar följt av en beskrivning av förändringar som rör förarprovet.

    Syftet med denna studie är att undersöka i vilken mån förändringar av andelen godkända på körprovet kan kopplas till förändringar i provtagar­gruppens sammansättning. Som underlag har använts de data som sam­lats in i samband med körprovet.

    För att undersöka om kunskapsnivån eller gruppens sammansättning förändrats mest har resultaten delats upp utifrån sexton delgrupper. Indelningsgrunderna har varit ålder, kön och anmälningssätt. Inom respektive grupp har resultaten varit tämligen stabila över tid. Det är bland de privatanmälda eleverna man kan se en nedåtgående trend. Samtidigt har andelen privatanmälda ökat. Om man viktar resultaten för att återspegla den fördelning mellan grupper som rådde 1998 kan man konstatera att om provtagargruppen hade haft samma sammansättning under de följande åren skulle andelen godkända totalt ha varit högre under dessa år än vad som nu är fallet.

  • 8.
    Alger, Susanne
    et al.
    Umeå University, Faculty of Social Sciences, Department of applied educational science, Departement of Educational Measurement.
    Eklöf, Hanna
    Umeå University, Faculty of Social Sciences, Department of applied educational science, Departement of Educational Measurement.
    Trender 1998-2015 avseende förändringen av andelen godkända på körprov för körprovsbehörighet B2016Report (Other academic)
    Abstract [en]

    The Swedish Road Administration has, over the years, gathered a lot of data in connection with the driving test. One of the statistics used is the percentage of passed tests and this number has decreased since 1998. In a previous report factors which can affect the result and possible interpretations of results were discussed in terms of to what degree changes in the percentage of passed tests were connected to changes in the composition of the group of test-takers. This is a follow-up of that study with more recent data which includes analyses of the composition of test-takers both based on individuals and on tests.

    Many results are presented in terms of 16 categories based on test-takers’ age, sex and how they were registered for the test. Within each category the results are comparatively stable over time. It is among test-takers who did not register for the test via a driving school that you can discern a downward trend regarding the percentage of passed tests. The percentage of such test-takers in the group as a whole has increased.

    In 2008 a major change was introduced as all who took the theory part of the test also took the driving test. Previously this was restricted to those who passed the theory test. Such a change affects the percentage of passed tests which got slightly higher for driving tests preceded by a passed theory test and slightly lower for driving tests preceded by failed theory tests.

    If results are weighted to reflect the distribution of groups as it was in 1998 one may conclude that if the total group of test-takers had had the same composition as 1998 over the years that followed the total percentage of tests resulting in a pass would have been greater than what is now the case.

  • 9.
    Alger, Susanne
    et al.
    Umeå University, Faculty of Social Sciences, Department of applied educational science, Departement of Educational Measurement.
    Eklöf, Hanna
    Umeå University, Faculty of Social Sciences, Department of applied educational science, Departement of Educational Measurement.
    Uppföljning av sammanhållet förarprov: Utvärdering av effekter av ett sammanhållet teoretiskt och praktiskt förarprov mellan 2007 och 20132014Report (Other academic)
    Abstract [en]

    On September 1, 2008 a new test model for the Swedish driving licence test was introduced. In the new test model the theory test and the driving test are viewed as integrated and they are booked at the same time. First the test-taker takes the theory test and then, on the same day or within a short time period, he or she takes the driving test. The driving test shall always be performed, irrespective of the result on the theory test. In order to obtain a driving licence the test-taker must pass both the theory test and the driving test within a two-month period from the point when the first test is passed – otherwise new fees must be paid and both tests must be retaken. Previously only the test-takers who had passed the theory test were allowed to take the driving test and the result on the theory test was valid for a year.

    After the new model was introduced a study was conducted to examine whether the change had resulted in an improved pass-rate for the two parts of the test – the theory test and the driving test – and whether the integration of the test parts had improved. The results were presented in a report in 2010 (Alger, Henriksson, & Wänglund, 2010).

    In order to examine whether the initial positive effects are lasting a follow-up study has now been conducted where data from the period before the change (2007) are compared with samples from the corresponding periods in later years.

    As for the theory test a somewhat higher percentage of the tests result in a pass after the introduction of the new test model. However, the major difference between 2007 and later years is the percentage of tests registered via a driving school. The number of tests registered via driving schools is significantly higher after the change and the test-takers registered for the test in this manner are more likely to pass the test than those who registered themselves directly. The mean score has increased for almost all competence areas, but the differences are small.

    When it comes to the driving test the results of tests carried out between December 2007 and the end of February 2008 were compared with the results of the driving tests carried out during the corresponding period in later years. If all tests are included, i.e. also those following on a failed theory test, 59 per cent of the tests were passed in 2008/09 and 56 per cent in 2011/12. In order to make a fair comparison the results of the driving tests preceded by passed theory tests were compared. In the old system, where everybody who took the driving test first had passed the theory test 59 per cent of the driving tests resulted in a pass. Out of those who had passed the theory tests in the new system 66 per cent passed the driving test 2008/09 and 62 per cent 2011/12.

    The analyses also showed that there was a connection between results on the theory test and the driving test as test-takers with a high score on the theory test were more likely to pass the driving test.

    In summary the downward trend for the percentage of passed theory tests that was apparent before the introduction of the new model seems to have been broken. There was a marked increase in the percentage of tests registered via a driving school in connection with the change. The total percentage of passed theory tests has increased somewhat after the introduction of the new test model and the mean score has increased even though the difference is rather small. When it comes to the driving test the percentage of passed tests has increased after the introduction of the new system if you compare the driving tests preceded by passed theory tests. If the driving tests after failed theory tests are also included the percentage of passed tests has decreased, albeit perhaps not as much as could be feared. All in all more tests are carried out in the new system, but the average number of tests per passed test has only increased marginally. From the data we have access to, only a few test-takers have passed the same type of test more than once as a consequence of failing to pass both the theory test and the driving test within a two-month period. In closure the new system has not brought about any major changes (apart from the substantial increase of theory tests registered via a driving school). Even though we cannot know what the situation would have been without the introduction of the new test model, it seems like the changes carried out have had an effect in breaking the clearly negative trends prevailing before the change.

  • 10.
    Alger, Susanne
    et al.
    Umeå University, Faculty of Social Sciences, Department of applied educational science, Departement of Educational Measurement.
    Henriksson, Widar
    Umeå University, Faculty of Social Sciences, Department of Educational Measurement.
    Eklöf, Hanna
    Umeå University, Faculty of Social Sciences, Department of applied educational science, Departement of Educational Measurement.
    Prov för förarbevis för moped klass II2012Report (Other academic)
    Abstract [en]

    In October 2009 a requirement to obtain a driver’s certificate was introduced for riding a moped class II. In order to obtain a driver’s certificate test-takers are required to complete a compulsory course and pass a test. Those who applied for permission to arrange tests for moped class II had to submit an example of a test to the Swedish Transport Agency with their applications. We have studied a sample of those tests. The content of the tests were analysed in order to determine to what extent the stipulated content of the regulations was covered. Compliance with general guidelines for item construction and whether the regulations concerning number of questions were met was also examined.

    Among the 34 selected examiners 11 had developed their own test. The remaining examiners stated that they used the test developed by the Swedish National Association of Driving Schools (STR). Regulations for the test (TSFS 2010:80) stipulate that it should consist of 25 to 35 items. All tests in the study contained at least 25 items. With a couple of exceptions there are items from all four content areas outlined in the regulations. The main emphasis in the tests is on Traffic rules. The goals stipulated for the training and the goals stipulated for the test do not correspond entirely which probably is the reason why many items contain content not defined in the requirements for the test. However, most of such items can be linked to goals for the training.

    As to shortcomings in the test some items could for example have been phrased more clearly and contained more information. Some response options deviate from the others or are just too unlikely which unfortunately makes the number of selectable options smaller.

    Both content and design of the tests vary considerably why it would be best to let the Swedish Transport Agency develop the test for the conditional driving licence for mopeds class II. The test would be more uniform and equivalent, which would mean a fairer testing. It would be possible to strive for parallel tests. It would also be easier to supervise the results and the quality of the tests than is possible with the current system of supervision.

  • 11.
    Alger, Susanne
    et al.
    Umeå University, Faculty of Social Sciences, Department of Educational Measurement.
    Henriksson, Widar
    Umeå University, Faculty of Social Sciences, Department of Educational Measurement.
    Sundström, Anna
    Umeå University, Faculty of Social Sciences, Department of Educational Measurement.
    Likvärdigheten i körprovet: en studie av samstämmighet i bedömningen2009Report (Other academic)
    Abstract [en]

    As the consistency of assessment of the practical driving test has been called into question by the Swedish National Audit Office the purpose of this study was to examine the consistency of assessment of test-takers’ performance on the driving test. In order to examine inter-rater reliability the study was designed so that two raters assessed the same test-taker. The assessment was done on a two-step rating scale (pass/fail). Since the result can be affected by factors associated with the test-taker and the two raters, questionnaires were developed and these were filled in by the test-takers and the raters. Information about the administration of the test was collected via a specially designed form filled in by the second rater. The ordinary examiners’ performance was also rated by the supervising raters on a number of aspects. The result from the study indicated that the agreement between the assessments was very good. For 93 per cent of the tests the two raters chose the same mark on the two-grade scale. In the cases where ratings differed the analyses indicated only a few systematic differences among variables designed to provide possible explanations for differences in opinion. However none of these was problematic with respect to consistency of assessment. Results indicated that most tests were carried out in a satisfactory manner. All in all, the results indicate high consistency of assessment, both from a national and international perspective.

  • 12.
    Alger, Susanne
    et al.
    Umeå University, Faculty of Social Sciences, Department of applied educational science.
    Henriksson, Widar
    Umeå University, Faculty of Social Sciences, Department of applied educational science.
    Wänglund, Anna
    Umeå University, Faculty of Social Sciences, Department of applied educational science.
    Sammanhållet prov: Utvärdering av effekter av ett sammanhållet teoretiskt och praktiskt förarprov2010Report (Other academic)
    Abstract [en]

    On the 1st of September 2008 a new model for the Swedish driving-licence test was introduced. The new test model means that the theory test and the driving test are viewed as integrated and they are booked at the same time. First, the test-taker takes the theory test and then, on the same day or within a short time period, he or she takes the driving test. The driving test shall always be performed, irrespective of the result of the theory test. In order to obtain a driving licence the test-taker must pass both the theory test and the driving test within a two-month period from the point when the first test is passed – otherwise new fees must be paid and both tests must be retaken.

    The aim of this study was to examine whether the introduction of the new test model for licence category B has resulted in an improved pass-rate for the two parts of the test – the theory test and the driving test – and whether the integration of the test parts has improved.

    The study was conducted by comparing data from the driving-licence test before and after the new test model was introduced. The results on the theory test were examined by comparing results on the theory test from week 50 in 2007 with results from week 50 in 2008. The results indicated that the percentage of passed tests increased from 51 to 62. The average score on the test increased from 50.4 points in 2007 to 52.6 points in 2008. Thus, the average score in 2008 is above the cut-score for pass, which is 52 (out of 65 points).

    The results on the driving test were examined by comparing the driving tests carried out from the beginning of December 2007 to the end of February 2008 with driving tests conducted during the same period one year later. In order to obtain comparable groups, the analysis was based on those who passed the theory test in the new system. The result showed that the percentage of passed tests increased from 59 to 66.

    Moreover, the analysis indicated that there is a relationship between the theory and driving test in the sense that test-takers with higher scores on the theory test are more likely to pass the driving test.

    In conclusion, the results on both the theory and practical test have improved after the introduction of the new model for the driving-license test and the correlation between the theory test and the driving test is stronger after the change.

  • 13.
    Alger, Susanne
    et al.
    Umeå University, Faculty of Social Sciences, Department of applied educational science.
    Sundström, Anna
    Umeå University, Faculty of Social Sciences, Department of Psychology.
    Agreement of driving examiners' assessments: evaluating the reliability of the Swedish driving test2013In: Transportation Research Part F: Traffic Psychology and Behaviour, ISSN 1369-8478, E-ISSN 1873-5517, Vol. 19, p. 22-30Article in journal (Refereed)
    Abstract [en]

    The purpose of this study was to examine the consistency of examiner assessments of test-takers' performance on the Swedish driving test. The study included 535 tests and was designed so that the ordinary examiner and a supervising examiner assessed the same test-taker. The assessment was done on a two-grade rating scale (pass/fail). Since the result can be affected by factors associated with the test-taker and the two examiners, questionnaires were developed and these were filled in by the test-takers and the examiners. Information about the administration of the test was collected via a specially designed form filled in by the supervising examiner. Using this form, the ordinary examiners' performance was rated on a number of aspects. The result from the study indicated that the agreement between the assessments was very good. For 93% of the tests the two examiners chose the same mark on the two-grade scale. In the cases where ratings differed, the analysis indicated only a few systematic differences among variables designed to provide possible explanations for differences in opinion. However, none of these was problematic with respect to consistency of assessment. Results indicated that most tests were carried out in a satisfactory manner.

  • 14.
    Alger, Susanne
    et al.
    Umeå University, Faculty of Social Sciences, Department of applied educational science, Departement of Educational Measurement.
    Sundström, Anna
    Umeå University, Faculty of Social Sciences, Department of applied educational science, Departement of Educational Measurement.
    Beskrivning av data om det praktiska förarprovet 1998-2005 och analys av tillförlitligheten2011Report (Other academic)
    Abstract [en]

    The aim of this report was to examine the reliability of data from the driving test 1998-2005 and give a basic presentation of data from these years. In order to improve the quality of the data tests that do not appear to have been completed and unnecessary duplicates were removed. After cleaning and recoding, the data can be regarded as reliable.

    From the basic presentation in the report you find that there are certain differences between groups during this period. A larger proportion of test-takers below the age of 20, who make up around half the test-takers, pass the test compared to older test-takers and had a lower average of failed aspects. More men than women took the test and a larger percentage of women were registered by a driving school. A larger proportion of test-takers who were registered by a driving school passed than those who registered themselves. If you divide the test-takers by gender and type of registration a somewhat larger percentage of the men passed the driving test. As for average number of mistakes there are no major differences between the sexes except for manoeuvring.

    As for the content of the test, attention and traffic behaviour are the areas with the largest number of fails. Security check, street crossing and driving towards a goal are the most common areas to test while efficient braking is rare. When you study the content of the test at the main offices and sub-offices you will find that motorway, lanes, signalled crossing and roundabouts are more common at the main offices, while the opposite is true for narrow and wide roads, turning around, left or right.

    As for changes over time you find that the percentage of test-takers registered by driving schools has decreased over the period, as has the percentage of passed tests, both as a total and the percentage who passed their first test. The percentage of passed tests among those who registered for the test themselves has decreased, but there is no obvious downwards trend for those who were registered via a driving school. Efficient braking has become even rarer.

  • 15.
    Alger, Susanne
    et al.
    Umeå University, Faculty of Social Sciences, Department of applied educational science, Departement of Educational Measurement.
    Sundström, Anna
    Umeå University, Faculty of Social Sciences, Department of applied educational science, Departement of Educational Measurement.
    Beskrivning av data om det praktiska förarprovet 2007-2008 och analys av tillförlitligheten2011Report (Other academic)
    Abstract [en]

    The aim of this report was to examine the reliability of data from the driving test for passenger cars 2007-2008 and give a basic presentation of data. The quality of the data was improved by removing tests that do not appear to have been completed and unnecessary duplicates.

    Two major changes occurred during these years. One was that a new driving test and a new test form, for which the data was registered via a digital pen, were introduced in December 2007. Previously data from the test forms had been scanned optically. The work with the new type of data registration led to data for 2007 not being scrutinized in the usual way, which in turn led to these not having the same quality as previous years. Introduction of the digital pen meant that problems caused by the scanning disappeared. However, it turned out that unintentional markings with the digital pen could lead to duplicates being registered. The other major change was the introduction of a new test model in September 2008. From then on it was not only those who had passed the theory test who took the driving test, which must be considered when comparing data with previous years.

    From the basic presentation in the report you will find that there are certain differences between groups. Test-takers below the age of 20 pass the are more likely to pass the test than older test-takers. More men than women took the test and a larger percentage of women were registered by a driving school. A larger proportion of the test-takers who were registered by a driving school passed the test than those who registered themselves. If you divide the test-takers by gender and type of registration a somewhat larger percentage of the men passed the driving test. As for average number of mistakes there are no major differences between the sexes except for manoeuvring. When you study the content of the test at the main offices and sub-offices you find that motorway, driving in lanes, light signal controlled intersection and roundabouts are more commonly tested at the main offices, while the opposite is true for narrow and wide roads/highways, turning around, left or right/ approaching or turning off main roads.

    As for changes over time the security check, which earlier was tested on almost all tests, has become less common in 2008 while revers­ing has been tested very often.The percentage of test-takers registered by driving schools have decreased over the period, as has the percentage of passed tests. Bear in mind, though, that the test model has changed.

  • 16.
    Alger, Susanne
    et al.
    Umeå University, Faculty of Social Sciences, Department of applied educational science.
    Sundström, Anna
    Umeå University, Faculty of Social Sciences, Department of applied educational science.
    Provtagares uppfattning om förarprovet: Enkätundersökning före och efter införandet av sammanhållet förarprov2011Report (Other academic)
    Abstract [sv]

    Den första september 2008 infördes en ny provmodell, ”sammanhållet förarprov”, för körkort behörighet B. Detta innebar att kunskapsprovet och körprovet bokas samtidigt och genomförs samma dag eller nära i tid. Provtagarna börjar med att genomföra kunskapsprovet och oavsett resultat på kunskapsprovet genomför de därefter körprovet. Provtagarna måste få godkänt på båda proven inom en tvåmånadersperiod från det första godkända provet för att få körkort. I annat fall får de göra om båda proven och betala nya provavgifter.

    Denna studie hade som syfte att undersöka om införandet av sammanhållet förarprov påverkat hur provtagarna ser på provet och hur de förbereder sig. En enkät skickades ut till ett slumpmässigt urval bestående av 500 provtagare som genomfört kunskapsprovet före förändringen (augusti 2008) och 500 provtagare som genomfört det efter förändringen (augusti 2010). Urvalsgrupperna utgjordes till hälften av provtagare som fått godkänt och till hälften av provtagare som fått underkänt.

    Det som den nya provmodellen fört med sig är att fler anmäls via trafikskola och något färre väljer att först läsa teori och sedan övningsköra. Inställningen till reglerna kring provet har också förändrats och de nya reglerna har vunnit större acceptans än när de först infördes. I övrigt är det, utifrån de påståenden som prövades i enkäten, inga större skillnader.

  • 17.
    Näsström, Gunilla
    et al.
    Umeå University, Faculty of Social Sciences, Department of Education.
    Alger, Susanne
    Umeå University, Faculty of Social Sciences, Department of applied educational science, Departement of Educational Measurement.
    Sundström, Anna
    Umeå University, Faculty of Social Sciences, Department of applied educational science, Departement of Educational Measurement.
    Samstämmighet mellan kursplan och prov: En studie av kunskapsprovet för körkortsbehörighet AM2011Report (Other academic)
    Abstract [en]

    In order to improve traffic safety a new driving licence category – AM – was introduced on the 1st of October, 2009 which is required to drive a moped class I. In order to obtain this driving licence the aspiring driver must attend a mandatory training course, which includes driving practice, and pass a theory test. The Swedish driving training system is criterion-referenced and consists of three parts – a syllabus with aims, a driver education and a driving test. In order for this system to work these three parts must be aligned. The aim of this study was to examine the syllabus for driver education for driving licence category AM with regard to its content and structure as well as to examine the alignment of the aims of the syllabus to the items in the theory test. Six assessors have analysed the syllabus and a theory test with the help of Bloom’s revised taxonomy. Since certain aims in the syllabus are richer in content these were given more weight in order to give a more correct analysis which resulted in an acceptable level of alignment between the syllabus and the theory test. However, some of the curricular aims are expected to be met during the mandatory training course which means that the alignment between syllabus and the theory test cannot be expected to be perfect.

  • 18.
    Näsström, Gunilla
    et al.
    Umeå University, Faculty of Social Sciences, Department of Education.
    Alger, Susanne
    Umeå University, Faculty of Social Sciences, Department of applied educational science, Departement of Educational Measurement.
    Sundström, Anna
    Umeå University, Faculty of Social Sciences, Department of applied educational science, Departement of Educational Measurement.
    Utvärdering av kunskapsprovet för körkortsbehörighet AM2011Report (Other academic)
    Abstract [en]

    In order to improve traffic safety a new driving licence category – AM – was introduced on the 1st of October, 2009 which is required in order to drive a moped class I. In order to obtain this driving licence the aspiring driver must attend a mandatory training course, which includes driving practice, and pass a theory test. The aim of this study was to analyse the results from the theory test for driving licence category AM during its first year in order to examine its reliability as well as any differences in test results between groups based on various background factors. All test results that the Swedish Transport Agency has gathered during the year were analysed. 17 275 persons took at least one theory test. 98.4 of these were 15 years old. The results showed that the items and the test were of adequate quality since reliability was found to be at a reasonable level and two concurrent test versions were parallel as to level of difficulty. For many items the degree of discrimination was deemed to be at a too low level and this has to be taken into consideration in future test construction. The analysis of the test results showed that 46 per cent of the test-takers passed their first test. While there were no differences in the percentage of passed tests in regard to gender and age there were differences in regard to how they were registered for the test. Test-takers who had been registered for the test by a driving school were more likely to pass the theory test on their first attempt. Private driving practice is not allowed so all who registered for the test had completed mandatory training. The analysis of test results over time showed that the percentage of passed tests increased during the spring and summer. The conclusion from the study is that the theory test for AM is of adequate quality. Since the test has only been in use since 2009 repeated studies of the test results and the quality of the test should be carried out in order to examine whether the quality is maintained and to study trends in test results.

  • 19.
    Sundström, Anna
    et al.
    Umeå University, Faculty of Social Sciences, Department of Educational Measurement.
    Henriksson, Widar
    Umeå University, Faculty of Social Sciences, Department of Educational Measurement.
    Wiberg, Marie
    Umeå University, Faculty of Social Sciences, Department of Educational Measurement. Umeå University, Faculty of Social Sciences, Department of Statistics.
    Alger, Susanne
    Umeå University, Faculty of Social Sciences, Department of Educational Measurement.
    "Ett givet inslag i förarutbildningen": Umeå universitet försvarar självvärderingen2008In: Mitt i trafiken, no 1, p. 33-33Article in journal (Other (popular science, discussion, etc.))
    Abstract [en]

    Sammanfattningsvis framstår det som en självklarhet att självvärdering ska vara ett inslag i såväl förarutbildning som förar­prov eftersom det utgör en väsentlig del i den nya kursplanen. Det står också klart att många vinster, både för lärandet och tra­fiksäkerheten, kan göras genom att införa självvärdering i utbildningen. Nästa steg är att diskutera hur detta ska genomföras i praktiken genom att finna sätt för hur självvärdering ska användas både i utbild­ning och i förarprov.

  • 20.
    Sundström, Anna
    et al.
    Umeå University, Faculty of Social Sciences, Department of Psychology.
    Lyrén, Per-Erik
    Umeå University, Faculty of Social Sciences, Department of applied educational science, Departement of Educational Measurement.
    Alger, Susanne
    Umeå University, Faculty of Social Sciences, Department of applied educational science, Departement of Educational Measurement.
    Att mäta allmänhetens kunskaper om trafiksäkerhet: Utveckling och utprövning av instrument2011Report (Other academic)
    Abstract [sv]

    Nollvisionen är grunden för trafiksäkerhetsarbetet i Sverige och målet är att ingen ska dödas eller skadas allvarligt i trafiken. För att vägtransportsystemet ska användas på ett säkert sätt krävs både kunskaper om trafiksäkerhet och trafiksäkra attityder. Attityder är ett relativt väl utforskat område, medan studier om allmänhetens kunskaper om trafiksäkerhet saknas. Därmed är det önskvärt att kartlägga allmänhetens kunskaper om olika aspekter av trafiksäkerhet och följa utvecklingen av dessa över tid.

    Syftet med studien var att utveckla ett instrument för att mäta allmänhetens kunskaper om trafiksäkerhet och att utvärdera kvaliteten i detta instrument. Två parallella enkäter med kunskapsfrågor utvecklades med utgångspunkt i de indikatorer på trafiksäkerhet som Trafikverket följer i trafiksäkerhetsarbetet. Enkäterna innehöll också bakgrundsfrågor, frågor om attityder och självrapporterat beteende. Enkäterna administrerades till 1000 slumpmässigt utvalda personer i åldrarna 15 till 85 år. Enkäterna kunde besvaras dels via Internet och dels i form av en pappersenkät. Ungefär 50 procent av urvalet besvarade enkäterna och en bortfallsanalys visade att de svarande var representativa för populationen med avseende på kön, ålder och födelseland.

    Resultaten visade att kunskaper om trafiksäkerhet är ett mångdimensionellt begrepp. Detta indikerades bland annat av sambanden mellan de åtta olika innehållsområdena i instrumentet, vilka var svaga. Resultatet visade också att frågorna generellt sett är ganska lätta. Vidare visade resultaten att ett större antal frågor inom varje område behövs för att öka mätsäkerheten. Vissa kunskapsfrågor hade positiva samband med attitydfrågor, vilket indikerar att kunskaper är relaterade till mer positiva attityder till trafiksäkerhet. Resultaten från studien kan ligga till grund för vidare utveckling av ett instrument för att göra en skarp mätning av allmänhetens kunskaper om trafiksäkerhet. Baserat på att begreppet kunskaper om trafiksäkerhet visat sig vara mångdimensionellt bör man som nästa steg göra en bedömning av vilka, specifika aspekter av kunskap om trafiksäkerhet som är relevanta att mäta och baserat på detta, samt resultaten från denna utprövning, välja ut frågor som kan ingå i ett instrument. Ett sådant instrument skulle kunna användas för att kartlägga kunskapsläget i befolkningen samt för att följa kunskapsläget över tid.

1 - 20 of 20
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf