Guidelines for creating good Custom and Adaptive Tests

Print Friendly, PDF & Email

e-asTTle testing is not about answering all questions correctly. 100% correct is not a desirable outcome! In fact, you will get the most useful information from e-asTTle when you set a test where you expect students to answer about 50% of the test correctly.

To understand this, it can be useful to understand the principles behind e-asTTle scoring.

Imagine the questions in the test ordered from easiest to hardest. e-asTTle is trying to find the point where the student is transitioning from answering questions mostly correctly to mostly incorrectly – i.e. the “tipping point” where they have a 50/50 chance of answering correctly. The difficulty level associated with the “tipping point” question becomes the student’s score.

To find this “tipping point”, the test should have some questions that are easy for the student, and some that are challenging. This means choosing appropriate curriculum levels is a crucial decision.

This is a simplified version of how things work, but it is useful to remember that a student ‘at’ 3A is not expected to get 100% of 3A questions correct! A 3A student is considered to have a 50% chance of answering a Level 3A question correctly. (For more in-depth information, see The Rasch Model in e-asTTle)


Test Creation Advice


1. Target your test to the students’ current ability

A common misconception is that no matter where you set the curriculum levels, your students can be accurately assessed across curriculum levels 2-6.

However, when students are at a much higher or lower curriculum level than the questions in the test, e-asTTle scores are much less precise. You will not get much formative information from the Individual Learning Pathways.

Without questions at the student’s level in the test, e-asTTle must extrapolate to work out the “tipping point”. When students answer everything correctly, all we know is that the student is operating above the level of the test – and not how far above. When students answer very few questions correctly, all we know is the student is operating below the test – and not how far below.

If you have new students, you may wish to check their Student Result Summary. This will show you any e-asTTle results they have from previous schools for you to use as a reference.


 2. Consider creating multiple tests for a class or cohort

e-asTTle is not a ‘one-size-fits-all’approach – it is often advisable to create multiple tests and assign them to students of different abilities. This will give you higher-quality formative information across all students.

For example, if your students are achieving from Levels 2-4, create tests for levels 2/3, 3/4 and 4/5.

Download an example planning template  for a school-wide testing approach.  Or, download a blank template to use in your school.



3. Testing across two levels is a good rule of thumb

If in doubt, creating each test across two levels is a good starting point. Tests that cover only one level tend to not have enough scope, particularly for adaptive tests. Tests covering three levels can also be useful, however it is advisable to select ‘Most’ for at least one of the sliders.  This prevents the test from becoming too broad.


4. Review the Test

Always review the test to ensure it meets your expectations. You are able to edit the levels selected and/or replace some questions if needed. See: Reviewing the Test Summary



 Choosing Curriculum Levels for an Adaptive Test

This is a similar process as for Custom Tests, however for an Adaptive test you simply set Yes/No for a level. Selecting ‘yes’ indicates the test may adapt to that level. It is advisable to select at least two curriculum levels.

A common misconception about e-asTTle Adaptive tests is that no matter where you set the curriculum levels, the test can adapt from Levels 2-6. This is not the case. Even with Adaptive tests, when you create the test, you are in charge of the boundaries of the test. The levels you select dictate the lower and upper limits of the test. Within those levels, e-asTTle will attempt to present the student with questions most appropriate to them. For example if you create a Level 3 and 4 test the lowest level question will be at 3B and the highest at 4A.

As with Custom tests, consider creating multiple Adaptive tests and targeting them to the appropriate students.