Information about ZBF
Some comments about using the ZBF
Dear students,
This just a brief note about using the big 5 personality questionnaire in your role of assessor. And about understanding your own outcomes as a candidate.
This version of ZBF is an internal tool, it’s used in assessments by professionals and it’s not publicly available. That’s why there is no statistical manual to read. There is no official publication with confidence-inspiring tables. You won’t find roaring reviews in distinguished journals :). And that’s a good thing. Because it emphasizes the fact that we should always be very cautious in using the outcomes of personality questionnaires.
You could read this clarification as a ‘message from the trenches’ – some facts, tips and nice-to-knows, based on day-to-day use in real life assessments by professional assessors.
General remarks
In general, it’s good to be very cautious regarding things like predictive validity. And this is especially relevant in the context of this practical. Remember that tests and questionnaires are just tools. Tools are not perfect. The user of the tool is not perfect. But the combination of a well-developed tool and a well-trained user can yield awesome results. ‘A test’ usually is not that reliable on its own. But the combination of different tools, multiple moments of measurement and multiple assessors is what makes assessments reasonable reliable. My point is: never trust an outcome only because the test is ‘a good test’.
We do not dive into the details of the definition of big5 personality traits in this note – you can easily look it up, there is a lot of information freely available.
Context
In this practical, as test-result-analyzer we’re confronted with a problem. There is not one clear, homogeneous context. Some candidates will regard this as a high-stakes situation. They will really try to get the job and their answering behavior will be guided by their desire to ‘win’. Others will see themselves as sparring partners for the assessors. Some will be very honest because they see this as a research-data-gathering situation. And quite a big proportion of the candidates probably didn’t really think about it. ZBF is just one the gazillion questionnaires you have to do when you’re a psychology student.
In real life assessments, the vast majority of candidates will show a faking-good, high stakes style of response. Much more predictable behavior.
Nice to know: test results from psychologists and psychology students are often weird :). In some ways, this group of people is untestable. They know too much, sometimes they give strange answers because they think they know what the questions are about and so on.
Quite often, having a psychology background leads to extreme profiles. So don’t be surprised when this happens.
Predictive validity
Indications are that the predictive validity of the ZBF will be comparable to other (well established) Big5 personality questionnaires. So how does that translate to your job as assessor in this practical?
1 You can trust the factor-outcomes. These are based on about 30 items each. That is a solid base for conclusions.
2 You should use the underlying components only as an indicator. These are based on less than 10 items. That is statistically sufficient, but not every component has the same level of statistical reliability. That means that the validity actually can change quite a bit, depending on your candidate. That’s why we use the term ‘indicator’.
Normed scores and norm groups
If you do not fully understand normed scores, just ignore them and use POMP (percentage of maximum possible) scores. Don’t mix them up.
Extreme results
To conclude, some information about extreme results. A stanine score 9 or 1 indicates that the candidate belongs to a group of 4% with the most extreme results. Stanines 8 and 2 contain the top or bottom 11%. That sounds extreme, but often, you will encounter extreme scores, while the candidate looks and acts quite normal. So: extreme scores can co-exist with not-so-extreme behavior. How to unify these widely differing observations?
The answer to this question lies in the comparison group. Of course, part of the explanation has to do with the individual that chose to give the answers they gave. But knowing the norm group is very important. ZBF is used mostly for assessments in a career context. All respondents are working or studying adults. So an extreme result means that the candidate is extreme in relation to a very normal group of people. Not extreme in relation to the total population. Keep that in mind.
At the moment, the norm group used is about n = 700, you can find some details about the composition of the norm group in the report.
Sjoerd A. Dingemanse
November 30 2023