Being a director does not mean that you just watch the others working; sometimes it is necessary to produce some theorethical work too. Not necessary complex works, you just try to find practical solutions to real problems encountered in your experience, or of your colleagues.
The subject of the annual Workshop of the Accademia Italiana di Lingua (AIL), next March, will be the evaluation of the oral part of the DALI (Diploma of Italian Language, Advanced Level, C1 according to the European Language Portfolio), and I am working on my communication at the Workshop.
The exams of Accademia Italiana di Lingua are employed basicly in Italy and Switzerland, and the (rather) low number of candidates (about one thousand per year) allows AIL to delivery an high quality exam – for example, with an oral part administered and evalued by human examiners.
There is a scorecard like that

Meaning that the two candidates (they work in pair) have to perform three oral works: a synthesis, a discussion and a role playing; and they can receive a score from 0 to 2 points on their comunicativeness(efficacia comunicativa), expressiveness (ricchezza espressiva), correctness (correttezza formale); furthermore, they receive a score from 0 to 2 based on their pronunciation.

How to give the points from 0 to 2 is explained in a chart like that,

which for every activity tells the examiner how to value the performance of the candidate.

It looks quite granular, therefore quite precise. Still, as a matter of fact, in many cases the couple of examiners gives very different scores to the same candidate, especially if the examiners are not used to work together. (In these cases, there is a short discussion and they find a compromise, or make the mathematical average between the two scores)
This phenomenon became evident when, at one Workshop, we run some exam simulations, with real candidates, and the evaluations differed dramatically, with the nationalities of the examiners (yes, it is true that the Italian tend to give higher scores), with the age of the examiners, with the experience of examiners (expert examiners tend to give higher scores) but also according to other variables, for examples, Japanese students tended to be judged with higher scores by theachers who are used to work with Japanese students, etc.

So, the question AIL Workshop will address is – is there a less subjective way, more independent by the cultural background, personality, age etc. of the examiner, to evaluate the candidates ?
For me, it is a very interesting question also because it lays at the crossing of humanities and science – maybe you really can not measure some skills, like language abilities, but you are forced to try anyway, otherwise you renounce to make exams, release certificates, diplomas etc…
(end of part I)