Crafting a questionnaire requires extensive exploration and careful consideration. The instrument has been designed to investigate students’ metacognitive knowledge and regulation regarding soft skills—particularly those related to STEM—and to assess the effectiveness of my newly developed curriculum and pedagogy in fostering these skills.
To develop the questionnaire, I drew on data gathered during Cycles 1 and 2 of my Action Research, my intervention plan from the previous PG Certificate term, and my ongoing observations of student behaviour that is the target of my intervention. After drafting the questionnaire, I used Copilot AI to test its effectiveness. The feedback was positive, but it suggested reframing some questions to avoid vagueness and potential confusion.
I submitted the questionnaire, along with its intended purpose, to my Course Leader for review and he aprooves of it.
Another strategy to evaluate the questionnaire’s effectiveness involves consulting my tutor, Kwame. As someone outside my academic and industry subculture, he serves as a “cultural stranger,” which helps reduce bias. As Presser and Converse (1986) highlight, it is important to seek “differing perspectives and experiences to get new information.”
To trial the questionnaire as a pretest, I used a group of MA students who are nearing the end of their course. These students are not part of my intervention group. The aim was to identify any issues such as missing response options, inappropriate vocabulary, or potential misinterpretations. This pretest was conducted without informing participants that the questionnaire was still under development, to ensure authentic responses. It was also applied in a different class without notice, so there was no appropriate atmosphere or expectation for it.
Since these trialling students were not involved in the December workshop and were unfamiliar with any activity involving soft skills development, I designed a mini lecture to provide context (details below). This segment lasted approximately 35 minutes and about 10 students were in class.
Trial Structure:
1- Mini Lecture on Importance of Soft Skills in VFX/CG Industries (35 minutes):
Students were introduced to the soft skills—particularly STEM-related ones—most in demand in the industry. I also presented relevant data on employer expectations and a complete list of soft skills: those exclusive to VFX/CG and those more general and still valuable.
2- Questionnaire Completion (1.5 hours):
Since these students were amidst final project delivery and already applying for jobs, the questionnaire was adapted to reflect their context.
Unfortunately, student engagement to voluntarily respond the questionnaire was very poor: only 3 students agreed to participate out of 10.
However, surprisingly, three students were enough and I was able to reformulate all the questionnaire. Below I have uploaded two Word documents:
1- Questionnaire Trial Analysis and Commentaries.
2- Questionnaire Ellaboration showing indications and explanations of which questions I changed after the trialling.
Please use control + mouse wheel to zoom out and read the documents.
Reference:
Converse, J. M. and Presser, S., 1986. Survey Questions. Quantitative Applications in the Social Sciences. Thousand Oaks, CA: SAGE Publications, Inc. Available at: <https://doi.org/10.4135/9781412986045> [Accessed 5 Nov 2025].