Job description

Job summary

Applications are invited for a Research Associate to work on an Imperial’s Digital Innovation Fund project Module Evaluation in the age of Artificial Intelligence. The post will be based in the Department of Physics while liaising with collaborators at Imperial as well as external collaborators.

Duties and responsibilities

Evaluating modules (courses) to determine whether students are achieving the intended learning outcomes is a challenge when dealing with large class sizes. Traditionally, in Physics, research-based assessment instruments such as the force concept inventory have been used to evaluate the effectiveness of courses in achieving their learning outcomes. These require students to complete additional work and are not designed for one specific course. Furthermore, these multiple-choice assessments naturally lack some of the richness that students are capable of demonstrating. In lab courses, students often produce written work in the form of lab reports and lab notebooks. The purpose of this project is to use those artefacts to evaluate student learning by training large language models to systematically identify skills students demonstrate related to the learning outcomes of the course.

The Research Associate will take the lead on building the training data set for this project before evaluating the possibility of large language models to identify skills such as argumentation and iteration in student work. As part of the project, an instructional intervention is envisioned that we will attempt to measure the impact of using the developed artificial intelligence lab-module evaluation tool.

The role will begin with qualitative work to build the training data set and will draw on experience of teaching or demonstrating in undergraduate physical science laboratories to interpret student work when applying qualitative education methods to analyse that work. The next step is to then select, train, and evaluate large-language models to automate the analysis drawing on knowledge of supervised machine-learning methodologies. We recognise that these are a diverse set of activities and, therefore, are interested in hearing from applicants who have the experience in at least one of these areas and would take the opportunity of this role to gain expertise in the other areas. Hence, we emphasise that we welcome candidates with an interest in diversifying their skillset and will support those candidates in acquiring the needed skills to successfully complete the project.

The project has developed from a collaboration with Natasha Holmes’ Group based at Cornell, which will plan to maintain during this project. The project forms part of a loose collaboration with the Departments of Aeronautical and Mechanical Engineering at Imperial, who are also interested in utilising artificial intelligence to analyse lab reports. This project is a fantastic opportunity to be part of a growing community of education focussed researchers exploring how AI can benefit both students and staff at Imperial.

Essential requirements

You will hold a PhD* in either a numerate discipline (e.g., Physics, Engineering, Mathematics, Computing) or in Education (secondary, tertiary, further or higher) with a STEM focus in a relevant area or have an equivalent level of professional qualifications and experience.

You must also be able to demonstrate the following essential criteria:      

• Practical experience within a research environment and / or publication in relevant and refereed journals
• Proficiency in computer programming with Python (or other similar programming languages)
• Teaching experience (at any level)
• Experience in project leadership (e.g., PhD project work) or evidence of leadership potential
• Knowledge of first and second-year undergraduate physics, engineering, or other relevant degree programme
• Knowledge of statistical analysis techniques used in education and/or physical sciences
• Knowledge of issues of diversity and inclusion in higher education
• Propensity for innovation and R&D including:

-Problem solving and problem analysis / redefinition
-Critical thinking
-Creative thinking and risk taking
-Fostering innovation and creativity
-Leading on new, experimental ideas
-Judging ideas and knowing when to stop
-Collaborating

• Excellent verbal communication skills and the ability to deal with a wide range of people
• Capacity, motivation, and willingness to learn about new disciplines and to apply associated methodologies
• Ability to organise own work with minimal supervision
• Ability to manage time on complex projects and prioritise own work in response to deadlines
• Be a reflective, responsive practitioner committed to personal and professional development, and the development of own research, teaching and learning skills
• Ability to work well in a diverse team environment interacting with individuals at all levels within the College

Further information

This is a full-time position which has to start by 1^st August 2024 at the latest.

*Candidates who have not yet been officially awarded their PhD will be appointed as Research Assistant within the salary range £40,694 - £43,888 per annum.

More information about the Department of Physics can be found on our webpages.

The Department of Physics is an IoP JUNO Practitioner and an Athena Silver SWAN Award winner reflecting our commitment to building a supportive, inclusive and highly motivated community. See our EDI webpage for more information. We strongly encourage applications from all backgrounds, especially those historically underrepresented in Physics.

Further details about the positions are available from Dr. Michael Fox michael.fox@imperial.ac.uk

Should you have any queries about the application process please contact Louise Hayward l.hayward@imperial.ac.uk

Documents

• Job Description - NAT01630.pdf