Table of Contents
Towards a Voice-Based Chatbot for Language Learners (ChaLL)
In this Innosuisse-Project, we take first steps towards developing a voice-based chatbot that provides language learners with opportunities to practice speaking in both focused and unfocused task-based conversations and receive feedback, free from the time constraints and pressures of the traditional classroom setting.
Speaking practice is essential for successful foreign language learning; however, it can be difficult to achieve this in the classroom: often, there is not enough time to allowfor all learners to speak, and fear of being judged can make it difficult to speak freely. We take first steps towards developing a voice-based chatbot that will provide learners with vital speaking opportunities. In order to become the ideal language learning companion, the chatbot will need to
- adjust its level of speech complexity to each learner to ensure that the interactions are in the “zone of proximal development”, which is the optimal level for of potential development.
- provide real-time feedback and support to achieve an ideal learning effect.
However, the success of this project depends on
- Speech-To-Text technology to adequately “understand” learners‘ free speech - including the errors they make - as the basis for interacting with the chatbot.
- Automatically detecting and classifying errors in the transcribed speech as the basis for providing feedback.
- The degree to which learners' skill levels can be identified automatically.
- The adaptability of the chatbot's output language to the detected learner level.
The proposed project therefore conducts a series of feasibility studies to address these questions and lay out a vision of how to adapt and expand NLP technologies and, crucially, combine them with a didactically sound syllabus curriculum.
Motivation
Speaking is one of the core competencies to be developed in foreign language classes and the second most-widely used skill in everyday-life communication. If students are to be prepared for the ‘real’ world to successfully perform on the international stage, they must demonstrate highly functional communicative foreign language proficiency levels.
But, first, there are usually not enough opportunities to speak: the average primary school student engages in roughly one minute of speaking English per lesson. Second, not all primary school teachers encourage their students to speak the second/foreign language (L2) frequently, or they use their first/local language (L1) as their language of instruction. Third, teachers often use similar question-answer patterns which do not allow learners to use English beyond the one-sentence-level and to acquire a broad range of speaking functions. Fourth, teachers do not necessarily teach speaking skills.
Our solution
Our proposed solution, ChaLL, will provide both focused/closed and unfocused/open speaking tasks: in the former, learners can practise and automatise specific skills via structured tasks based on task-oriented learning in a didactically curated syllabus, while the latter encourages learners to speak freely and the focus is on fluency.
There are numerous challenges, from text-to-speech to deep understanding. In these areas, neural networks and large language models have made tremdous progress recently, as witnessed e.g. by DeepL and ChatGPT. We want to create a chatbot that is entertaining, encouraging and friendly, but which at the same time also detects the competence level of each pupil and his/her frequent errors, on order to be able to select suitabl exercises, and paraphrase texts to a level suitable for the learner.
Our research partners
ChaLL is an Innousuisse collaboration by three academic partners:
- ZHAW (Project Website)
- PHZH (Project Website)
- UZH (our contribution)
Project ID
Innosuisse Project No. 102.580.1 IP-ICT
Project period
1.2.2023 -– 30.6.2024
Project team
Prof. Dr. Mark Cieliebak, Prof. Dr. Michael Geiss, Dr. Johannes Graën, Manuela Hürlimann, Dr. Luzia Sauer, Prof. Dr. Gerold Schneider, Dr. Don Tuggener.