Chatbot to improve learning punctuation in Spanish and to enhance open and flexible learning environments

  1. Esteban Vázquez-Cano
  2. Santiago Mengual-Andrés
  3. Eloy López-Meneses
Revista:
International Journal of Educational Technology in Higher Education

ISSN: 2365-9440

Any de publicació: 2021

Número: 18

Tipus: Article

DOI: 10.1186/S41239-021-00269-8 DIALNET GOOGLE SCHOLAR lock_openAccés obert editor

Altres publicacions en: International Journal of Educational Technology in Higher Education

Objectius de Desenvolupament Sostenible

Resum

The objective of this article is to analyze the didactic functionality of a chatbot to improve the results of the students of the National University of Distance Education (UNED / Spain) in accessing the university in the subject of Spanish Language. For this, a quasi-experimental experiment was designed, and a quantitative methodology was used through pretest and posttest in a control and experimental group in which the efectiveness of two teaching models was compared, one more traditional based on exercises written on paper and another based on interaction with a chatbot. Subse‑ quently, the perception of the experimental group in an academic forum about the educational use of the chatbot was analyzed through text mining with tests of Latent Dirichlet Allocation (LDA), pairwise distance matrix and bigrams. The quantitative results showed that the students in the experimental group substantially improved the results compared to the students with a more traditional methodology (experimental group / mean: 32.1346 / control group / mean: 28.4706). Punctuation correctness has been improved mainly in the usage of comma, colon and periods in diferent syntac‑ tic patterns. Furthermore, the perception of the students in the experimental group showed that they positively value chatbots in their teaching–learning process in three dimensions: greater “support” and companionship in the learning process, as they perceive greater interactivity due to their conversational nature; greater “feedback” and interaction compared to the more traditional methodology and, lastly, they especially value the ease of use and the possibility of interacting and learning anywhere and anytime

Informació de finançament

Finançadors

Referències bibliogràfiques

  • Ali, S. S., Amin, T., & Ishtiaq, M. (2020). Punctuation errors in writing: A comparative study of students’ performance from different Pakistani universities. Sir Syed Journal of Education & Social Research, 3(1), 165–177. https://doi.org/10.36902/sjesr-vol3-iss1-2020(165-177)
  • Angelillo, J. (2002). Teaching young writers to use punctuation with precision and purpose. Profile Books.
  • Araujo, T. (2018). Living up to the chatbot hype: The influence of anthropomorphic design cues and communicative agency framing on conversational agent and company perceptions. Computers in Human Behavior, 85, 183–189. https://doi.org/10.1016/j.chb.2018.03.051
  • Bailey, D. (2019). Chatbots as conversational agents in the context of language learning. Proceedings of the Fourth Industrial Revolution and Education, pp 32–41. Dajeon, South Korea.
  • Beale, R., & Creed, C. (2009). Affective interaction: How emotional agents affect users. International Journal of HumanComputer Studies, 67, 775–776. https://doi.org/10.1016/j.ijhcs.2009.05.001
  • Benotti, L., Martinez, M. C., & Schapachnik, F. (2018). A tool for introducing computer science with automatic formative assessment. IEEE Transactions on Learning Technologies, 11(2), 179–192. https://doi.org/10.1109/TLT.2017.2682084
  • Bentivoglio, C. A., Bonura, D., Cannella, V., Carletti, S., Pipitone, A., Pirrone, R., Rossi, P. G., & Russo, G. (2010). Agenti intelligenti supporto dell’interazione con l’utente all’interno di processi di apprendimento. Journal of e-Learning and Knowledge Society, 2(6), 27–36
  • Bii, P. (2013). Chatbot technology: A possible means of unlocking student potential to learn how to learn. Educational Research, 4(2), 218–221
  • Blei, D. M., Andrew, Y. N., & Jordan, M. I. (2003). Latent Dirichlet allocation. Journal of Machine Learning Research, 3(4–5), 993–1022
  • Bram, B. (1995). Write well improving writing skills. Kanisius.
  • Bruck, P. A., Motiwalla, L., & Foerster, F. (2012). Mobile learning with micro-content: A framework and evaluation. Proceedings of the 25th Bled eConference, 527–543. Bled, Slovenia.
  • Bruni, E., Tran, N. K., & Baroni, M. (2014). Multimodal distributional semantics. Journal of Artificial Intelligence Research, 49, 1–47. https://doi.org/10.1613/jair.4135
  • Budan, I. A., & Graeme, H. (2006). Evaluating WordNet-based measures of semantic distance. Computational Linguistics, 32(1), 13–47. https://doi.org/10.1162/coli.2006.32.1.13
  • Bullinaria, J. A., & Levy, J. P. (2012). Extracting semantic representations from word cooccurrence statistics: Stop-lists, stemming and svd. Behavior Research Methods, 44, 890–907. https://doi.org/10.3758/s13428-011-0183-8
  • Cabero, J., & Ruiz-Palmero, J. (2018). Technologies of information and communication for inclusion: Reformulating the “digital gap.” IJERI: International Journal of Educational Research and Innovation, 9, 16–30
  • Cabero, J., Vázquez-Cano, E., López-Meneses, E., & Jaén-Martínez, A. (2020). Posibilidades formativas de la tecnología aumentada. Un estudio diacrónico en escenarios universitarios. Revista Complutense De Educación, 31(2), 143–154. https://doi.org/10.5209/rced.61934
  • Caddéo, S. (1998). L’usage de la ponctuation chez les enfants. In J.-M. Defays, L. Rosier, & F. Tilkin (Eds.), Actes du colloque international et interdisciplinaire de Liège: A qui appartient la ponctuation? (pp. 255–274). De Boeck.
  • Cassany, D. (1999). Puntuación: Investigaciones, concepciones y didáctica. Letras, 58, 21–54
  • Chen, J. A., Tutwiler, M. S., Metcalf, S. J., Kamarainen, A., Grotzer, T., & Dede, C. (2016). A multi-user virtual environment to support students’ self-efficacy and interest in science: A latent growth model analysis. Learning and Instruction, 41, 11–22. https://doi.org/10.1016/j.learninstruc.2015.09.007
  • Ciechanowski, L., Przegalinska, A., & Wegner, K. (2018). The necessity of new paradigms in measuring human–chatbot interaction. In M. Hoffman (Ed.), Advances in cross-cultural decision making. (pp. 205–214). Springer.
  • Colace, F., Santo, M. D., Lombardi, M., Pascale, F., Pietrosanto, A., & Lemma, S. (2018). Chatbot for e-learning: A case of study. International Journal of Mechanical Engineering and Robotics Research, 7(5), 528–533. https://doi.org/10.18178/ijmerr.7.5.528-533
  • Coniam, D. (2008). Evaluating the language resources of chatbots for their potential in English as a second language. ReCALL, 20(01), 98–116. https://doi.org/10.1017/S0958344008000815
  • Coniam, D. (2014). The linguistic accuracy of chatbots: Usability from an ESL perspective. Text & Talk, 34(5), 545–567. https://doi.org/10.1515/text-2014-0018
  • Cordova, D. I., & Lepper, M. R. (1996). Intrinsic motivation and the process of learning: Beneficial effects of contextualization, personalization, and choice. Journal of Educational Psychology, 88(4), 715–730. https://doi.org/10.1037/0022-0663.88.4.715
  • Crown, S., Fuentes, A., Jones, R., Nambiar, R., & Crown, D. (2010). Ann G. Neering: Interactive chatbot to motivate and engage engineering students. American Society for Engineering Education, 15(1), 1–13
  • Daffern, T., & Mackenzie, N. (2015). Building strong writers: Creating a balance between the authorial and secretarial elements of writing. Literacy Learning: the Middle Years, 23(1), 23–32
  • Fang, Z., & Wang, Z. (2011). Beyond rubrics: Using functional language analysis to evaluate student writing. Australian Journal of Language and Literacy, 34(2), 147–165
  • Farkash, Z. (2018). Education Chatbot: 4 ways chatbots are revolutionizing education. Chatbot Magazine. https://chatbotsmagazine.com/education-chatbot-4-ways-chatbots-arerevolutionizing-education-33f36627964c
  • Feng, Y., Bagheri, E., Ensan, F., & Jovanovic, J. (2017). The state of the art in semantic relatedness: A framework for comparison. Knowledge Engineering Review, 32, 1–30. https://doi.org/10.1017/S0269888917000029
  • Ferreiro, E. (1999). Cultura escrita y educación. Conversaciones con Emilia Ferreiro. Fondo de Cultura Económica.
  • Ferreiro, E., & Teberosky, A. (1979). Los sistemas de escritura en el desarrollo del niño. Siglo XXI.
  • Fryer, L. K., & Carpenter, R. (2006). Bots as language learning tools. Language Learning and Technology, 10(3), 8–14. http://llt.msu.edu/vol10num3/emerging/
  • Fryer, L. K., Nakao, K., & Thompson, A. (2019). Chatbot learning partners: Connecting learning experiences, interest and competence. Computers in Human Behavior, 93, 279–289. https:// doi. org/ 10. 1016/j. chb. 2018. 12. 023
  • Fuente, M. (1993). Los signos de puntuación: Normativa y uso. Universidad de Valladolid. Garcia Brustenga, G., Fuertes-Alpiste, M., & Molas-Castells, N. (2018). Briefing paper: Los chatbots en educación. eLearn Center. Universitat Oberta de Catalunya.
  • García-Valdecasas, J. (2011). Agent-based modelling: A new way of exploring social phenomena. Revista Española De Investigaciones Sociológicas, 136, 91–110. https://doi.org/10.5477/cis/reis.136.91
  • Ghose, S., & Barua, J. (2013). Toward the implementation of a topic specific dialogue based natural language chatbot as an undergraduate advisor. Proceedings of the International Conference on Informatics, Electronics and Vision, 1–5. Dhaka, Bangladesh. doi: https://doi.org/10.1109/ICIEV.2013.65726 50
  • Giurgiu, L. (2017). Microlearning an evolving elearning trend. Scientific Bulletin, 22(1), 18–23. https://doi.org/10.1515/bsaft-2017-0003
  • Goda, Y., Yamada, M., Matsukawa, H., Hata, K., & Yasunami, S. (2014). Conversation with a chatbot before an online EFL group discussion and the effects on critical thinking. The Journal of Information and Systems in Education, 13(1), 1–7. https://doi.org/10.12937/ejsise.13.1
  • Grossman, J., Lin, Z., Sheng, H., Wei, J. T.-Z., Williams, J. J., & Goel, S. (2019). MathBot: Transforming online resources for learning math into conversational interactions. http://logical.ai/story/papers/mathbot.pdf
  • Gupta, S., & Jagannath, K. (2019). Artificially intelligently (AI) tutors in the classroom: A need assessment study of designing chatbots to support student learning. Proceedings of the Twenty-Third Pacific Asia Conference on Information Systems, 1–8. Chicago, United States.
  • Hasler, B. S., Tuchman, P., & Friedman, D. (2013). Virtual research assistants: Replacing human interviewers by automated avatars in virtual worlds. Computers in Human Behavior, 29, 1608–1616. https://doi.org/10.1016/j.chb.2013.01.004
  • Hattie, J., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77, 81–112
  • Heidig, S., & Clarebout, G. (2011). Do pedagogical agents make a difference to student motivation and learning? Educational Research Review, 6, 27–54. https://doi.org/10.1016/j.edurev.2010.07.004
  • Hill, J., Ford, W. R., & Farreras, I. G. (2015). Real conversations with artificial intelligence: A comparison between humanehuman online conversations and humanechatbot conversations. Computers in Human Behavior, 49, 245–250. https://doi.org/10.1016/j.chb.2015.02.026
  • Ho, A., Hancock, J., & Miner, A. S. (2018). Psychological, relational, and emotional effects of self-disclosure after conversations with a chatbot. Journal of Communication, 68(4), 712–733. https://doi.org/10.1093/joc/jqy026
  • Hsu, H.-C.K., Wang, C. V., & Levesque-Bristol, C. (2019). Reexamining the impact of self-determination theory on learning outcomes in the online learning environment. Education and Information Technologies, 24(3), 2159–2174. https://doi.org/10.1007/s10639-019â-09863-w
  • Huang, W., Hew, K. F., & Gonda, D. E. (2019). Designing and evaluating three chatbot enhanced activities for a flipped graduate. International Journal of Mechanical Engineering and Robotics Research, 8(5), 813–818. https://doi.org/10.18178/ijmerr.8.5.813-818
  • Io, H. N., & Lee, C. B. (2018). Chatbots and conversational agents: A bibliometric analysis. Proceedings of the IEEE International Conference on Industrial Engineering and Engineering Management, 215–219. Singapore.
  • Jeno, L. M., Adachi, P. J., Grytnes, J. A., Vandvik, V., & Deci, E. L. (2019). The effects of m-learning on motivation, achievement and well-being: A self-determination theory approach. British Journal of Educational Technology, 50(2), 669–683. https://doi.org/10.1111/bjet.12657
  • Johnson, W. L., & Lester, J. C. (2016). Face-to-face interaction with pedagogical agents, twenty years later. International Journal of Artificial Intelligence in Education, 26, 25–36
  • Jomah, O., Masoud, A. K., Kishore, X. P., & Aurelia, S. (2016). Micro learning: A modernized education system. BRAIN Broad Research in Artificial Intelligence and Neuroscience, 7(1), 103–110
  • Jones, M., & Mewhort, D. (2007). Representing word meaning and order information in a composite holographic lexicon. Psychological Review, 114(1), 1–37. https://doi.org/10.1037/0033-295X.114.1.1
  • Klevjer, R. (2006). What is the avatar? Fiction and embodiment in avatar-based single player computer games. Dissertation for the degree doctor rerum politicarum. University of Bergen.
  • Klopfenstein, L. C., Delpriori, S., Malatini, S., & Bogliolo, A. (2017). The rise of bots: A survey of conversational interfaces, patterns, and paradigms. Proceedings of the 2017 Conference on Designing Interactive Systems, DIS ’17, 555–565. New York, United States. https://doi.org/https://doi.org/10.1145/3064663.3064672
  • Labuhn, A. S., Zimmerman, B. J., & Hasselhorn, M. (2010). Enhancing students’ self-regulation and mathematics performance: The influence of feedback and self‑evaluative standards. Metacognition and Learning, 5(2), 173–194. https://doi.org/101007/s11409-010-9056-2
  • Liew, T., Mat Zin, N., & Sahari, N. (2017). Exploring the affective, motivational and cognitive effects of pedagogical agent enthusiasm in a multimedia learning environment. Human-Centric Computing and Information Sciences, 7(1), 1–21. https://doi.org/10.1186/s13673-017- 0089-2
  • Liu, Q., Huang, J., Wu, L., Zhu, K., & Ba, S. (2019). CBET: Design and evaluation of a domain-specific chatbot for mobile learning. Universal Access in the Information Society. https://doi.org/10.1007/s10209-019-00666-x
  • López-Meneses, E., Sirignano, F. M., Vázquez-Cano, E., & Ramírez-Hurtado, J. M. (2020). University students’ digital com-petence in three areas of the DigCom 2.1 model: A comparative study at three European universities. Australasian Journal of Educational Technology, 36(3), 69–88. https://doi.org/10.14742/ajet.5583
  • Macken-Horarik, M., & Sandiford, C. (2016). Diagnosing development: A grammatics for tracking student progress in narrative composition. International Journal of Language Studies, 10(3), 61–94
  • Mohammed, G. S., & Wakil, K. (2018). The effectiveness of microlearning to improve students’ learning ability. International Journal of Educational Research Review, 3(3), 32–38
  • Nikou, S. A. (2019). A micro-learning based model to enhance student teachers’ motivation and engagement in blended learning. Proceedings of the SITE 2019, Society for Information Technology and Teacher Education, 255–260. Las Vegas, United States.
  • Nikou, S. A., & Economides, A. A. (2017). Mobile-based assessment: Integrating acceptance and motivational factors into a combined model of self-determination theory and technology acceptance. Computers in Human Behavior, 68, 83–95. https://doi.org/10.1016/j.chb.2016.11.020
  • Nikou, S. A., & Economides, A. A. (2018). Mobile-based micro-learning and assessment: Impact on learning performance and motivation of high school students. Journal of Computer Assisted Learning, 34(3), 269–278. https://doi.org/10.1111/jcal.12240
  • Paschoal, L. N., Turci, L. F., Conte, T. U., & Souza, S. R. S. (2019). Towards a conversational agent to support the software testing education. Proceedings of the XXXIII Brazilian Symposium on Software Engineering, 57–66. Curitiba, Brazil.
  • Polo, J. (1990). Manifiesto ortográfico de la lengua española. Visor.
  • Procter, M., Lin, F., & Heller, B. (2012). Intelligent intervention by conversational agent through chatlog analysis. Smart Learning Environments, 5(30), 1–15. https://doi.org/10.1186/s40561-018-0079-5
  • Reyes-Reina, D., Vilaça, L., Spolidorio, S., & Martins, M. (2019). El desarrollo sociotécnico de un chatbot o ¿Cómo se construye una caja negra? Revista Tecnologia e Sociedade, 16(39), 23–40
  • Rosenthal, R. (1991). Effect sizes: Pearson’s correlation, its display via the BESD, and alternative indices. American Psychologist, 46(10), 1086–1087
  • Rosenthal, R., & Rubin, D. B. (1982). A simple, general purpose display of magnitude of experimental effect. Journal of Educational Psychology, 74(2), 166–169. https://doi.org/10.1037/0022-0663.74.2.166.
  • Ruan, S., Willis, A., Xu, Q., Davis, G. M., Jiang, L., Brunskill, E., & Landay, J. A. (2019). BookBuddy. Proceedings of the Sixth ACM Conference on Learning @ Scale L@S ’19, 1–4. New York, United States. https://doi.org/https://doi.org/10.1145/3330430.3333643
  • Schroeder, N., Adesope, O., & Gilbert, R. (2013). How effective are pedagogical agents for learning? A metaanalytic review. Journal of Educational Computing Research, 49(1), 1–39. https://doi.org/10.2190/ec.49.1.a
  • Schroeder, N. L., Romine, W. L., & Craig, S. D. (2017). Measuring pedagogical agent persona and the influence of agent persona on learning. Computers & Education, 109, 176–186. https://doi.org/10.1016/j.compedu.2017.02.015
  • Scull, J., & Mackenzie, N. M. (2018). Developing authorial skills: Child language leading to text construction, sentence construction and vocabulary development. In N. M. Mackenzie & J. Scull (Eds.), Understanding and supporting young writers from birth to 8. (pp. 89–115). Routledge.
  • Sha, G. (2009). AI-based chatterbots and spoken English teaching: A critical analysis. Computer Assisted Language Learning, 22(3), 269–281. https://doi.org/10.1080/09588220902920284
  • Shail, M. S. (2019). Using micro-learning on mobile applications to increase knowledge retention and work performance: A review of literature. Cureus, 11(8), e5307. https://doi.org/10.7759/cureus.5307
  • Sheth, A., Yip, H. Y., Iyengar, A., & Tepper, P. (2019). Cognitive services and intelligent chatbots: Current perspectives and special issue introduction. IEEE Internet Computing, 23(2), 6–12
  • Shum, H.-Y., He, X., & Li, D. (2018). From Eliza to XiaoIce: Challenges and opportunities with social chatbots. Frontiers of Information Technology & Electronic Engineering, 19(1), 10–16. https://doi.org/10.1631/FITEE.1700826
  • Smutny, P., & Schreiberova, P. (2020). Chatbots for learning: A review of educational chatbots for the Facebook Messenger. Computers & Education, 151, 103862. https://doi.org/10.1016/j.compedu.2020.103862
  • Stickler, U., & Hampel, R. (2015). Transforming teaching: New skills for online language learning spaces. Palgrave Macmillan.
  • Subramaniam, N. K. (2019). Teaching & learning via chatbots with immersive and machine learning capabilities. Proceedings of the ICE 2019 Conference Proceedings, 145–156. Jyväskylä, Finland.
  • Tamayo, P. A., Herrero, A., Martín, J., Navarro, C., & Tránchez, J. M. (2020). Design of a chatbot as a distance learning assistant. Open Praxis, 12(1), 145–153. https://doi.org/10.5944/openpraxis.12.1.1063
  • Taraban, R. (2018). Practicing metacognition on a chatbot. Improve with metacognition. http://www.improvewithmetacognition.com/2035–2/
  • Tegos, S., Demetriadis, S., & Tsiatsos, T. (2014). A configurable conversational agent to trigger students’ productive dialogue: A pilot study in the CALL domain. International Journal of Artificial Intelligence in Education, 24(1), 62–91. https://doi.org/10.1007/s40593-013-0007-3
  • Tegos, S., Psathas, G., Tsiatsos, T., & Demetriadis, S. (2019). Designing conversational agent interventions that support collaborative chat activities in MOOCs. Proceedings of EMOOCs 2019: Work in Progress Papers of the Research, Experience and Business Tracks, 66–71. Naples, Italy.
  • Thompson, A., Gallacher, A., & Howarth, M. (2018). Stimulating task interest: Human partners or chatbots? Proceedings of the Future-proof CALL: language learning as exploration and encounters, 302–306. Jyväskylä, Finland.
  • Van Rosmalen, P., Eikelboom, P., Bloemers, E., Van Winzum, K., & Spronck, P. (2012). Towards a game-chatbot: Extending the interaction in serious games. Proceedings of 6th European Conference on Games Based Learning, 1–8. Cork, Ireland.
  • Vázquez-Cano, E. (2012). Mobile learning with Twitter to improve linguistic competence at secondary schools. The New Educational Review, 29(3), 134–147
  • Vázquez-Cano, E. (2014). Mobile distance learning with smartphones and apps in higher education. Educational Sciences: Theory & Practice, 14(4), 1–16. https://doi.org/10.12738/est.2014.4.2012
  • Vázquez-Cano, E., Fombona, J., & Fernández, A. (2013). Virtual attendance: Analysis of an audiovisual over IP system for distance learning in the Spanish Open University (UNED). The International Review of Research in Open and Distance Learning (IRRODL), 14(3), 402–426. https://doi.org/10.19173/irrodl.v14i3.1430
  • Vázquez-Cano, E., Holgueras, A. I., & Sáez-López, J. M. (2018). An analysis of the ortographic error found in university students’ asynchronous digital writing. Journal of Computing in Higher Education, 31(1), 1–20. https://doi.org/10.1007/s12528-018-9189-x
  • Vijayakumar, R., Bhuvaneshwari, B., Adith, S., & Deepika, M. (2019). AI based student bot for academic information system using machine learning. International Journal of Scientific Research in Computer Science, Engineering and Information Technology, 5(2), 590–596. https://doi.org/10.32628/CSEIT1952171
  • Wang, N., Johnson, W. L., Mayer, R. E., Rizzo, P., Shaw, E., & Collins, H. (2008). The politeness effect: Pedagogical agents and learning outcomes. International Journal of Human Computer Studies, 66, 96–112. https://doi.org/10.1016/j.ijhcs.2007.09.003
  • Weizenbaum, J. (1966). ELIZA—A computer program for the study of natural language communication between man and machine. Communications of the ACM, 9(1), 36–45
  • Wing Jan, L. (2009). Write ways: Modelling writing forms. Oxford University Press.
  • Winkler, R., & Soellner, M. (2018). Unleashing the potential of chatbots in education: A state-of-the-art analysis. Proceedings of the 78th Academy of Management Annual Meeting, 1–40. Chicago, Illinois.