Evaluating the Effectiveness of Teaching Information Systems Courses: A Rasch Measurement Approach
Abstract
Systems analysis and design (SAND) is an information systems (IS) course that is taught around the world in most higher education management of information systems (MIS) programmes. However, the theoretical nature of this type of course presents challenges for instructors as they devise instructional strategies to convey the abstract concepts that are necessary for their students to understand, such as, how to draw data flow diagrams (DFD) to correctly represent the informational specifications of an IS. Evidence suggests that one of the factors of the low success rates of many IS-design projects in the workforce is due to the graduate recruits’ failure to acquire basic SAND knowledge. While a considerable amount of literature focused on integrating technology into the teaching practices to facilitate the knowledge acquisition, a few investigated its effectiveness to fulfil this particular purpose. This paper reflects on such challenges and proposes an evaluation approach to assess the effectiveness of technology integration in teaching an IS course like SAND. The empirical interpretations represented in this paper are gathered through a series of quasi-experimental 2x3 factorial experiments that were conducted at four higher education institutions and based on the Rasch item response theory and measurement analysis. The preliminary analysis from this study provides reliable evidence to delineate key instructional strategies when designing higher education IS courses.
Keywords
systems analysis and design; courseware design; higher education; Rasch model; instructional design; information communications technology tools
References
Boyle, E. A., Duffy, T. & Dunleavy, K. (2003). Learning styles and academic outcome: The validity and utility of Vermunt’s Inventory of Learning Styles in a British higher education setting. British Journal of Educational Psychology, 73, 267–290.
Branch, R. M. & Kopcha, T. J. (2014). Instructional design models. In Handbook of research on educational communications and technology (pp. 77–87). New York: Springer.
Branson, R. K., Rayner, G. T., Cox, J. L., Furman, J. P. & King, F. (1975). Interservice procedures for instructional systems development. Executive summary and model. DTIC Document.
Cybulski, J. L. & Linden, T. (2000). Learning systems design with UML and patterns. IEEE Transactions on Education, 43(4), 372–376.
Fatima, S. & Abdullah, S. (2013). Improving teaching methodology in system analysis and design using problem based learning for ABET. International Journal of Modern Education and Computer Science (IJMECS), 5(7), 60–68.
Gagné, R. M. (1985). The conditions of learning and theory of instruction. New York: Holt, Rinehart and Winston.
Izard, J. (2005). Overview of test construction. In Kenneth, N. R. (Ed.), Quantitative research methods in educational planning. Paris: UNESCO International Institute for Educational Planning.
Kiviniemi, M. T. (2014). Effects of a blended learning approach on student outcomes in a graduate- level public health course. BMC Medical Education, 14(1).
Knowlton, D. S. & Simms, J. (2010). Computer-based instruction and generative strategies: Conceptual framework & illustrative example. Computers in Human Behavior, 26(5), 996–1003.
Kock, N. 2006. Systems analysis & design fundamentals: A business process redesign approach. Sage Publications India.
Kyei-Blankson, L. & Godwyll, F. (2010). An examination of learning outcomes in HyFlex learning environments. In Proceedings from E-learn: World Conference on E-learning in Corporate, Government, Healthcare, and Higher Education (pp. 532-535). Chesapeake, VA: Association for the Advancement of Computing in Education (AACE).
McKay, E. (2000). Instructional strategies integrating the cognitive style construct: A meta-knowledge processing model (contextual components that facilitate spatial/logical task performance) (Doctoral dissertation). Melbourne: Deakin University. Accessed 12 February 2010. Retrieved from http://tux.lib.deakin.edu.au/adt-VDU/public/adt-VDU20061011.122556/.
Nawaz, A. & Kundi, G. M. (2010). Digital literacy: An analysis of the contemporary paradigms. International Journal of Science and Technology Education Research, 1(2), 19–29.
Porto, S. & Aje, J. (2004). A framework for operational decision-making in course development and delivery. Online Journal of Distance Learning Administration, 7(2), 199–215.
Rayner, S. & Cools, E. (2011). Style differences in cognition, learning, and management: Theory, research, and practice. New York: Routledge.
Riding, R. & Cheema, I. (1991). Cognitive styles: An overview and integration. Educational Psychology, 11(3–4), 193–215.
Riding, R. J. & Rayner, S. (1998). Cognitive styles and learning strategies: Understanding style differences in learning and behaviour. London: D. Fulton Publishers.
Rob, M. A. (2006). Development of project documentation: Key ingredient in teaching Systems Analysis and Design. Issues in Information Systems, 7(1), 83-87.
The Standish Group. 2004. Accessed 10 November 2008. Retrieved from http://www.standishgroup. com/sample_research.
Thomas, P. R. & McKay, J. B. (2010). Cognitive styles and instructional design in university learning. Learning and Individual Differences, 20(3), 197–202.
Topi, H., Valacich, J. S., Wright, R. T., Kaiser, K., Nunamaker Jr, J. F., Sipior, J. C. & De Vreede, G.-J. (2010). IS 2010: Curriculum guidelines for undergraduate degree programs in information systems. Communications of the Association for Information Systems, 26, 18.
Urtel, M. G. (2008). Assessing academic performance between traditional and distance education course formats. Educational Technology & Society, 11(1), 322–330.
Zhang, L.-F. (2004). Field-dependence/independence: cognitive style or perceptual ability?–– validating against thinking styles and academic achievement. Personality and Individual Differences, 37(6), 1295–1311.
PhD scholar, School of Business Information Technology and Logistics, RMIT University Melbourne, Australia
PhD, associate professor, School of Business Information Technology and Logistics, RMIT University Melbourne, Australia
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
The Copyright Owners of the submitted texts grant the Reader the right to use the pdf documents under the provisions of the Creative Commons 4.0 International License: Attribution-Share-Alike (CC BY SA). The user can copy and redistribute the material in any medium or format and remix, transform, and build upon the material for any purpose.
1. License
The University of Silesia Press provides immediate open access to journal’s content under the Creative Commons BY-SA 4.0 license (http://creativecommons.org/licenses/by-sa/4.0/). Authors who publish with this journal retain all copyrights and agree to the terms of the above-mentioned CC BY-SA 4.0 license.
2. Author’s Warranties
The author warrants that the article is original, written by stated author/s, has not been published before, contains no unlawful statements, does not infringe the rights of others, is subject to copyright that is vested exclusively in the author and free of any third party rights, and that any necessary written permissions to quote from other sources have been obtained by the author/s.
If the article contains illustrative material (drawings, photos, graphs, maps), the author declares that the said works are of his authorship, they do not infringe the rights of the third party (including personal rights, i.a. the authorization to reproduce physical likeness) and the author holds exclusive proprietary copyrights. The author publishes the above works as part of the article under the licence "Creative Commons Attribution-ShareAlike 4.0 International".
ATTENTION! When the legal situation of the illustrative material has not been determined and the necessary consent has not been granted by the proprietary copyrights holders, the submitted material will not be accepted for editorial process. At the same time the author takes full responsibility for providing false data (this also regards covering the costs incurred by the University of Silesia Press and financial claims of the third party).
3. User Rights
Under the CC BY-SA 4.0 license, the users are free to share (copy, distribute and transmit the contribution) and adapt (remix, transform, and build upon the material) the article for any purpose, provided they attribute the contribution in the manner specified by the author or licensor.
4. Co-Authorship
If the article was prepared jointly with other authors, the signatory of this form warrants that he/she has been authorized by all co-authors to sign this agreement on their behalf, and agrees to inform his/her co-authors of the terms of this agreement.
I hereby declare that in the event of withdrawal of the text from the publishing process or submitting it to another publisher without agreement from the editorial office, I agree to cover all costs incurred by the University of Silesia in connection with my application.
