Research in Learning Technology

ALT's Open Access Journal

Introduction

As teachers are constantly engaged in various academic activities such as preparing learning materials, quizzes, assignments, group discussions, and news forums to organise a better learning environment, supporting them in such endeavours is an indirect form of sustaining a student's learning ability (Gaudioso, Olmo, and Montero 2012). A teacher can use a mobile-based authoring tool to create an interactive learning environment using a mobile phone to teach theories and obtain assessments. Such authoring tools assist the teachers to evaluate students’ performance, record and monitor academic progress, provide advice and communicate with the students in a class (Virvou and Alepis 2014). Moreover, the mobility and instant communication capabilities of mobile devices are the two most important factors affecting the learning process in accessing information, transferring data and interacting among the users in real time (Shih, Chu, and Hwang 2011).

As the motivation and the pressure are considered to be major factors that impact education and learning in higher educational institutions, the combination of mobile technology and Internet-based services such as short messaging service (SMS)+email or SMS+online forums will certainly enhance the motivation of students’ learning without causing undue stress (Lai et al. 2014).

Activity-based learning theories associated with mobile technology list out six broad areas in new learning and teaching practices, namely, learning and teaching support, behaviourist, informal and lifelong, constructivist, situated and collaborative (Jonassen and Land 2012). Acquiring and applying knowledge continuously is a lifelong learning process that can occur at any time anywhere. Students’ assessment and grading mechanism are vivid goals that should be embedded in the learning process for them to pursue knowledge in lifelong learning (Arrigo et al. 2013). Establishing a conversation between different knowledge systems can be referred to as conversational learning in which people practice learning via questioning, answering and sharing the knowledge among each other (Sharples, Taylor, and Vavoula 2010). As the present mobile technology, with its enhanced technical capabilities, can make a profound impact on learning, the combination of mobile technology and learning theories sprouts ubiquitous learning environments. As a result, the concept of learning is shifting towards learners’ environment from the classroom and as such, seamless integration of mobile technology to learner-centred education has become a challenging task that requires a thorough understanding of both theoretical and technical aspects of mobile learning (Jonassen and Land 2012).

Unfortunately, the best practices to be followed when using mobile technology in a learning environment are still not defined at all. In this context, a systematic investigation of students’ and teachers’ experiences in using a mobile learning platform will definitely provide us an opportunity to overcome the challenges encountered in a classroom setup (López-Pérez, Pérez-López, and Rodríguez-Ariza 2011).

SMS is one of the best qualitative mediating tools adopted in the arena of mobile communication pedagogically, economically and technologically in the past decade (Moura and Carvalho 2010). Recent reports support this claim in that more than 6.6 trillion text messages were sent out over carrier networks worldwide in the year 2010 (Whitfield 2010). The use of mobile technologies for distance education has become a popular tool in higher education across the globe due to its ever-increasing demand for student-centric learning (Brett 2011). In this scenario, an SMS can be established for university students as a supportive tool to capture students’ retention in the existing learning management system (LMS) that provides a blended learning environment (Wanja 2014). SMS-based tools can be implemented in an in-class or after-class environment to encourage students’ interactivity. Such a tool will be of immense assistance to teachers to view messages and form interactive message loops verbally with the students throughout the classroom and also during online sessions done after class (Lauricella and Kay 2013).

A majority of university students use mobile phones for routine voice and message communication. Consequently, SMS can be used as a trusted technology to form a better learning environment having a huge potential in higher education with active (sending a message) and passive (reading a message) interactions (Moura and Carvalho 2010).

A mobile platform, referred to as University Mobile Learning Environment (Premadasa and Meegama 2013), has the ability to offer its services to a larger segment of users than existing web-based learning environments via mobile devices. Although there are number of web-based paid text message services available in the Internet at present that facilitate sending secure messages through an LMS, such paid services restrict the number of outgoing messages as well as the number of users (DeWitt, Alias, and Siraj 2013; Ismail and Azizan 2012). Teachers who work with this LMS can send messages to a relevant course unit so that students who are currently logged in will receive this message on their mobile devices. The two-way text message interaction is complete if the student who receives a message relevant to a particular course is able to send back a reply to the same course unit from where the message was originated.

A standard text message does not fully support two-way texting when there is a necessity to include a part of the message received in the reply body. This has hampered two-way communication within a mobile community when one entity uses a mobile gateway (Aranyász 2012) to send out messages using a unique GSM number shared by all the users.

This article presents a two-way text messaging system by concealing data in the message body using the user data header (UDH) to include a part of the message received in the reply body while an encoding and a decoding technique in the mobile phone is used in the background to support such a texting mechanism. The proposed novel mechanism creates a campus-wide interactive mobile learning environment for teachers and students to obtain students’ assessments and grading process.

The proposed system is a combination of Java 2 Micro Edition (J2ME) (Belvin 2011; Ericsson 2011) and Personal Home Page (PHP) technologies. The reason for using such emerging technologies is that the PHP and J2ME combination facilitates integrating the proposed framework to a widely used LMS (such as Moodle) (Pearce 2011).

Related work

Khan and Beg (Khan and Beg 2011a) have presented a methodology where stereo images can be sent through a number of consecutive short messages over a GSM network. A stereo image is converted into characters and these characters are set as the payload text of the short message. This payload text is sent as a concatenated short message on a specified port of the mobile device.

A method concerning multiple-choice quizzing over concatenated messages is revealed by Galih, and Sodiq (2011) where students receive questions via short messages and after answering these questions, the results can be obtained in the client side. In this research, the authors have used steganography in a picture message to hide quiz answers. There are apparently two programs running on instructor-side and student-side where the instructor prepares and sends quizzes over messages and answers over steganography picture message. The student in turn takes the quiz, answers it and sends answers back to the instructor. Chung, Hung, and Chow (2014) proposed two integrated SMS frameworks in their blended learning environment to provide an interactive e-learning facilities among lectures and students. One framework is SMS which is for homework, references and class events. The other SMS framework is for creating pop-up quiz and short-answered examinations.

The Open University Malaysia has implemented such a messaging system to prepare text-based quizzes for students (Ismail and Azizan 2012) where a client messaging server is connected via a GSM modem with related software to administer these text-based quizzes. Khalid, Sujan, and Haque (2011) have also implemented a similar message-based system in Bangladesh to attend text-based multiple-choice questions (MCQ). In contrast, our proposed system is a secure, open source short messaging system that can be easily integrated into the Moodle LMS for student assessment where the cost involved is only for sending messages. In recent literature, as presented in the previous section, applications were developed based on concatenation of messages and also complex data conversion techniques mainly to send voice, image and animation as well as to perform MCQ tests only between two mobile devices where large amounts of messages are required, thereby increasing the cost of implementation. Our proposed system is capable of achieving the same functionality of previous methods with a minimum number of messages exchanged between the communicating devices using a specific port number with associated software to exchange necessary information as concealed data. For this purpose, an open source SMS gateway, a messaging daemon and specific port associated software in a mobile device are used. The system is fully functional at any time anywhere between both parties (a teacher and a student) in a secure, cost-effective and time-efficient manner, providing a mobile learning environment for assessing students’ work.

Theoretical framework

It is traditionally known that short message service centre (SMSC) performs a ‘store and forward’ mechanism for sending and receiving messages (Ryan 2011). The mobile originated (MO)-SMS is transported from a mobile station to the SMSC where these MO-SMSs may be destined to other mobile stations or other services. In contrast, the mobile terminated (MT) SMS is transported from an SMSC to a mobile station. The MO-SMS is transferred from a mobile phone to SMSC by composing it with SMS-SUBMIT protocol description unit (PDU) format and similarly, an MT-SMS is received at a mobile phone in SMS-DELIVER PDU format from the SMSC (Golde 2011).

SMS-SUBMIT PDU

As depicted in Figure 1, the SMS-SUBMIT PDU can be expressed with values as with service centre address and the rest of transport protocol data unit (TPDU). The PDU format is a hexadecimal encoded binary format, which means that two hexadecimal digits represent a byte of data. A header with control information and user data make up a TPDU (Butau 2010).

[Figure ID: F0001] Figure 1. SMS-SUBMIT protocol description unit (PDU) with values.

User data header

As illustrated in Figure 2, the UDH is divided into two parts, the first part, which is only one byte in length, is identified as a user data header length (UDHL) of the UDH. Following this UDHL, an information element (IE) instructs the mobile phone to perform some action based on the information received. The common information element identifiers are port number addressing (representing byte 05) and message concatenation (representing byte 00). Each port number is addressed to a specific port in the phone that provides clear instruction on putting together two or more messages to make a single message during concatenation of several messages.

[Figure ID: F0002] Figure 2. The architecture of a message with user data header (UDH).

The second byte of the IE is the information element data length that tells the phone how many of the following bytes are part of the information identifier. The subsequent bytes of the IE are the actual information being conveyed and is referred to as the information element data (Golde 2011; Ryan 2011).

System function and architecture

As illustrated in Figure 3, the proposed system enables a teacher to post messages to students, to establish an assignment session via a mobile browser interface through an Internet-enabled mobile device. Once a teacher logs in to the system, a message relevant to a particular course (each course has a course code) is created with a maximum of 153 characters (this is a standard character length for a single text message with a UDH). Thereafter, this message is inserted into a database where an SMS daemon checks the database periodically and sends out stored messages through an SMS gateway to a GSM modem. The modem in turn sends out SMS messages to all the recipients with access to that particular course unit through the SMSC.

[Figure ID: F0003] Figure 3. The architecture of the proposed system.

The course code, student number, time allocated for the assignment and the character length allowed of the answer are wrapped automatically by the system. The information about the course code is extremely important for the students to identify from which course the message is originating from as several courses are available in the LMS and, moreover, a student may receive multiple messages from each course unit. The student number is also important to the teacher for identification purposes. A student may receive this message-based assignment via specific port-associated software installed in the mobile phone. A student can accept this assignment and generate an answer depending on the time duration and character length defined by the teacher. While the answer is sent back to the system as concatenated messages (if the answer is more than 153 characters), the course code and the student number are sent as concealed data by wrapping them inside the message body itself.

The theoretical concept behind this system functionality is built upon the convention of port number addressing and the UDH of the SMS PDU. In the standard format, in contrast to an email message, a text message cannot enclose the full or part of the original text message in the reply text. The mobile device only recognises the sender's phone number, which is displayed in the reply message. As such, it is required to wrap information about the course code, student number, time allocation for the assignment and the character length of the answer within the message, and this piece of information should come back to the system with the reply message to be categorised into different course units as well as user identification.

The port number addressing technique with a destination port number of the UDH can be used to perform this concealment of additional data in the message. Such a text message is not directly saved in the standard INBOX of the mobile device. Rather, the mobile application software installed in the recipient's phone extracts this message having a specially defined destination port number. Course unit details (say COM321), student number (say EA1234), time allocated for the assignment (say 5 minutes) and the character length of the answer (say 459 characters=3 messages) are added automatically by the system to the body of the message to be identified at the recipient's end. Based on this information, for example, an assignment containing ‘Question #1: What do you mean by subnetting in network?’, can be expressed as in Figure 4 where the SMS-SUBMIT PDU converts a message to a PDU format using the port number addressing (destination port 50,000 is converted to C350 in Hex). This will be the PDU format of the message that is sent from the database by the daemon.

[Figure ID: F0004] Figure 4. The theoretical concept behind the system functionality.

Details pertaining to the course unit are compared with the database to acquire a unique course code of the relevant course unit. While this function occurs in the background, the course code, student number, time allocated to the assignment and the character length of the answer are appended to the message body by the system. At the next stage, the complete SMS-SUBMIT PDU is sent to the SMSC to be forwarded to the student within a valid time period. The student then generates an answer as a reply message with the course unit details (i.e. COM321) with the student number (i.e. EA1234) and sends it back to the database. This is done by the J2ME application installed in the mobile device, which captures the incoming message and acquires the course unit information (i.e. COM321) and the student number (i.e. EA1234) contained in the message body. While the student is ready to accept an assignment and create an answer as a reply message, the application software automatically conceals the information of course unit and student number securely in the background to the reply. The student, however, only sees the phone number of the message originator as well as the reply text and does not see course unit or student number information.

In order to implement the proposed framework, it is essential to have mobile devices that support the UDH concept. When the mobile application is installed, another message INBOX, in addition to the standard INBOX, appears in the menu. All the messages originating from the proposed system are delivered to this newly created INBOX. Figure 5 depicts the interfaces of the mobile phone giving different functionalities of the proposed system.

[Figure ID: F0005] Figure 5. The interface of the mobile phone with (a) message received, (b) question received, (c) answer typed, (d) alert and (e) grade received.

Results and discussion

Students’ feedback is a vital factor in evaluating the efficiency, effectiveness, educational impact and benefits of the proposed system in an academic environment. Two groups of students, from sciences and non-sciences, familiar with traditional paper-based assessment method, were asked to fill out two questionnaires within a realistic timeframe according to pre- and post-usage of the proposed system. Initially, students in a class answered a question posed by a teacher who then marked answer scripts and gave grades to each script. At first, a pre-usage questionnaire was given to them after completing a class-room-based learning process with the teacher. Subsequently, the teacher was given an introduction to the proposed system who posed the same question to the same batch of students via a text message. Students, in turn, replied back with the answers for assessment by the teacher. Later, students received their grades immediately through text messages.

A few common problems encountered during class room assessments in a university environment can be summarised as follows:

1. Students may have to come from a faraway distance to the class room on time to attend assessments
2. Student attendance
3. Delay in receiving grade of the assignment

The analysis was carried out using both samples as science (50 students) and non-science (50 students) separately at first and, finally, as an overall sample (100 students). A score value (−10=strongly disagreed, −5=disagreed, 0=neutral, 5=agreed, 10=strongly agreed) was assigned to each option in the list of answers at the pre- and post-usage survey.

Pre- and post-usage intervention surveys

Pre-usage survey results

Both groups of students were questioned to identify the usage of mobile technology for education prior to their admission to the university. At the high school level, a majority of them (84%) has used mobile phones mainly for voice calling and sending text messages, but hardly for educational activities. However, this mobile technology usage among the students has increased to 96% once they are enrolled at the university. Thus, it reveals that the social preference of mobile phone usage among university students when compared with their pre-university education.

Student evaluation process is one of the most important tasks that can be used to motivate students in their learning endeavours. Evaluation and subsequent grading of quizzes, assignments and examinations should be performed without any undue delays. In this scenario, teachers reported that they need approximately 2–5 days to complete the evaluation and grading process of a group having 100 students who submitteda paper-based assessment. Initially, the teachers evaluate each answer script and assign a grade to each. Then, the grades of each student is printed on a pre-prepared mark sheet and placed on a notice board for display. The pre-usage survey results indicate that only 13% of the students agreed with the paper-based evaluation and grading process and they had clearly indicated long delays in receiving grades.

Moreover, student attendance during compulsory assessments is always a critical issue with traditional paper-based method where students give various unacceptable excuses for being absent. During this survey, 83% of the students were present for the paper-based assessment while the remaining 17% (absentees) had given various reasons for their absence. The teachers who were engaged in this survey reported that some reasons given by the students for being absent are totally unacceptable. According to prevailing regulations in the university, teachers need to prepare another follow-up assessment (referred to as a ‘make up’ assessment) for the absentees. This places a further burden on the teachers who are presently loaded with heavy academic work and research. The secondary data collected during the past 3 years show that the attendance of such compulsory assessments is 81%, 85% and 83%, relatively somewhat equal in each year.

Post-usage survey results

The paired sample t-test (parametric) is carried out to analyse the acceptance of the proposed mechanism by comparing data obtained during pre- and post-usage intervention surveys with a same sample set of students. The post-usage survey is mainly focused on observing the usage and the satisfaction level of the end users of the proposed mechanism.

After defining a null hypothesis (H ₀), a paired sample t-test (parametric) is applied to compare the two final score values. H ₀ was accepted by considering the p-value of the test (i.e. if p <0.05, then reject H ₀ or accept otherwise).

Assume a null hypothesis H ₀ as:

H ₀=Students are not satisfied with the SMS-based assessment mechanism.

Table 2 provides statistical results obtained after applying the paired sample t-test. Questions focused on pre- and post-surveys are verified using a five-point Likert scale as shown in Table 2 in which for the overall sample as well as for each specific sample, the related p-value was <0.05. This results in H ₀ being rejected and H ₁ accepted. These results verify the students’ satisfaction in using the proposed mechanism for assessments is significantly higher than traditional paper-based method.

Table 2. Results of paired sample t-test for proposed mechanism.

Sample	Variable	N	Mean	Standard deviation	p
Applied Science	Pre	50	4.95000	4.17994	0.023
	Post	50	3.20000	4.81918
	Difference	50	1.75000	6.02397
Social Science	Pre	50	3.45000	5.07319	0.039
	Post	50	1.30000	6.60936
	Difference	50	2.15000	8.42146
Overall	Pre	100	4.20000	4.68557	0.004
	Post	100	2.25000	5.83333
	Difference	100	1.95000	7.28722

The post-usage survey data are plotted in consistent with a normal distribution curve as shown in Figure 6. The median value can be used to describe the data set by variation in the interval for the median between 0.0000 and 5.0000 with a 95% level of confidence. These post-usage survey results indicate the students the acceptance of the proposed SMS-based assessment mechanism.

[Figure ID: F0006] Figure 6. Distribution of post-usage survey data to evaluate the reliability of the proposed mechanism.

However, it is observed a 96% attendance when the assessment is conducted using the proposed mechanism. The absentees (4%) had honestly reported that they had not received text messages containing the assignment because they are unaware that this text message is saved in a separate message box. While a significant percentage (96%) was present in the class during the mobile-based assessment, 3.125% (three students) did not complete the assignment due to delays in responding (for instance, if a student is unable to reply back an answer within a given time frame, the software in the mobile device automatically discards the reply message). This means the student had accepted the assignment but did not complete it. These drawbacks could be mitigated once the students get themselves familiarised in using the proposed system with time management skills.

User-friendliness and cost-effectiveness of the proposed system

The following crucial dimensions were used to evaluate the efficiency and effectiveness of proposed system. They are:

1. User-friendliness
2. Cost-effectiveness

Table 3 illustrates the post-questionnaire, providing three questions on each of the above two dimensions.

Table 3. Questions relevant to each dimension.

User-friendliness
(1) It is easy to use mobile devices in this proposed method.
(2) This text-message-based assessment experience is encouraging.
(3) I would like to obtain assignments continuously with the proposed mechanism instead of the traditional method.
(4) Given instructions in the assignment process are formative and easy to understand.
(5) I would recommend this technique as a method of assessment to other students as well as teachers.
Cost-effectiveness
(1) Proposed system is accessible with existing mobile devices.
(2) Cost per text message is affordable in replying back with answers.
(3) Other related costs can be minimised due to anytime anywhere accessibility.

The students were asked whether it was convenient to use mobile devices with the proposed method. The responses were satisfactory with an overall mean score of 6.10 in agreement (user-friendliness – question 1 as in Table 3). In this trial, participants from the non-science stream accounted for a mean of 5.50 while it was 6.70 for the science students. This can be attributed to the fact former group's unfamiliarity with mobile device while using the proposed system. However, when taking into account the overall mean score, the participants had responded in the affirmative.

A high mean score of 7.40 from non-science students indicates a satisfactory response with this proposed technique rather than the science students who scored a mean of 5.30 for the second question for user-friendliness as in Table 3. The students did not wish to participate in conventional face-to-face interactions with the teacher, as the proposed technique would have given them a novel experience in their learning environment. The science students, however, had given several uncertain responses to this question which may have been due to the familiarity of such technological aspects in learning. This fact was later verified during follow-up discussions with the students.

In the third question in the scale of user-friendliness as given in Table 3, the participants were asked whether they would like to continue their assignments through the proposed system instead of the traditional method. A high mean score of 7.1 from science students’ indicates a satisfactory response to continuing the proposed system in future for assignments instead of traditional method while non-science scored a mean 4.9. However, when considering an overall mean score of 6.0, the participants somewhat agreed to continue their assignments with the proposed system than the traditional methods. This result differentiate the impact of technology acceptance and interest among science and non-science students in their educational activities. When made an after discussion with two groups, most of non-science students responded that the proposed mechanism is an extra burden into the mind-set on their assessments because they need to pay attention for the technological aspects of the proposed system than paper-based method. However, most of the students in science group responded that the proposed mechanism is much more effective on their tough learning schedule.

The high mean score 7.25 from the overall students indicates the satisfactory response to the formative and understandable instructions of the entire assignment process when using the mobile devices. Especially non-science students also scored a mean value of 6.7, while science students scored a mean value of 7.8. In the later discussion made with two groups, students mentioned that the given instructions in the assignment process are clear, formative and easy to understand. The reason behind this high value of mean scores is the acceptability of given instructions in the proposed mechanism.

In the fifth question associated with user-friendliness as given in Table 3, the participants were asked whether they would recommend the proposed technique as a method of assessment to other students as well as teachers. Fourteen (14%) of the participants in both groups were uncertain in responding to this question as this was the first time that many (78%) of them ever had an experience with a mobile device in a learning environment. Also being not exposed to such an assessment method in a previous occasion may have influenced this result and, hence, a mean score of almost 5.00 is observed in both groups, individually as well as when taken as an overall sample. However, when considering an overall mean score of 5.55, the participants somewhat agreed in recommending the proposed system as an assessment method in a learning environment. This indicates that social attitudes among the students need to be changed in using such novel technologies and also further improvements are required in the proposed mechanism to ensure a wide acceptance in pursuing mobile-based assessments.

The proposed system is accessible by a wide range of mobile devices into which specific software associated with port number addressing is installed. Presently, the students possess mobile devices with sufficient technology to execute the proposed assessment mechanism and, therefore, they entirely agreed with the system's accessibility using existing devices (the first question in cost-effectiveness as in Table 3 with an overall mean score of 7.85).

The length of a text message (answer) sent by a student during an assessment is a vital factor affecting the affordability of the mechanism. Presently, the teacher limits the number of text messages that can be sent by a student for a particular assessment to three (this can be varied). As the cost in sending a text message is considerably low (around US$ 0.002), replying back the answers with a maximum of three text messages is affordable to the students. An overall mean score of 6.05 for the second question in cost-effectiveness as given in Table 3 indicates the students’ agreement with the cost per text message to be paid to the service provider.

With the proposed mechanism, the students and teachers do not have to personally come to a class room, which take up time and involves a cost as it requires travel through a mountainous region. As such, this mechanism is ideal to cut down expenses incurred by both students and teachers especially during travel. This fact generated an entire agreement with the proposed procedure as depicted by the third question in cost-effectiveness given in Table 3 with an overall mean score of 7.70.

The responses given for the three questions related to cost-effectiveness given in Table 3 highlight the enormous potential of the proposed system in minimising the cost of an assessment. Not only the cost of a text message, but also the university as a whole, is able to save costs associated with stationery, electricity, water, janitorial services, etc. that are required to conduct a class-room-based assessment. Figure 7 depicts the mean scores on user-friendliness and the cost-effectiveness of the proposed system.

[Figure ID: F0007] Figure 7. The mean score on Likert scale for the user-friendliness and cost-effectiveness.

Conclusion

A novel text messaging technique, integrated into an LMS, offering two-way texting is presented in this article. The proposed framework provides a more flexible and convenient way of passing messages between the users than a traditional web-based e-learning platform. The pilot survey carried out with 100 messages reveals performance of the system in sending messages with minimum delay.

The system has made a profound impact on improving the learning environment of a campus located in a mountainous region. The system has benefitted both the students and staff, especially the members of this community having physical disabilities who find it difficult to negotiate sharp slopes and ridges encountered when visiting different schools to meet up with peers.

The technology reported here would also be useful in any situation where many users must share a single phone number. Since almost all students use mobile phones for texting already (ESCAP 2011), the proposed two-way texting mechanism would provide a useful facility in the educational sector.