salvetore

The blog of Michael de Raadt


Leave a comment

A New Battleground

From the static Web to dynamic mobile browsing

Moodle 1.9

Moodle 1.9

In the beginning, when Learning Management Systems (LMSs) were young battlers, Moodle came about as a combatant that succeeded through its stubborn simplicity. Other LMSs attempted to overload interfaces with Java to achieve an edge. Moodle, on the other hand, stuck to standard Web interfaces to achieve the same result. The result was that Moodle was considered simpler and more user-friendly. If you knew how to use a Web browser, you could use Moodle; you didn’t have to have any additional browser plugins installed. Moodle’s usage grew rapidly, overtaking its competition, because people could understand it.

With the advent of AJAX and JavaScript libraries, the Web has started to move beyond being a place to present content and has become a place where applications (once bound to the desktop) now live. JavaScript, having existed for many years, is finally being used to its fullest potential, allowing greater interactivity on Web pages. AJAX is a means of communicating between a browser and a server without a page reload, in other words, things are happening in the background without the user needing to leave the current page. The Web is now truly dynamic; this new Web is the place LMSs were meant to be. The challenge LMSs face, is to achieve simplicity, while adding dynamic elements in an intuitive and unobtrusive way, always maintaining usability and accessibility.

LMSs are also being used beyond the desktop. Now that we are finally seeing consistency among desktop browsers,  developers are faced with a new challenge in the form of mobile devices. The standards set for the Web are still followed (although I think a mobile browser war is just getting started), but the physical interface to the browser is different on mobile devices. No longer can we rely on users with a mouse, keyboard and monitor; the Web has to work with touch interfaces also. We aren’t even afforded the luxury to assume a reasonable minimum screen size.

A new battleground

I have been involved in the bureaucratic effort to select a new LMS for a university. Battle was fought by lining up each LMS candidate side-by-side against a set of features. The LMS with the most checkmarks next to its name was the victor. Moodle won this battle many times because it was well featured. If the feature didn’t exist in the standard distribution, there were add-ons to supplement it, and if that wasn’t enough, you could always customise. The other thing Moodle had going for it was its underdog status, which I’ve talked about before.

About two years ago, at the 2011 Australian Moot, I sensed a new set if biases creeping into the public consciousness. No longer were people asking for more features, instead they were wanting style and speed. Does this mean Moodle is feature-complete? Probably not, but at least most people seem satisfied with the current feature set and seem to have shifted priorities. A new battleground has been forming in my mind in the last couple of years.

So what is Moodle doing to arm itself for this new battleground? Here are some newish additions to Moodle’s arsenal.

Simpler Editor

People spend a lot of time in Moodle using the editor. The WYSIWYG editor has been around from very early in Moodle’s history, but now it’s is being simplified. We’re still using TinyMCE for now, but keep your eyes open in future for a brand new, home-grown editor alternative that will be slicker still.

New editor look

Drag-and-drop

The most obvious use of AJAX and JavaScript is drag-and-drop and Moodle now uses this in a number of places. You don’t have to leave the course page to add a file resource (“dump-and-pump” is so much easier). If you want to rearrange the placement of activities, resources or blocks, no page reloading is needed. In a similar way, if you want to rename something, you don’t have to edit a whole page of settings. Where ever a file is needed, if you can drag it into the file picker, that will get the file uploaded quickly.

Drag-and-drop

Access to the world’s data

Repositories are sources of files. They could be files on your computer, files on the institution’s server, files from the Web or files from “the cloud”. This concept seemed to stump some people at first, but it is now starting to make sense. At the advent of Moodle 2.0, there were a few teething problems with repositories, but this part of Moodle has settled down into something smooth and reliable.

Repositories

An interface that works on anything

Apparently students and teachers have new-fangled mobile devices now, and they want to access their Moodle sites on these devices. Responsive themes allow a single Web interface to react to different screen sizes. On a large screen, the view is not too different from the standard theme, with a few rounded edges. On a small screen, things are rearranged: menus collapse into icons, blocks shift to below content and pop-ups fill the screen. There are a number of other changes that the use of touch devices have promoted as well. Not only is the interface becoming more usable on different devices, it’s also becoming more accessible to users with disabilities.

Moodle on a small screen

Is it working?

Well, none of the things I’ve mentioned above appeared on the feature list a few years back, so are they needed now? There are a large number of registered sites still on 1.9 – why? Is it a case of “If it ain’t broke, don’t fix it”, or is the simplicity of older Moodle versions still more attractive to some users? Change, it seems, happens slowly in the world of education. Change can be dramatic for people.

When Mary Cooch conducted some training for existing Moodle users at Our Lady’s Catholic High School, the new interface was different enough that they did not recognise they were still using Moodle. One participant’s response was that the new system was “Unbelievably simpler than Moodle!” Others had similar comments, and even though it’s a small sample size, I think we can see that as evidence that Moodle is getting simpler.

The battleground of the future

The battle goes on.

The question now is where the battles of the future will be fought. Predicting the future is precarious, and I’m undoubtedly going to be proven wrong, but I have to speak at a conference next week, so I’d better come up with some ideas that sound slightly visionary.

Scale

learn.moodle.netMassive Open Online Courses (MOOCs) are a hot topic at the moment, with large courses being offered online to anyone willing to participate. Many are anticipating that MOOCs will have an impact on the future of higher education. Moodle has recently conducted what could be seen as an experimental step into the MOOC world. Check out learn.moodle.net.

Massive is big, but is there something bigger. Moodle and other LMSs have traditionally focussed on tertiary education and corporate training. There is a smattering of use in primary and secondary education, but it is limited to a relatively small number of classrooms. However when you compare the student numbers and budgets of these sectors side-by-side, primary and secondary education dwarf the other sectors. So why are LMSs not being used widely in primary and secondary education. I believe the answer is that primary and secondary teachers are not well supported and have less time to attempt such ventures than their colleagues in higher levels of education. Where LMSs could start to become useful is through large-scale integration at state or federal levels. If an LMS is set up where the curriculum is defined, teachers would be freed of the laborious tasks of gathering resources, establishing assessment and conducting grading. Instead they would be free to focus on what they do best: teaching.

End of one-size-fits-all education

At almost any level of education, once the class grows beyond a handful of students, necessity prevents teachers from implementing individual learning plans. The burden of assessing students regularly enough, measuring their performance and adjusting the curriculum to suit them becomes nearly impossible. But that is where LMSs can help. At the moment providing an individual path through a curriculum that automatically adjusts for a student is possible, but it is cumbersome. Hopefully we can improve on that in the future.

Advertisements


3 Comments

Incentives for Developers

How do you encourage developers to be more productive?

A few months ago, I was intrigued by a presentation by Dan Pink, an American public speaker. Here is a version of that presentation (and there are a few similar presentations around, including a TED talk).

In the presentation, Pink claims that extrinsic motivators, specifically financial incentives (bonuses, raises, promotions, stocks,…), can be counter-productive to the goal of encouraging workers in certain circumstances. In the presentation, Pink refers to studies at MIT, so I went searching for publications for these studies and found Ariely (2005) and Awasthi & Pratt (1990).

While people can be motivated by financial incentives, the studies found that financial incentives can reduce performance for tasks involving a cognitive component. Software development certainly involves cognitive tasks, in fact programming is about as cerebral as you can get.

So if money doesn’t work, what does? Pink’s thesis is that employees will be more productive when they have a sense of:

  • autonomy,
  • mastery and
  • purpose.

Pink refers to cases at Atlassian and Google, where employees are reported (in the media) to receive many perks. I’ve been to Google, and while I did enjoy the free food, the work environment was certainly not anarchistic, in fact it seemed quite ordinary on the inside. What Pink emphasises is that these companies offer a degree of autonomy to their workers, that employees have the potential to develop professional masteries for their current job and for future jobs, and that employees are able to see a sense of purpose in what they do day-to-day.

Developer Incentives at Moodle?

Some aspects suggested by Dan Pink were already in place at Moodle, but some have been added or enhanced in recent months. I will describe how we offer a sense of autonomy, master and purpose to members of the STABLE team at Moodle (the devs who work on the existing releases of Moodle).

Autonomy

Apart from being a relatively relaxed working environment, there are some specific differences that may set Moodle apart from other development offices.

  • Devs choose, set-up and maintain their own development environments. Code meets at the repository, but how it gets there is up to the developer.
  • Using the Scrum framework, devs choose issues they will resolve from a prioritised backlog of issues. This ensures that the highest priority work gets done, but devs have a sense of ownership over, and responsibility for, the issues they choose.
  • After every two sprints (sprints are typically three weeks long), devs have a week to work on a project of their own choosing. The projects have to benefit the Moodle community, but is open to interpretation by the developer. This means that one week out of every seven, the developer is completely autonomous.

Mastery

Mastery is an area we could be working more on, but there are a few initiatives in place at Moodle.

  • Devs can nominate external training courses and are supported to attend.
  • Devs nominate areas of interest in Moodle and are allowed to specialise in those areas.
  • Devs receive in-house productivity training . There are also irregular presentations on development related topics related to the current focus of work (for example, in-code documentation writing, Web services, etc.)

Purpose

Purpose is something that Moodle has a lot of. Moodle allows many people to access education, some of whom would not be able to do so otherwise.

In saying that, it is easy to lose sight of that purpose when devs are focussed on lines of code while reading the grumbles of users on bug reports.

It is important t0 regularly remind developers that there is a community out there and they really appreciate the work devs are doing. We have, in the past, dragged devs to a Moodle Moot, where there is a lot of back-patting. We are hoping to do that again this year.

If you are a member of the community and wish to express your gratitude, please do so. Send me an email or post a message on the Moodle forums. It will really help.

Do these incentives work?

From my perspective, I would have to say “yes” – encouraging a sense of autonomy, mastery and purpose does help developers, their progress, as well as the general working environment. It’s hard to quantify the effect of making these aspects more obvious to developers, but I have noted some improvements since we have.

  • Our turn-over of staff is low. The devs seem are content and passionate about their work, particularly when they have a chance to work on what they are interested in. This really helps avoid slacking off when it comes to doing “more of the same”; with sufficient variety, developers are quite happy to switch to unstructured work and then back to structured sprints again.
  • General productivity is higher and being maintained. The number of issues being led through our process has increased and that is a good sign.
  • The STABLE team is producing some significant contributions to Moodle, and not always in the same way. We had a very colourful show-and-tell session last Friday with some very excited developers (including devs from outside the STABLE team). Here are some examples of what was put on show…

An optimised view for the Gradebook (Rajesh Taneja)

There are a number of issues relating to the usability of the Moodle Gradebook, which can become unwieldy. With some simple modifications, the Gradebook becomes a much more usable space.

Optimsed Grader Report

Previews for Database activity uploads (Adrian Greeve)

Currently, uploading data into a Database Activity provides little feedback or control. Adding in a preview, with field matching, allows easier uploading.

DB Activity Upload Preview

A Moodle development kit (MDK) (Frédéric Massart)

The MDK automates many regular dev tasks including Git operations, adding information to issues on the Moodle Tracker and automation of site instantiation and population with dummy data.

Moodle Development kit

This project has been quite a collaborative effort and is still growing.

  • Documentation is available on the Moodle Dev docs.
  • The MDK can be accessed from Github.


2 Comments

Open Technical Debt

What is technical debt?

Technical debt is accumulated future work that has amassed as the result of decisions made during the development of software (or more generally during the development of information systems). The term was coined in 1992 by Ward Cunningham, who realised that as soon as software is created there is an element of deferred work, which can be equated to “going into debt” (Cunningham, 1992).

Analogously, technical debt is often equated to financial debt, such as a loan. The value of a technical debt is not dollars, but the cost of the time needed to rectify problems. As software is created, compromises are made between delivering a flawed but acceptable system now or delaying and delivering a superior system later. These compromises result in a backlog of work that needs to considered before a future release.

Technical debt comes about when developers create less than ideal code. Fowler (2009) suggests that developers can do this deliberately or inadvertently. A developer can deliberately decide to use a quick-and-dirty solution now with the intention of replacing this solution with a better one later. “I’m not sure if that database query will scale, but I’ll write a TODO to fix that later.” Developers can choose to sacrifice non-essential software elements by failing to write documentation, avoiding creating reusable abstractions or failing to follow coding standards. Alternately a developer can inadvertently introduce problems into the code. This can happen through forgetfulness as deadline pressure builds or simply when the skill required to solve a particular problem exceeds a developer’s technical experience. “I’m not exactly sure how this code works, but I’m going to reuse it now as it seems to solve the problem.” This approach is sometimes referred to as Cargo Cult programming (McConnell, 2004).

Fowler also suggests that technical debt can be introduced through behaviour that is either reckless (“I don’t know how significant this is but I don’t have time for it now.”) or prudent (“The cost of introducing this now will be greater than the cost we will incur by delaying our release.”).

By crossing these deliberate-inadvertent and reckless-prudent dimensions as axes, four quadrants appear; these quadrants can be used to categorise the sources of technical debt.

Fowler's technical debt quadrants

Fowler’s technical debt quadrants

As no system is perfect, technical debt is something that cannot be avoided. It is something that needs to be managed rather than ignored.

Is technical debt bad?

Technical debt, like financial debt, is not all bad. Any debt is doomed if there are no means to repay it. “Few of us can afford to pay cash for a house and going into debt to buy one is not financially irresponsible, provided that we know how to pay it back” (Allman, 2012). Projects need to consider the level technical debt they are capable of supporting and be aware of their technical debt at all times, ensuring that it does not exceed this level.

If a software project accumulates more technical debt than can be “repaid”, the quality of the software becomes poorer. If lacking quality reaches a level that is obvious to users, this can affect their decision to use that software in future.

What is “open technical debt”?

Open technical debt is the technical debt accumulated by an open source project. To understand it you have to know that open source projects differ from commercial developments in terms of code ownership, project management and in the philosophy that motivates the project.

Open source software is freely given to a community of users and that community is invited to provide feedback to the project to guide its future. Compared to a software system created by a commercial vendor, where the code ownership is simple, in open source projects the community owns the software and benefits from the effort they invest.

Moodle communityOpen source projects vary in scale from small projects, involving a small number of loosely organised volunteer developers, through to large-scale projects, that are bigger than many commercial software undertakings. The project I am involved in is Moodle, which involves hundreds of developers and has many thousands of registered sites with over 60 million users worldwide. The project employs 25 full-time employees and works with a large network of commercial Partner organisations who deliver services to the community and help support the project financially. Managing such a project is often difficult as there is no single product owner who you call on to make decisions and set priorities.

When technical debt accumulates in an open source project and impacts on the quality of the software, it is obvious to the community. But this is balanced by the community’s sense of responsibility to fix these problems, improve the quality of the software and pay off that technical debt.

In open source projects, strengths can also be weaknesses. The potential of a large community and large number of developers can lead to powerful software, but left unchecked it can also lead to technical debt. If that debt is not recorded and “paid off” it could lead to the downfall of the project.

Where does open technical debt come from?

It is important to be aware of where technical debt is coming from in a project. Using Fowler’s technical debt quadrants it is possible to categorise the sources of problems in open source code.

You might think that most technical debt in an open source project is the result of reckless developers contributing code with inadvertent consequences to the project as a whole. In fact this is quite the opposite of the behaviour that an open source project elicits from developers. When someone contributes code, their code becomes part of the open source and is open to scrutiny by all. Metaphorically, their dirty washing is being aired for the entire world to see. If a problem is later found it is easy to track it back to a change made by a specific developer. This tends to lead to well-conceived code with a sense that reputations are on the line.

As the person responsible for triaging issues as they are reported for the Moodle project I know every freckle and bump in its complexion. On a regular basis I track the sorts of issues that are left in our bug tracker and a large chunk of these are unfulfilled improvement requests. When releases are finalised, decisions are made to “draw a line”, even though improvements could be made. So the technical debt of the Moodle project, as an example of an open source project, is predominantly in the prudent-deliberate quadrant with lots of ideas for making the software better being known but not acted upon.

Technical debt quadrants in open source

Technical debt quadrants in open source

Does this differ from a commercial project? Well I can’t say for sure, but I suspect it does. I would say that closed source software lacks the pressure that openness creates. Also, when the priority setting falls to a single decision maker with commercial deadlines to meet, I think that technical debt would shift more to the reckless side of this field. But then, I’m biased.

Avoiding and embracing open technical debt

While accepting that some technical debt is unavoidable, there are ways that it can be minimised.

Openness of flaws

Having an open bug tracking system allows anyone to see what bugs have been reported and what improvements have been suggested. This means that the extent of the technical debt of a project is on display to all. Being open in this way creates incentives for developers to avoid creating technical debt in the first place, and to reduce technical debt in the long-term. It also shows the community that work is being done in a way that follows defined priorities.

Agility

Following agile software development practices allows developers of an open source project work together to fulfil the priorities of the project. As priorities shift (and they do when you are responding to a community), being agile means that developers can respond quickly. In fact I can’t conceive of an open source project being managed any other way.

Review

Moodle process including reviewContributed code in an open source project is not automatically accepted. Before it is integrated with the codebase it usually has to satisfy experienced developers involved in the project. This is certainly the case at Moodle where all code goes through at least three levels of review before it is integrated into rolling releases and even more before major releases. When this is done politely, as well as ensuring software quality, this also helps to assure contributing developers and instil a sense of confidence.

Modularity

Once any software project grows to more than a trivial size, it needs to be modularised. In open source this is especially beneficial for two reasons.

  • Modularity provides focus points for developers who want to contribute to a project without needing to understand the entire codebase.
  • Modularity allows a project to designate code as official and unofficial. Official code is what is distributed to users as the core project code. Unofficial code can be plugins that individuals have written. Technical debt can then be measured against the official core code while keeping the potential “high-risk” debt of unofficial code “at arm’s length”. That’s not to say that developers sharing plugins should not be supported and recognised.

Willingness to deprecate

As a project develops, changes will occur over time. Often modules become neglected, particularly if no one from the developer community has an interest in maintaining that module. When this happens, the community has to recognise the state of the module and deprecate it. Deprecation is like writing off technical debt; while it comes with a loss of functionality it also notionally frees up resources to focus on other parts of the project.

References

Allman, E. (2012). Managing Technical Debt. Communications of the ACM, 5(5), 50 – 55.

Cunningham, W. (1992). The WyCash portfolio management system. In OOPSLA 1992. http://c2.com/doc/oopsla92.html

Fowler, M. (2009). Technical debt quadrant. Retrieved from http://martinfowler.com/bliki/TechnicalDebtQuadrant.html

McConnell, S. (2004). Professional Software Development. Boston, USA: Addison-Wesley.


19 Comments

Analytics: getting helpful information out of an LMS

At the recent Australian Moot, a hot, emerging topic was “analytics”. Analytics have the potential to help students, teachers and institutions make better choices that lead to improved learning outcomes. Analytics were not possible in traditional education – they are, however, something a learning management system (LMS) can easily provide, if we can just figure out what they are…

What are analytics?

Analytics is a relatively new term; so new, in fact, that my spell-checker doesn’t recognise the word. It’s a real word, though; it must be as there is a Wikipedia entry. The term has been popularised in the online world by Google who offer analytics about your website so you can see details about your site visitors.

But what does the term analytics mean for education in the context of an LMS? At a panel session during the aforementioned moot, the topic was raised and this was the first question asked – what are analytics? One of the panel members, Alan Arnold from the University of Canberra bravely offered a definition: “Analytics are any piece of information that can help an LMS user do their job.”

“Analytics are any piece of information that can help an LMS user do their job.”

Thinking more deeply about the subject, I propose that LMS analytics are useful calculable quantitative gathered collections of information, based on the activities of users within or around the LMS, presented in forms that allow decisions leading to improved learning outcomes. (Quite a mouthful.) There’s lots of information about a user that can be collected in the LMS. The trick is to tease out and combine the useful bits and present them simply.

So the question is not so much “what are analytics?” but is instead “what analyics do you need?” and perhaps “how can you get them?”.

What analytics do you need?

Not all analytics are relevant to all users. If you are a teacher, you’re probably thinking about getting information that can allow you to teach better. If you’re a policy maker at an institution, you’re probably wanting to know how successful your teachers are with their students. Bet let’s not forget the students as well; there is information in the LMS that can help them also.

On the plane back from the Moot I decided it would be worth starting a catalogue of all the different analytics that could be captured in Moodle. At lunch I threw these at Martin and we cranked out a few more.

Analytics useful to Students

Progress
My progress bar blockWith an LMS, it is possible to achieve regular assessment within a course based on a rich set of finely chunked multi-modal activities, and while this can lead to deep learning, it can also be overwhelming for students. It is, therefore, useful for a student to know where they are up to in a course and what they have to do next. Students who use short-term planning tend to be more successful; they just need a quick snapshot of their progress.
Relative success
Deep learners are more successful and deep learners are characterised by meta-cognition about their learning. Providing analytics about their relative success can allow students to know whether they are on track of if they need further exposure to a topic. Relative success can also be used to introduce a competitive element into a cohort, which some educationalists recommend.
Opportunities to interact
If students are studying in isolation, it may not always be apparent when there are chances for them to interact with peers or teachers. Determining the level at which a student is interacting could be seen as an analytic that can be used to direct them to opportunities for communication and collaboration.

Analytics useful to Teachers

Student participation
In an online world, it is more difficult for a teacher to know which students are participating and those needing a push. Students can fail to participate for numerous reasons, usually valid ones. Sometimes a student may need to be encouraged to withdraw from a course and re-enrol later. Where analytics can help is in the determination of the timing of when such decisions need to be made. That’s not to say that such information needs to be complex; it could be as simple as “traffic light” coloured icons next to a list of names of students, ordered by risk.
Student success
Engagement analytics block
Assuming a student is involved, a teacher also wants to know how successful they are. This could be the product of assessment and views of resources. If students are progressing through the course with unsuccessful results, then they may need to be encouraged to re-expose themselves to a topic within the course before progressing further.
Student exposures
Moving away from a course modality where “one size fits all”, it is useful to know how many times a student was exposed to a topic before they were successful. This is a differentiating factor among students in a cohort. If students are progressing with few exposures, perhaps they are finding the course too easy, perhaps even boring, and may need to be challenged further. If students are requiring numerous exposures before they are successful, then perhaps alternate presentations of a topic need to be created to suit the learning preference of particular learners. Such an analytical tool can assist a teacher to deliver learning at an individual level.
Student difficulty in understanding
Through an analysis of exposures and assessment results, it may be possible to determine which topics, or areas within a topic, students are finding difficult. This may indicate areas that need to be revisited in the current delivery or enhanced in a future delivery of the course.
Student difficulty in technical tasks
When students are undertaking learning, the last thing they want is to be stifled by an inability to express their understanding because of by the way a course is set up within the LMS. Students patterns of use within the LMS may indicate they are having such difficulties, and a teacher can be alerted to take action.
Feedback attention
Teachers take time and spend effort creating feedback for students as a reflection of their understanding. It is useful to know which students have paid attention to such feedback, and which students may need to be encouraged to do so. Going beyond this it may be possible to deliver information to a teacher about the effectiveness of their feedback on students’ understandings as reflected in subsequent assessment.
Course quality
In several institutions that I know of, part of the measurement of a teacher’s effectiveness is judged by the quality of the courses they are producing within the LMS, based on a set of metrics. Such measurements can be used for promotions and to drive the development of PD activities. If such metrics can be automated, then analytics can be produced for teachers that encourage them to improve their course by increasing the richness of their resources, improving the quality of their activities, including more activities of different kinds, providing more opportunities for students to interact or collaborate.

Analytics useful to Institutions

Student retention
Analytics can provide more information about students than simple pass/fail rates. Analytics can help determine when students may be at risk of failing and in which courses this is more likely to happen. Such analytics can help an institution to send resources to where they are needed most and to plan resources for the future.
Teacher involvement
There may be ethical implications in monitoring teacher involvement in a course as it is akin to workplace survelance. However there is information in an LMS that can be presented in a useful way in relation to training and promotions. It might also be useful to anonymously tie in a teacher involvement analytic with other analytics to find correlations.
Teacher success
As well as looking at success in terms of pass and fail, it may also be possible to determine where teacher interventions have encouraged students to achieve beyond their expected outcomes.
Relative course quality
Clearly not all courses are equal, but how do you determine which is better. There have been a number of attempts to manually measure aspects of a course such as accessibility, organisation, goals and objectives, content, opportunities for practice and transfer, and evaluation mechanisms (Criteria for Evaluating the Quality of Online Courses, Clayton R. Wright). If such metrics can be automated, then analytics can be created with can reflect the quality of courses. Such metrics could also be fed back to teachers as an incentive to improve their courses.
What analytics would you add to this list?

How can you get them?

So, you want these analytics, but how can you get them? Some of them may already be accessible via various mechanisms, however I think we still need to work out how best to draw this information together in a simple way for specific users.

Moodle currently logs most actions that take place within the LMS. It is possible to view log reports, but they are limited to interaction in terms of activities within a course.

There are a number of existing plugins and extensions to Moodle that attempt to provide analytics to users. Among these there are a batch of report generators, many of which are quite configurable.

  • Configurable reportsThe Configurable reports block plugin is a configurable system that allows reports to be created and used by various roles. It may be a good model to use to start a set of standard analytics reports within an institution.
  • The Custom SQL queries report plugin allows an administrator to run any query against the database used by Moodle. It’s clearly flexible, but not something you can put into the hands of all users.
  • The Totara LMS is a rebranded, extended version of Moodle. One of the aspects built onto the standard Moodle install is a reporting facility that provides customisable reports to users of different roles.

There are also a number of blocks available and in-the-works that attempt display analytical information to users.

  • Graph statsMy own Progress Bar block shows a simple view of course progress to students and an overview of student progress to a teacher.
  • The Engagement analytics block is currently in development, but I have seen a demo of this and it looks good. now available. The block allows a teacher to specify expected targets for students then presents to teachers simple traffic-light icons next the names of students at risk.
  • The Course Dedication block estimates the time each student has spent online in a course.
  • The Graph Stats block shows overall student activity in a course over time.

Simple queries

A lot of these analytics can already be queried or calculated from the data already stored in the Moodle databse. The Report plugin type is available for presenting pages if information to users and is applicable for analytics. The Block plugin type is available for simple, compact presentation of information. Both of these APIs can present different displays to users with differing roles.

New logging

Currently, most of the logging that takes place in Moodle ends up in a single table. For simple lookups, this is not a problem, but for complex conjunctive queries, working with a large log table can hog the resources of a server. The current system of logging is likely to be a target of future work at Moodle HQ so that both the recording and retrieval of information can be achieved efficiently.

Measurement of a number of the interactions required for the analytics above is not possible using the current log and module tables. Viewing the recording of user interactions from an analytical perspective may lead to new information being captured for later analysis and presentation.

AI or user optimised queries

When you have a wealth of user interaction information available, why stop at the limits of human developers.

  • Genetic algorithms, neural networks and other heuristic approaches may reveal newly refined analytics or even new analytics altogether.
  • Crowd sourced optimisation of analytics reports may allow a collective intelligence to refine analytics so that they are even more valuable and reliable.
Are there other tools or technologies that you can think of that would help gather analytics?

Analysing analytics

Providing analytics allows us to study the impact that analytics can have on the users who view them. This allows general research questions such as “what analytics promoted better learning, teaching or retention?” Also, specific questions can be asked about individual analytics, such as “does feedback attention have an impact on learning outcomes?” Queue the savvy educationalists…


4 Comments

Moodle is Mainstream

The Underdog

BlackborgA few years ago, while attending a Moodle Moot, I realised that the Moodle community had a serious underdog complex. The mere mention of Blackboard raised emotions reminiscent of an Orwellian hate rally. But even then, I had the impression that Moodle was taking hold and becoming more commonplace, and I was forced to consider: what happens when Moodle is no longer the underdog?

When I first started working at Moodle, I had to explain to my friends and family what Moodle was. Some of my former university-related colleagues knew about Moodle, but weren’t sure how I could be working for Moodle.

Of late, I’ve been frequently encountering random people who know what Moodle is (and have opinions about it), which gives me cause to believe that Moodle is no longer just a small competitor in the LMS game.

  • While at the recent Australian Moot, while in a hotel elevator I was wearing my Moodle shirt; I was asked by another guest if I was here for the “Moodle Conference”. I said I was and asked if she was also. Apparently she was not, but had heard that the conference was happening in town and went on to tell me that her school was using Moodle and she thought it was great. All this in a few floors.
  • At a barbecue with my son’s swimming club, I was asked where I worked. When I mentioned Moodle, several people spontaneously started sharing their Moodle experiences – submitting work online, discussing ideas in forums and monitoring children’s homework. The scientist in me made me sit back and gather evidence, which turned out to be positive.
  • I’ve had similar fleeting experiences on trains, at church gatherings and with old school friends.

It’s even more surprising to hear random people mention Moodle when schools and universities usually re-brand their instance of Moodle, calling it “Study Desk” or “iLearn” or “Wattle” or something else. People are starting to recognise that the underlying system is Moodle, which is the same system their colleagues are using elsewhere.

In truth, Moodle has been mainstream for some time. In some parts of the world, Moodle has reached saturation at the university and vocational training levels. Even Moodle’s competitors are recognising this and attempting to buy into Moodle.

Why has Moodle become mainstream?

There are a number of reasons I can think of that have caused this shift in status.

  • PluginsTeachers want the LMS to do what they want it to do. Moodle, with it’s treasure-trove of plugins, allows them to do more of what they want, and there’s always the potential to create another plugin. Moodle gains much of its strength from the Developer community.
  • People are becoming more accepting of open-source software. With the successes of Firefox, OpenOffice and others, people are starting to see the value in such software and the quality a community effort can produce. Talking to people involved in other open-source projects, I know that Moodle is one of the most organised projects around. The structure and funding model set up for Moodle is very sustainable and ensures Moodle will not disappear for many years yet.
  • Being open source and free, Moodle is making inroads into smaller educational institutions, and this is something commercial LMSs have not been able to do. Not being pigeon-holed in higher education means Moodle is becoming more of a household name.
  • Moodle is open about its flaws. In my position I can claim to know all the blemishes on the face of Moodle, but you can see them too at tracker.moodle.org. Users now realise that no software is bug free and when a commercial product is presented to them as flawless, it is greeted with scepticism.

Perhaps the biggest reason Moodle is shifting into the mainstream mindset is because of the community sentiment that surrounds it…

Satisfaction

This is a bit tongue-in-cheek, but it is clear that public sentiment about Moodle is high.

What does the internet think

This is not infallible evidence, but I dare you to try this test with other LMSs.


2 Comments

Moodle – Now with more SERIOUS

I’ve been thinking of writing this post for a while now, and the list of things to include seems to have been increasing – so here goes.

New SERIOUS staff

Since I have arrived at Moodle HQ there has been a push to increase the number of developers. As Moodle popularity grows, so does the need for maintenance and new feature development. When I arrived in April there were nine employed developers, six more have been added and two more are arriving in coming months. My hope for the near future is to have two teams working on stable development and a third on new features development.

But Moodle is not just about development and in recognition of this, two new roles have been added to the HQ complement.

As a sign of the determination to get things right, we now have a Test Manager. Tim will oversee testing during development and release processes. He brings a great deal of experience as a tester and developer, and will improve testing processes so that the result is serious testing and a more solid Moodle codebase.

Image of Tim Barker

Tim the Test Manager

Barbara brings experience working as a graphic designer in universities, including work with Moodle. She also has teaching experience (in design), so she is highly qualified. But beyond that, she brings a level of style that Moodle has been lacking for some time. A quick look at moodle.org will demonstrate that developers are not very artistic. Undoubtedly the benefit of Barbara joining us will be Moodle – with more serious style.

Image of Barb

Barbara the Designer

New SERIOUS release schedule

It took a long time for Moodle 2.0 to come to fruition. This was a significant step forward (one that many Moodle users are still taking). Afterwards it was decided that it is not necessary to wait for the a complete release roadmap to be fulfilled for each version. Instead, a timed release schedule is now used to create a greater sense of certainty. Part of my role is to “bulldog” developers, to ensure that major releases (such as 2.1 and 2.2) happen on time. We now have major releases every six months in June and December, and since 2.0 releases have happened on time.

Moodle also has minor releases (AKA point releases), such as 2.2.1, that occur between major releases. The main reason for minor releases is to send fixes to security issues out as soon as possible. This might sound like a bit of a ‘number’s game’, but for Moodle Partners and the Moodle community, keeping their Moodle sites up-to-date is a ‘big deal’, and we want to do what we can to help them.

Up to now, minor releases have been completed irregularly when a number of security fixes have come about. But now, we’re getting even more serious. In order to assist in planning upgrades now and into the future, minor upgrades are scheduled to take place every two months, offset by one month to major releases (January, March, May, July, September and November).Release calendar

For more details, see the General release calendar.

New SERIOUS documentation

DocBlock comments in the Moodle codebase

Moodle documentation has, in the past, been fragmented and inconsistent. For new developers, the task of discovering how to work in Moodle as a coding framework has been a daunting one. So to remedy this, HQ developers have put down their development tools to focus on documentation. Starting with core APIs, descriptions are being added and updated to create consistent codebase documentation. This is being linked to Dev Docs wiki pages, again in a consistent manner, to provide developers outside Moodle HQ with the information they need. A planned future addition is consistent ‘getting started’ documentation for each plugin type (currently there are 32 to choose from).

Staying SERIOUS

One of the main focus areas leading up to the release of Moodle 2.3 will be accessibility. With the help of staff from North Carolina State University in the United States, including Greg Kraus and Glenn Ansley, a solid list of accessibility issues has been compiled. Together we will work through these issues to create a release of Moodle that will be accessible to more people. If you would like to be involved in this effort, please help where you can.

As well as this, you can anticipate improvements in usability, specifically file handling, coming soon. (I bet you’re holding your breath.)