Only the most trivial learning management system used in very small organisations can really be owned and run by one person. In most cases, several stakeholders could be involved. Our support team usually knows when our customer teams aren’t working together. We can hear their frustration in support tickets, responses to invoices and during training. Ineffective teams blame each other, the system, the last vendor, the current vendor. The list goes on…
The problem
There can be any number of reasons why a team might find it difficult to work together, but a good way to determine whether “the new learning management system” is the problem is to ask and honestly answer this simple question:
“Did this all start when we brought the new LMS, or did we always have problems working as a team?”
Only an answer that reflects the honest views of all stakeholders will do.
If the system really is the problem, what led you and your team to purchase it?
What research did you do before hand, what questions did you ask the vendor?
Did you have a clear set of agreed objectives in writing?
What training and support was expected from and provided by the vendor?
Was there a clear understanding of the onboarding and adoption path for your team?
How did you plan to assess performance and effectiveness of the new system?
Did you ask if there was a trial or evaluation period? If so how thoroughly did your team perform the evaluation process?
How many vendors did you consider before deciding on this one?
These are not questions specific to an LMS purchase. They are questions about systems, teamwork and communication. When we respond to well-written tenders its clear to see the processes, thinking and stakeholder engagement that went into creating them. As a respondent you feel confident that if you are the successful vendor you’ll be delivering a system that will throw up very few surprises, with a good runway to successful adoption. The closer you can get to that ideal tendering model the more successful your adoption is likely to be.
Steps to a successful team-purchased LMS?
- Appoint a trusted and capable facilitator
- Identify representative stakeholders and invite them to participate as a team
- Draft and state the high-level objectives of the team
- Identify any competing interests amongst the group and work on those last so that you can get broad agreement, before tackling the harder questions.
- Demonstrate / Discuss a few initial candidates to give the team a sense of what’s available in the market, or issue an Expression of Interest (EOI) to the market.
- Learn to identify and screen out any vendor bias in your team that might skew the objectives.
- But make use of any previous system experience that might provide insights.
- Using this information, get each user to list, describe and prioritise their system needs.
- Integrate individual stakeholder lists into the overall system requirements.
- Decide how you want to be trained and supported and add this to the requirements.
- Identify a team who will be responsible for evaluating the system in a controlled user acceptance test (UAT) environment.
- Get the team to create relevant scenarios that can be used to evaluate candidate systems.
- The list of candidates (ideally 3) may have changed at this point
- Invite the candidates to present and provide quotes
- Make sure the system is easy to use?
We’ve never encountered a buyer that asked for a system that was difficult to use. “Easy-to-use” is always a requirement. But what does that mean for you? Are your users likely to access the system via mobile phones? If yes, then at least for the ones that do, the user interfaces should be responsive, i.e. they adjust page elements to optimise the view for best readability and usability. Responsive design is usually tested for optimal display on desktop, tablet and mobile devices. There are guidelines and good examples covering best practice User Interface (UI) and User Experience (UX). Consult these resources and decide how they apply in your organisation. This article is intended for designers, but is also a very useful guide for evaluating user experience:
Ref: The 15 Rules Every UX Designer Should Know – Nick Babich, Adobe Feb 21, 2020
2. Ask if you’ll get the support you need
Many of the largest software vendors today outsource training and support services for their products. The quality of support and training is determined by what your organisation (the customer) is willing to pay. Good support requires good people to deliver it and they need ongoing training. That comes at a cost to the vendor and ultimately to you. If training and support is included in your package you should understand the fine-print and make sure it suits your needs.
- Is there a ticketed helpdesk system?
- Does the helpdesk allow email, phone and iChat one-on-one support?
- What training is available for which users?
- Are software updates and patches included?
- How is support delivered (F2F, Online, Web Conference – Blended)?
- Will the more complex issues be escalated to a proficient team?
- What is the service level agreement (SLA) and is it published?
- What is the guaranteed-up-time for the service?
- How are SLAs and Uptime monitored and reported?
- What happens if/when the vendor fails to meet SLAs?
3. Check that your data will be secure
Security questions and evaluations tend to be better than other areas in the evaluation process. That’s because the standards and expectations are much better defined world-wide, than they are for say Usability and Functionality. Even so, you need to consider what’s important to you. Again, no one has ever asked us for a system that broadcasts private information to the world. Compliance to the privacy act is a given and must be stated by the vendor. Other security-related and frequently asked questions include:
- Is the data hosted in Australia (for Australian customers)?
- Is the data encrypted ‘in-transit’ (over the internet) and ‘at rest’ (when stored)?
- What is the level of password protection?
- Does the system use 2-factor (2FA) or multi-factor (MFA) authentication?
- Who has access to what private and sensitive data?
- Can we (the organisation) control who has what access?
- Does the system support Single Sign On (SSO) for internal users on say, Microsoft 365?
- Is the system PCI-DSS compliant for online credit card transactions?
- Is the system designed to prevent cyber-attacks and is it tested?
- How are vulnerabilities managed and reported?
- What are the backup and disaster recovery policies?
4. Ask how much it’ll cost
Unless you have a very specific and tight budget, this question should always be left till last. In most cases what you’re really asking is “does the system provide good value for money?”. So, until you know what the system is and does you have no way of equating its value to other systems on your list. When comparing prices across multiple vendors the questions to consider are:
- What is the pricing model and is it appropriate in my organization?
The most common pricing model used by software vendors is based on the number of active (concurrent) users per month. If you employ or train several 1000 people in courses that last many months it may be better to ask for an Enterprise, Government or Education Edition with unlimited or high-volume users and special pricing. Monthly user pricing works well if you have a small number of active users / learners each month.
- Do prices vary based on user role?
Some vendors apply different pricing tiers for Admin users, Course Authors, Trainers or Managers. This may have a significant cost impact for your organisation. If, for example you employ many trainers who each need an individual login account. The costs could be high.
- What are the implementation costs?
Usually, the first-year pricing includes implementation costs. There may be one “Setup Fee” for a simple no-frills SaaS LMS, or a comprehensive suite of technical services individually priced, example:
- Requirements scoping
- Prototyping and mock-ups
- Hosting setup and deployment for staging and production
- Data and content migration from a legacy system
- Branding and configuration
- Integration with on-premise systems
- Custom workflows and reports
- User acceptance test (UAT) training and support services
- Go Live – deployment and test
5. Make sure this system complies with industry standards
If you are a registered training organisation (RTO) you are well aware of ASQUA’s requirements for registration, AVETMISS, USI, AQF and the Standards for RTOs (SRTO 2015 with updated amendments). These standards come with procedures for managing changes and version control. In other words, your new LMS not only needs to comply with today’s reporting standards. It needs to keep pace with ongoing changes and version upgrades.
- Do your LMS candidates meet your compliance needs?
- How do they keep current and up-to-date with changes to reporting standards?
- How will they inform you of compliance updates in the software?
- How will they support you in adopting those changes?
- Do you pay extra for compliance maintenance, or is it included?
- What experience do they have in supporting RTOs?
In conclusion
Over the past 20 years we have learned how to respond to many concerns and requirements expressed by LMS buyers. This guide should help you to formulate the right sort of questions to ask prospective vendors. The questions (and their responses) should help you to identify Learning Management Systems that will better meet your organisation’s needs and expectations. While the functionality, services and implementation details will differ dramatically, these principles suggest some of the common benchmarks you can use in your evaluation processes. We’re always learning better ways to help you deliver learning, so we invite you to join us for the ride and would love to get your input and insights. In the meantime, we wish you and your team every success in the uncertain and changing world we live in today.
Author
Bruno Cozzi - Managing Director, Bluegem Software.
Has over 30 years’ experience in the design and management of learning and development systems. He Leads a team of software development and support professionals dedicated to the delivery and support of outstanding Training Management Systems for the education and training sector
