When purchasing an LMS for your organisation it’s easy to get bombarded with over simplified, biased and unhelpful advice: make sure its fit for purpose, is it easy to use, ask if you’ll get the support you need, check that your data will be secure; and the big one, ask how much it’ll cost. These are good high-level questions for a vendor, but (a) you probably already thought of them and (b) they don’t offer much in the way of practical guidance to make a buying decision. You’re about to make a significant purchase for your organisation. So, “basic, biased and obvious” is not good enough. After responding to over 300 tenders and writing 100s of proposals over the years we’ve built up a pretty good picture of what concerned buyers are looking for.
1. Make sure its fit-for-purposeIn order to evaluate this, you’ll need to know what purpose and whose purpose. Who will use the system and what will each of those users need and expect. This sounds obvious but you’d be surprised how many LMS buyers assume they know what their users need, without ever asking them. Organisational needs analysis and scoping are the best tools for establishing this.
In a training context, your new LMS should at least cover the basics:
- Create and manage a course catalogues in your delivery modes (ILT, Online, Blended, VILT)
- Schedule courses to the level of detail you need
- Develop organisational training needs and training plans
- Enrol learners, members or employees into courses
- Assess learners using methods that are important to your organisation
- Mark results, attendance and give feedback
- Manage currency (professional development) and course expiry notification
- Access reports and dashboards that help you track training effectiveness and completions
- If you’re an RTO, the system must be AVETMISS / USI compliant.
2. Make sure the system is easy to use?
We’ve never encountered a single buyer that asked for a system that was difficult to use. “Easy-to-use” is always a requirement. But what does that question mean for you? Are your users likely to access the system via mobile phones? If yes, then at least for the ones that do, the user interfaces should be responsive, i.e. they adjust page elements to optimise the view for best readability and usability. Responsive design is usually tested for optimal display on desktop, tablet and mobile devices. There are guidelines and good examples covering best practice User Interface (UI) and User Experience (UX). Consult these resources and decide how they apply in your organisation. This article is intended for designers, but is also a very useful guide for evaluating user experience:
Ref: The 15 Rules Every UX Designer Should Know – Nick Babich, Adobe Feb 21, 2020
3. Ask if you’ll get the support you need
Many of the largest software vendors today outsource training and support services for their products. The quality of support and training is determined by what your organisation (the customer) is willing to pay. Welcome to the commercial world. Good support requires good people to deliver it and they need ongoing training. That comes at a cost to the vendor and to you. If training and support is included in your package you should understand the fine-print and make sure it suits your needs.
- Is there a ticketed helpdesk system?
- Does the helpdesk allow email, phone and iChat one-on-one support?
- What training is available for which users?
- Are software updates and patches included?
- How is support delivered (F2F, Online, Web Conference – Blended)?
- Will the more complex issues be escalated to a proficient team?
- What is the service level agreement (SLA) and is it published?
- What is the guaranteed-up-time for the service?
- How are SLAs and Uptime monitored and reported?
- What happened if/when the vendor fails to meet SLAs?
4. Check that your data will be secure
Security questions and evaluations tend to be better than other areas. That’s because the standards and expectations are much better defined world-wide, than they are for say Usability. Even so, you need to consider what’s important to you. Again, no one has ever asked us for a system that broadcasts private information to the world. Compliance to the privacy act is a given and must be stated by the vendor. Other security-related and frequently asked questions include:
- Is the data hosted in Australia (for Australian customers)?
- Is the data encrypted ‘in-transit’ (over the internet) and ‘at rest’ (when stored)?
- What is the level of password protection?
- Does the system use 2-factor (2FA) or multi-factor (MFA) authentication?
- Who has access to what private and sensitive data?
- Can we (the organisation) control who has what access?
- Does the system support Single Sign On (SSO) for internal users on say, Microsoft 365?
- Is the system PCI-DSS compliant for online credit card transactions?
- Is the system designed to prevent cyber-attacks and is it tested?
- How are vulnerabilities managed and reported?
- What are the backup and disaster recovery policies?
5. Ask how much it’ll cost
Unless you have a very specific and tight budget, this question should always be left till last. In most cases what you’re really asking is “does the system provide good value for money?”. So, until you know what the system is and does you have no way of equating its value to other systems on your list. When comparing prices across multiple vendors the questions to consider are:
- What is the pricing model and is it appropriate in my organisation?
The most common pricing model used by software vendors is based on the number of active (concurrent) users per month. If you employ or train several 1000 people in courses that last many months it may be better to ask for an Enterprise, Government or Education Edition with unlimited or high-volume users and special pricing. Monthly user pricing works well if you have a small number of active users / learners each month.
- Do prices vary based on user role?
Some vendors apply different pricing tiers for Admin users, Course Authors, Trainers or Managers. This may have a significant cost impact for your organisation. If, for example you employ many trainers who each need an individual login account. The costs could be high.
- What are the implementation costs?
Usually, the first-year pricing includes implementation costs. There may be one “Setup Fee” for a simple no-frills SaaS LMS, or a comprehensive suite of technical services individually priced, example:
- Requirements scoping
- Prototyping and mock-ups
- Hosting setup and deployment for staging and production
- Data and content migration from a legacy system
- Branding and configuration
- Integration with on-premise systems
- Custom workflows and reports
- User acceptance test (UAT) training and support services
- Go Live – deployment and test
In conclusion
Over the past 20 years we have learned how to respond to many concerns and requirements expressed by LMS buyers. This guide should help you to formulate the right sort of questions to ask prospective vendors. The questions and their responses should help you to identify Learning Management Systems that will better meet your organisation’s needs and expectations. While the functionality, services and implementation details will differ dramatically, these principles suggest some of the common benchmarks you can use in your evaluation. We’re always learning better ways to help you deliver learning, so we invite you to join us for the ride and would love to get your input and insights.
Author
Bruno Cozzi - Managing Director, Bluegem Software.
Has over 30 years’ experience in the design and management of learning and development systems. He Leads a team of software development and support professionals dedicated to the delivery and support of outstanding Training Management Systems for the education and training sector.
