Import from CSV files

Wednesday, Feb 4, 2015

If you have more than 20 courses it might prove useful to consider automating the data entry of course data. Moving from one system to another will be tricky, there may be fields that do not transfer across or fields that need to be created from scratch. Contact us if you wish to use the Selenium Scripts that can automate the data entry.

It is important to realize that CourseSales.com comes from a sales perspective, then overlays the learning model of what is being taught. So Course Masters, like a course template, includes all the information pertaining to a course as a sale item. Via the course format course modules are linked to Units of Competency (UoC), this means the areas of learning are included. For this reason you are unlikely to want to create a Course Master for all of your UoC. For example if you sell the Qualification Certificate 3 in Demolition, you would create a Course Master for Certificate 3 in Demolition, not the 33 UoC under it. You would create Course Modules for each Unit of Competency, link these to the Course Format which is linked to the Course Master. The advantages of this arrangement is the same Course Module (UoC) can be used in multiple courses where the Course Module is shared. If the Course Modules are sold separately however, and need to be listed separately then they will need to have Course Masters created for those that are for sale.

You can set up your data in CourseSales.com by either using the spreadsheet that maps your existing data to the new data format, or manually by opening up the required sections of CourseSales.com and adding fields as required. It is not recommended that you try to combine these methods during initial setup as names and fields must be exact which, when using a spreadsheet is easily performed by cross references, but may take additional work when matching to manually entered fields ie, adding your course categories then trying to include these in automated data entry may cause extra work.

Consider testing your automated data entry by using your test environment eg: https://[short name].test.coursesales.com

You may wish to manually set up just a few courses as a proof-of-concept and to check how you may wish to use the various fields in CourseSales.com to represent existing functionality and provide new functionality. Start simple, displaying the course publicly perhaps integration with your website or sending emails.

There are three major areas of work to be ready to use CourseSales.com:

  • Course Masters

  • Workflow with automation

  • Existing/historical documents

Course Masters

  1. Become familiar with the required data, understand the structure of the Course Masters, course dates, options and Documents. Look as the pre-configured data to see how these are linked, eg. how Locations link to venues, and how Course Categories link to course masters.

  2. Define your options, especially: Course Categories, Locations & Course Modules, add these if you are doing a manual configuration

  3. Define all components required for a course master, eg. Terms, Process Steps, currencies, Tax rules etc, add these now if you are doing a manual configuration

    Create content:

    Course Description

    The short content and Section 1 content is by default used to display on the public pages. These can also be incorporated into emails or automatically generated course outlines and sent via email.

    Email

    Only Subject and Body content are used, there is no additional content. Remember that links to PDFs and File categories will need to be added if appropriate. Course category, Provider and location have an effect on display and inclusion.

    Field

    Used to incorporate into forms to give guidance and information. Short content and section 1 content will be concatenated and used on forms. Remainder content can be used in emails and on PDF.

    Format

    Used to describe how the course will be run and generally what topics will be covered. Topics are covered separately.

    PDF

    Creates dynamic content, PDF files with variables from fields within CourseSales.com, eg. invoices, registration forms, course outlines, certificates.

    Promotion

    Content available in emails and PDF that is contextualised depending on Provider, Location, Course Category, eg. promotion content at a location only means that all courses at that location will include that content in the email, providing there is not a closer match, eg location and course category.

    Residential Description

    Used when a residential course is purchased. This can be used for additional purposes and is course master dependent (specified at the course master level). Course category, Provider and location have no effect on this display only sorting in the list view.

  4. Course Master creation:

    • If automating the configuration by using the Selenium scripts with csv files populate the course master spreadsheet with the course categories and other items you have created. Use cross references within your spreadsheet to ensure the names are consistent. Export each spreadsheet in a csv file and create the components other than the course master and course dates.

    • Create course masters and course dates (all the necessary components should have been created before this step)

  5. Tidy up the components you have created, ensuring naming consistency and removing any double-ups resulting from false starts during importing.

  6. Add the necessary roles, and add users to send/receive emails and have access to the trainer portal.

Workflow with automation

  1. Map out your business processes and try to see if any existing Selenium IDE scripts supplied by CourseSales.com will meet your needs, if you have Selenium IDE skills you can modify those that exist to work for you.

  2. Create your fields and forms, formatting as required, try to keep the same names so that the Selenium Scripts knows what to look for, if you modify the names on the fields you will need to modify the scripts.

  3. Emails created automatically by CourseSales.com at sign up or via the Selenium Scripts will need modification and customisation. Check that the right people are getting the right information. Creating flow charts can be useful here.

Existing/historical documents

  1. Using the Selenium IDE scripts you will need to tweak these to match your forms and the data you wish to capture. Your data will need to be clean so that it can be validated as it gets entered into CourseSales.com.

  2. Run AVETMISS and other tests, cross reference where possible against past reports to confirm the reports include the same information - to confirm the migration has been successful.

Configuration using Selenium scripts

Selenium is an automation tool used by software testers, it is freely available and there is plenty of online support. To use the scripts supplied by CourseSales.com you will need to have a Firefox browser, Selenium IDE, CSV support and While statement support plugins. If you are not technical and you wish to customise these scripts you may find the learning curve steep. While CourseSales.com cannot provide support to learn Selenium IDE, we can supply support for correcting or improving the scripts.

CourseSales.com uses these script to both test our software and to configure customer setups.

Using the Selenium scripts

  • Download the required files from our GIT Repository. Each section is made up of:

    • Test case is a script required to carry out a particular action, eg. adding new fields

    • Test suites include the test case as a bundle for a work package eg. Adding new data for a training organisation

    • Excel (xls) database files, these are Microsoft Excel workbooks, each spreadsheet can be saved as a csv file once you have populated them with the required data

    • variables-standard.js will be used by Selenium to define the variables to be used by each script, this must be specified in the options/preferences ‘Selenium IDE Extensions’.

    • README.txt which gives guidance or information pertaining to this specific work package of files

  • Once open in Selenium IDE you will see that in each Suite there is a test case at the top called ‘Variables’ this MUST be run before any of the scripts will work as it defines the domain you wish to work on.

  • You must have a login to the CourseSales.com site with sufficient permissions to create the necessary items.

  • When you first run the scripts you do so in your test environment found under XXX.test.coursesales.com. Note that data in this environment is overwritten after each release.

The following is a list of the scripts available to you:

AVETMISS Database suite

You can use this to load data from csv files. It comes with data.xls workbook that contains spreadsheets matching test cases below that can be modified to include your data (sample data is provided). It includes the following test cases:

  • Variables - required to set up the variables for your environment eg. your CourseSales.com domain name.

  • Fields - to add or edit fields used to create forms.

  • Forms - to add new forms and the fields on the form (fields are expected to already exist) Fields must be in reverse order

  • CourseModules - creates Course Modules, and retrieves data from training.gov.au to populate fields for Units of Competency.

  • Qualifications - takes a list of qualifications, then adds all the Units of Competency to CourseSales.com as listed on training.gov.au

  • CourseMaster - using the spreadsheet it creates new Course Masters from the data supplied, all items used to create the course masters is assumed to have been created, and is the same name.

  • Formats - uses the course modules created and the spreadsheet to create formats to be added to Course Masters.

  • ContentCourseDescription - takes information from training.gov.au and adds it to a course description. This content will most likely require some tidying up as the information is straight from the training.gov.au web page.

Configuration Services we offer

CourseSales.com offers configuration services. We use data from Training.Gov.Au and data you supply to populate the spreadsheets ready for data entry, we will ask you to confirm that the data is correct before we add it to your installation.

  • We are able to offer a free 3 course master, maximum 300 historical documents import with one review. This requires an annual advance payment license commitment.

  • For those with more data, as a rough guide it takes us about 2 weeks to create, get approval, update, and then add 50 course masters, and costs $3000 + GST. This is a guide, factors that vary this cost include: Moodle integration, interface with multiple decision makers, number of reviews, if you have already entered data but need that merged with automated data. Please ask for a quote. This is a minimum, unfortunately even if you have fewer course masters it still takes about the same time and cost.

  • Alternatively we can offer a review of your spreadsheets or ad hoc advice during migration, charged at $120/hour.

  • If you have lots of data (500+ records) and Selenium IDE scripts are not going to get the job done we can enter the data directly into the database, we will require clean data in a spreadsheet format, please contact us for further information and costs.

Tips

  • Try not to use dash, apostrophe nor speech marks in field names, these are converted by some operating systems and spreadsheets, and may not be compatible with csv files, making import impossible as these are different characters and will not match existing data, eg. for course categories.

  • Try to keep your initial configuration simple, eg have just one document process path, one course process path, set of terms etc.

  • Consider, where appropriate, to create ‘placeholder’ items eg. create placeholder course descriptions with just a sentence, to be updated and modified later once you are sure how you will use that data. This is especially relevant for any new functionality, eg. Catalog Groups, where how you plan to use it is unknown.

  • Be sure that for the options eg. course categories and locations you choose “include in name” for all higher level items. You will need to reference this full name on the spreadsheets to ensure that each is unique

  • Pick a naming convention you wish to apply to items like Course Master names etc. Having a name that uniquely identifies the item, eg. course master is important for recognising the difference to other similiar items, eg. Certificate 3 in Demolition Online with f2f workshops

  • Don’t worry if you add components that then are not used, you can inactivate these or re-purpose them later when you need to create a similar component.

  • Remember that if you are adding historical data you will need to add past courses, associating documents to these.

  • Ensure that before adding data the necessary experts, trainers, course administrators etc, have given approval for the data to confirm it is correct.

  • Assume that some data will need to be added manually, not all data can be added automatically without manual intervention or tweaking.

  • Expect to come back to your configuration as you progress to tweak and update data you have already defined/created, as you begin to realise how the system works you will no doubt want to make changes.