Writing a Software Review
By Robert T. Foraker, II
University of Delaware
EDDV 667 Multimedia and Internet Literacy
December 15, 1997


Introduction

Writing a software review

Preparation

Testing and evaluation

Write the review

Sample Review: AutoCAD® Learning Assistance Software Description

Review Criteria

Review

Conclusion

Summary

References



Introduction

So much new software, so little time! Good software reviews can be very useful. They can help us make decisions about which new titles to obtain, how we might use them, and may alert us to potential problems. For educators, the lack of reviewers familiar with the challenges and requirements of the classroom compound the need for reviews.

This paper will detail methods and techniques to assist educators in creating "good" reviews. In addition, the author's example review of a multimedia application is used to illustrate the process. It is hoped this will encourage others to take the time to pass on their knowledge of, and experience with, new programs they have utilized in their own classrooms.

Table of Contents    

Writing a software review

Preparation

Identify the audience. Define the age and learning levels of the target student group. Analyze your intended use of the product to determine the objectives to be met.

Choose the appropriate hardware system. Try to utilize a student system, or a close approximation. Avoid testing on a system that significantly exceeds the capabilities of the typical student system, as acceptable performance here may not translate to acceptable performance for the student.

Install the entire software package or suite, using the software's own preset installation options for file locations, etc. This may become critical, as improper file locations may create instability in the product, or may create the appearance of errors in usage instructions, tutorials, help files, or examples.

Find several reviewer style sheets or evaluation instruments to act as guides. Try to find those that are appropriate for the age and learning levels suggested for the software to be reviewed. For example, a reviewer style sheet for post secondary software is located at Software review guidelines from H-Net, (Michigan State University, 1997). The Children's Software Review (CSR) Software Evaluation Instrument (Parents, Educators, Publishers, 1997) is suitable for software at the K-8 level.

Table of Contents    
Testing and evaluation

Use the software. Try out all components, and actually use every feature. Look for inconsistencies between the instructions and what you really have to do.

Change your point of view. If you are an experienced user, try to put yourself at the level of the intended user. Look for things that will "trip them up," confuse or irritate them.

Go bug hunting. Look for things that don't work at all, don't work as they should, do the unexpected, invite the three-fingered salute, etc. See if there are ways to work around these bugs (a.k.a. "undocumented features")

Do something else and unwind. Look at it again tomorrow, starting from Step 1 and see if you can find anything else worth noting, either pro or con.

Table of Contents    
Write the review.

State the review criteria used. Indicate the sources used for review format and style sheet. Also, indicate the anticipated grade and/or learning level of the target student audience.

Describe the product. Avoid a detailed description of every screen, sound, or video found. Try to concentrate on the overall product, its use of multimedia elements, its organization, and other strengths and weaknesses.

Detail the advantages and disadvantages of the product in its intended uses. Detail those areas or items having a positive impact on the learning experience as well as those creating a negative impact.

Try to include images or illustrations of the software screens to illustrate its features, look, and feel.

Table of Contents    


Sample Review: AutoCAD® Learning Assistance

AutoCAD Learning Assistance Version 1.0, (1997), CD-ROM, FREE (Included with AutoCAD Version 14), Uses the Webtrek Browser and ActiveMovie controls, included. Autodesk, Inc., 111 McInnis Parkway, San Rafael, CA 94903 USA 

Hardware System Requirements
Minimum
Recommended
As Tested
CPU
486
Pentium® 90MHz
Pentium® 75MHz
Graphic Resolution
640x480
800x600
640x480 & 800x600
Number of Colors
256
64K
256 & 64K
Audio
16-bit
 
16-bit
CD-ROM Speed
2X
4X
4X
Memory
16MB (Win95)
32MB
32MB
Hard Disk Space
10MB
 
659 MB
Operating System
NT3.51
Win95, NT4.0
Win95 (SR-1)
 
Table of Contents
Example Review
 


Software Description

AutoCAD® Learning Assistance (ALA) is a multimedia training package designed to assist the computer aided designer with the transition to AutoCAD® Version 14. The software utilizes a web browser control as the user interface. This interface provides a consistent look and feel as the user navigates the software. AutoCAD Learning Assistance was awarded the 1997 Editors choice award, Cadence Magazine, (1997) for excellence.

The software opens with a general AVI movie sequence about ALA (can be disabled by the user), and then to a start menu screen, permitting the user to select from five options:

 
Table of Contents
Example Review
 
Tutorials
There are 55 tutorial lessons included. These lessons are grouped into three main categories, with 11 sub-categories. The user navigates the lessons using a folder, sub-folder and document hierarchy. Individual folder contents can be collapsed or expanded, and each lesson, sub-folder, and folder is checked off as it is visited. Each folder and sub-folder also contains a brief description of the type of lessons found in that area. The lessons are accessible in any order, and the user may go directly to those topics of interest, and may return to any lesson as often as desired.

Each lesson contains an AVI sequence demonstrating the topic covered. The AVI movies show the AutoCAD 14 screen with a sample drawing file loaded, and step the user through each step of the tutorial. "Try it" is an additional feature of each lesson. Clicking the "Try it" button launches AutoCAD 14 with a floating "Try it" help window. This arrangement permits the user to work on their own AutoCAD drawing while reading the tutorial instructions. In addition, the user may click the "Load sample drawing" button to load the same drawing used in the tutorial's animation sequence, making duplication of the tutorial instructions easily accomplished.

Table of Contents
Example Review
Concepts

The concept section contains 28 pages explaining various CAD terms as utilized in AutoCAD 14. Each concept page contains an AVI sequence demonstrating the term. As in the tutorial section, the user can navigate the concept pages in any order, and each page visited is checked off on the navigation window hierarchy.

Table of Contents
Example Review
Fast Answers

The Fast Answer section contains links to all of the tutorial lessons and all of the concepts pages, arranged by use category folders. An additional feature of this section is a search page permitting the user to search the fast answers by over 500 keywords, with search results linked to tutorial lessons, concept pages, and the AutoCAD 14 command reference help file.

Table of Contents
Example Review
Using Learning Assistance

This section uses the folder/document navigation hierarchy found in the first three sections. It presents introductory information on the program and the use of each section and feature. Animation sequences are provided to step the user through each of these.

Table of Contents
Example Review
Orientation

The orientation selection launches the ALA orientation AVI sequence. This animation/video identifies each selection possible on the start screen, and includes information on the use of the program, toolbars, buttons, and each section.

Table of Contents
Example Review


Review Criteria

This software was assessed utilizing criteria from the Software review guidelines from the Computer Use in Social Services Network, (University of Texas at Arlington, 1997). The anticipated use for this software is in higher education and adult learner continuing education. The suitability for use was analyzed based on either a novice user in an Introduction to AutoCAD course or a user with AutoCAD 12 experience transitioning to AutoCAD 14 in an advanced CAD course.

The following factors were evaluated

 
Table of Contents
Example Review
 


Review

Ease of installation

Software installation was accomplished with ease. The AutoCAD Learning Assistance installation utilizes the InstallShield software, and no errors were found in the process. Options are provided to permit customization of file locations to suite individual requirements. Using custom locations instead of the default values did not update the tutorial lesson information, and could result in increased user frustration, especially for novice computer users.

The software also creates an uninstall file in conjunction with the Windows 95 Add/Remove programs tool. Removing the program created a minor problem, as it indicated that not all files associated with the AutoCAD Learning Assistance program could be removed, due to possible conflicts with other programs. The uninstaller did not indicate, however, which specific files could not be removed, leaving the user unable to determine if they could be removed manually. (Rated 9.5 of 10)

Table of Contents
Example Review
Performance

The software performed well on the test system, in all tested configurations. All features worked properly, including those features launching and controlling other software. Problems were encountered with the first few seconds of sound in all AVI sequences, consisting of audible skips in the sound during playback. In addition, the AVI clips run using the 640x480 graphics mode were larger than the allotted space in the browser panel. This necessitated the adjustment of browser element sizes to permit simultaneous clip viewing and instruction reading. (Rated 9 of 10)

Table of Contents
Example Review
Ease of Use

After reviewing the orientation clip from the main menu, the consistent interface provided an easy to use program. All toolbar buttons have a tool tip feature to describe them, however the usage of some items was not intuitive until they were explained in the orientation. The lessons were very easy to follow, with the AVI clips providing an excellent audio/visual demonstration of the procedure under study. (Rated 9 of 10)

Table of Contents
Example Review
Content Errors

The tutorial lessons contained several content errors. Several examples are detailed here, and others may be found throughout the tutorials.

In the first tutorial, "Making the Most of Multiple AutoCAD Sessions," the instructions indicate a sequence of events for the user to duplicate:

The AVI clip, however, shows actions performed between steps 8 & 9. The clip shows the mouse pointer clicking on the "Zoom Window" button and then selecting a very small area of the drawing for magnification. The novice user may not know what to do to duplicate the view presented on the screen during steps 9 & 10. Later in that lesson, the instructions again deviate from what is shown in the clip. The instructions tell the user how to switch between two concurrent sessions of AutoCAD 14 and perform a paste. In part, the sequence of instructions indicate: 5. Select the Minimize icon to hide the current AutoCAD session.
6. Activate the second AutoCAD session by selecting its title bar.
7. Click on View\Zoom\Window and zoom into the area as shown in the animation.
The sequence shown in the clip, however, does not show step 6 occurring at all. The View\Zoom\Window procedure, indicating selections from the pull down menus in other parts of the lessons, is shown executed by using the "Zoom Previous" then "Zoom Window" buttons.

In the second tutorial, "Exploring the Windows Graphical User Interface," the lesson instructions indicate the user will find the desired file (TABLET.DWG) in the C:\ACADR14\SAMPLE directory. This directory does not exist if the AutoCAD 14 software was installed using the default installation locations. This file is actually located in the C:\Program Files\AutoCAD R14\SAMPLE directory and is named tablet14.dwg. This error will stop most users dead in their tracks, as they will have no idea to look in the proper directory unless they installed the AutoCAD 14 and remember where it is located.

These content errors may cause the first time user to experience abnormal frustration levels, and seem to indicate a lack of detailed testing on the part of the producers of the product. It is worth noting the overwhelming majority of content is error free, and it is unfortunate that most of the errors are found in the early tutorial lessons. (Rated 6 of 10)

Table of Contents
Example Review
Feedback to user

Feedback to the user is not provided by the program, however, the use of the "Try it" feature permits the user to attempt the procedure, and compare their personal results with the results shown in the media clip. This does not indicate, however, the actual error made by the user, nor does it indicate the possible correct procedure to be used. (Rated 1 of 10)

Table of Contents
Example Review
Support

The Authorized Autodesk Dealer and Autodesk Systems Centers provide primary support for the product. Extensive online support is provided through Autodesk's web site. Additional support is available by using direct telephone support programs. The VIP program provides an unlimited support contract, while the Safety Net Program cost $65.00 per support incident. (Rated 7 of 10)

Table of Contents
Example Review
Documentation

No written documentation provided, except a small installation instruction leaflet in the CD-ROM case. Excellent documentation exists in the program, and additional documentation is provided on-line. (Rated 8 of 10)

Table of Contents
Example Review
Value for Money

Cost/Benefit really can't get much better than this. The program is free, and it does a good job, if used correctly. (Rated 10 of 10)

Table of Contents
Example Review


Conclusion

This software is an excellent value, considering its inclusion at no additional charge in the AutoCAD 14 package. The program is well suited for use by users transitioning to AutoCAD 14 from a previous AutoCAD version. Its use by novice CAD users, under the direction and with the guidance of an AutoCAD Learning Assistance experienced educator, can provide good results, however, the users must be cautioned and guided past the various content errors.

Hardware C.P.U., RAM, and hard drive requirements for this program are exceeded by the requirements for AutoCAD 14, therefore any system using AutoCAD 14 will be capable of running the AutoCAD Learning Assistance program. Not all of these systems are equipped with the required multimedia components, and may not be capable of making full use of the multimedia elements of the program. Network installation is described in the documentation, and this type of installation would eliminate the need for a CD-ROM drive at each user's computer.

Table of Contents
Example Review
 


Summary

Software review writing can be a rewarding experience. When working in collaboration with other educators, you will be able to evaluate many new software titles with a minimum time investment. In addition, your personal reviews will lead you to a deeper understanding of the software, its uses, and potential problems in the classroom.

Only educators familiar with the use of software in a teaching/learning environment can truly provide in-depth, accurate assessment of program applicability in the classroom. Others may be able to evaluate the technical merit of the software, but may not be capable of providing an analysis incorporating student needs and reactions.

Table of Contents    

References

Autodesk Software, (1997). [Online] Available: http://www.autodesk.com (1997, December 15).

Autodesk Software, Learning assistance product description, (1997). [Online] Available: http://www.autodesk.com/products/acadr14/features/imprprod.htm (1997, December 15).

Cadence Magazine, 1997 Editors choice award, (1997). [Online] Available: http://www.cadence-mag.com/1997/dec/edchoice.html (1997, December 15).

Michigan State University, Software review guidelines from H-Net, (1997). [Online] Available: http://www.h-net.msu.edu/reviews/revstyle/stylesoft.html (1997, December 15).

Parents, Educators, Publishers. Children's Software Review Evaluation Instrument. (1997). [Online] Available: HTTP://microweb.com/pepsite/Revue/evaluation.html. (1997, December 15).

University of Texas at Arlington, Software review guidelines from the Computer Use in Social Services Network, (1997). [Online] Available: http://www.uta.edu/cussn/swrevfm.htm (1997, December 15).

Table of Contents    
 

©1997 Robert T. Foraker, II
Last page update: 15 December, 1997
Official URL for this page: http://www.dtcc.edu/~foraker/alareview/review.html
Page Maintained by: Bob Foraker