News Alerts‎ > ‎Announcements‎ > ‎

Quality Audit Update

posted 25 Mar 2009, 09:48 by Michael Cousins   [ updated 17 Feb 2010, 06:43 by Professional Services ]
Customer Quality Auditing is progressing very well; as at March 25th over 50% of the submitted audits are either under way or have already completed. We thought it would be helpful to those customers still awaiting the outcome of their audit to understand how things are progressing.
Recall that the release conditions for Process Navigator 4.6x are:
  1. Have no known errors that are both
    1. Likely to occur
    2. Have no workaround
  2. Have no known errors that prevent the Publication Server completing a publish
  3. Pass a stability test equivalent to 1000 hours of continuous operation without stopping (41 days of continual Publication Server activity without human intervention) on both Visio 2003 and Visio 2007. The input test set for this stability is test is drawn from a range of tests constructed by Triaster in order to trigger failure, and every map we have in our possession.
  4. Have no known errors associated with maps submitted by customers for testing. Triaster are willing to exhaustively test all existing customers' maps for them. See Triaster Offer Free Exhaustive Testing Service to Publication Server Users.
Point 4 in particular is the relevant quality condition.
So, how does that translate in practical terms? Well, our aim is that when we release version 4.62, every customer audit will have passed. To put that another way, before we release 4.62, we will have finished all remaining audits and all of them will have passed. We believe we will be able to achieve this during April unless one of the remaining audits uncovers a problem that is hard for us to fix.
How can we ensure all audits will pass? We do this as follows:
  1. Perform the audit on the latest 4.62 candidate release (currently 4.61.5).
  2. If the audit passes, nothing further is required.
  3. If the audit fails, the set of maps are placed in the re-audit queue.
  4. For the set of maps at the front of the re-audit queue, the cause(s) of the failure are documented and the Triaster development team modify the code to correct for the cause of the audit fail.
  5. A "Triaster-only internal build" of Process Navigator is produced containing the fix. These are numbered 4.61.6; 4.61.7;.... and so on.
  6. The audit is repeated for the set of maps at the front of the queue, and we then go back to step 3 again until the audit passes. 
Once the re-audit queue is empty, we then regression test all audits to be double-certain that a fix on one area does not in turn produce a fail in another. We then promote the candidate release to 4.62 and make it available for download.
A practical side affect of this approach is that 100% of test and development focus at any one time is being given to the set of maps at the front of the audit queue, rather than across all audits simultaneously. The audit process can therefore appear to be making little progress on any particular audit, but in fact progress is being made on a daily basis towards a version of 4.62 for which all customer audits will succeed. We ask for your patience as we push this ahead, it is at times a complex and time-demanding process, but we feel certain that it is far better to go through this process rather than release 4.62 prematurely.