How to survive an audit or code review

, 734 words

How to survive an audit or code review

Hmf... Amongst our weaponry...are such elements as fear, surpr.... I'll come in again. It's not unusual that I'm asked to evaluate a start-up's technology platform, often by their investors or potential investors. Many developers or agencies are taken aback by this and are ill-prepared to respond. Competent developers with repeatable, structured process and methodology should find that handling a code review or platform audit is not arduous.

First of all, developers should not be surprised by the prospect of an audit. I've written before about the challenges agencies face when serving start-ups as opposed to established businesses, and the two are very different customers. Funding a new company can be a risky proposition and there are many factors that will lead an investor to request such a review.

"Surviving" such an audit is simple, and there are three golden rules:

  • Take it seriously. All businesses care about the money they're spending, but start-ups really care and their owners and investors will look for reassurance.
  • Be prepared. Don't wait until the auditor shows up at your office: wasting their time whilst you prepare will not get the meeting off to a good start. If you have everything ready beforehand, that's great. If you're able to send them information beforehand (or after) it will help them compile a more accurate report. Most audits are relatively short, and it's exhausting work for an auditor to review and understand everything that's been done. Having key staff available for interview really helps.
  • Be honest. If you don't have a swathe of documents that the auditor has asked for, that's fair enough. Just lay out what is available and they'll work their way through it.

For my sake, I will work on behalf of the investor or start-up, or can assist an agency in preparing for an audit where I have no connection with their client or client's investors.

Depending on the nature of the platform being built, I'd typically ask for and expect to receive the following information. Often remote access will not contractually be a possibility, so a visit to agency offices is routine and makes interviewing easier.

Planning

  • A list of user stories identified for the project, split into:
    • Developed stories (with completion date of each)
    • Partially complete stories (with last-modified date of each)
    • Stories not yet started
  • If the project is not agile, a description of methodology and specification, requirements or work unit documents
  • Any pre-sales or pitch documentation
  • A copy of the release plan and the iteration plan
  • Evidence of capacity planning, with key factors and variables

Documentation

  • Visual module summary of components (should make factoring & any design patterns clear)
    • Summary: How is the application tiered?
    • Summary: How is persistence managed?
    • Summary: Application hosting, with server and network topology and details of any cloud (like Amazon EC2 or S3) or CDN (like Akamai) used
  • Any platform architecture documentation
  • Unified modelling language (UML) or class diagrams to show to core of the application
  • Data flow, sequence, state diagrams
  • Database ERM/schema diagrams
  • Metrics from code coverage tool (FxCop, for example)
  • List of 3rd-party components used & licensing status of each
  • Access to bug-tracking tool, or full report from tool against project
  • A full report from whichever time-tracking system is used by the developers
  • Service level agreements (SLAs) from any relevant hosting or service providers

Source code and infrastructure

  • Access to SCM platform (SVN, CVS, VCS, SourceSafe, etc.)
    • Copy of HEAD source from SCM, and each release branch
  • Access to unit test source, results of a recent run of unit tests (access to any continuous integration tools like CruiseControl is a bonus)
  • Access to artwork PSDs if appropriate
  • Access to database schema creation scripts
  • Access to live IIS console or Apache configuration
  • RDP or SSH access to a pre-configured development machine with VisualStudio or other IDE

Authentication

  • Project administrator password for source-code management (SCM) tool
  • sa, root, Administrator and application-specific username & passwords for servers and database as appropriate
  • Any .htaccess passwords
  • Credentials for access to Google Analytics, Adwords and Webmaster accounts or alternatives if provided by another vendor

This is by no means a complete list and will vary according to the project.