National Library of New Zealand
Harvested by the National Library of New Zealand on: Jul 3 2006 at 4:42:47 GMT
Search boxes and external links may not function. Having trouble viewing this page? Click here
Close Minimize Help
Wayback Machine

New Zealand Department of Internal Affairs : Resource material : Part 3: Guidance Material (3.4.11 - 3.6) Skip to Access Key assignments for this site
Skip to the content of this page
Skip to other pages in this section
Skip to site wide navigation
Skip to links for help with this site
Logo of the New Zealand Department of Internal Affairs



About us

What's new




Other pages in this section

Draft Evidence of Identity Standard (html version)


Return to top of page Content area

Part 3: Guidance Material (3.4.11 - 3.6)

    3.4.11 Trusted referees
    Trusted referees are a vital component of the EOI process. In particular, trusted referees can assist an agency with determining whether the presenting person links to the claimed identity.

    For the purpose of the EOI Standard, a trusted referee is a person who:
    • confirms the accuracy of information supplied by an individual, and
    • confirms that the information, to their knowledge, belongs to that person (this may include biographical and biometric (usually a photograph) information.

    The two key elements that should exist for a trusted referee process to be effective are that the referee:
    • has the personal knowledge required to verify the individual’s identity
    • is trusted by the agency (according to the agency’s own criteria). Criteria for trusted referees
    Agencies will need to determine who qualifies as trusted referees for a particular service. Ideally, a trusted referee will be known to the agency. This means they will have previously had their identity established by the agency, thereby creating a level of trust between the agency and the referee.

    In addition, further criteria will need to be set around who qualifies as a trusted referee for specific services. The criteria chosen should be widely enough defined that individuals can reasonably be expected to find referees to fulfil the criteria.

    Criteria could include that a trusted referee:
    • is not related to the applicant
    • is not a partner or spouse of the applicant
    • is not resident at the same address as the applicant
    • has known the applicant for a specific amount of time (e.g. at least 12 months)
    • holds a particular position of standing in the community (e.g., registered professional, kaumatua5, religious or community group leader)
    • has an accessible contact address and phone number.
    5 Criteria for who is recognised as kaumatua are determined at the individual agency level.

    NOTE –
    To meet the High EOI Confidence Level process, there should be a process allowing the agency to contact the trusted referee directly to confirm their details. At a minimum, trusted referees should be contacted if any discrepancy is identified as part of EOI checking processes (see Legislative implications for trusted referee processes
    The practice of using trusted referees can be established through operational procedures and may not necessarily need to be enshrined in legislation. Current passports administration in New Zealand provides a precedent for this approach being introduced without legislation.

    Agencies should liaise with their legal advisors when considering implementation of a trusted referee process, to ensure that the process is legitimate in regard to the particular legislation that the agency operates within. Privacy implications for trusted referee processes
    A potential privacy issue arises with agencies requiring the use of trusted referees for particular services, given that the individual will be required to disclose to a trusted referee the service that he/she wishes to access from government; i.e. in order to meet the agency’s trusted referee requirements, should an individual have to let a third party know that he/she is applying for an unemployment or sickness benefit?

    It is advisable, if customers are expected to face personal costs or risks (of any nature) from disclosing to a third party the service that they are applying for, that the agency design service application forms in a manner that does not associate the core application data with the trusted referee’s input to the application. Strengths and limitations of trusted referees
    Use of trusted referees can be a cost-effective way to provide confidence that an individual ‘links’ to the identity they have claimed as their own.

    For a High EOI Confidence Level process, agencies should contact trusted referees for validation of the information that the applicant and possibly the trusted referee has provided (at least for a percentage of applications). In addition, a profiling approach should be considered for services for which an agency cannot justify contacting each referee for further validation, or for services for which the identity of some applicants should be subjected to more rigorous verification than others. This should include contacting trusted referees where discrepancies in individual applications are detected and on the basis of other risk indicators. Any trusted referee who verified the identity of an allegedly fraudulent customer should also be investigated.

    Specific risk indicators will need to be determined by the individual agency in relation to the nature of particular services. If there is an unacceptable level of doubt in the validity of the either the individual’s claimed identity or that of the trusted referee, the agency should progress the case to more thorough investigation.

    3.4.12 In-person verification processes
    In-person verification involves the individual appearing in person at a public counter with a photograph verified by a trusted referee, or with a trusted document that contains a photograph (such as a passport). Staff members then assess whether the person in front of them matches the person in the photograph, and/or whether the person before them appears to correspond with the biographical data on the document (e.g. whether they appear to be the correct age and gender).

    For an in-person verification process to be effective, it should be of the highest integrity. For example, staff should be trained in person recognition. The costs involved in carrying out a high integrity in-person verification process for all customers are high. Costs include financial cost to the agency and compliance costs for customers (e.g. inconvenience and accessibility issues). Therefore, when deciding whether in-person verification should be adopted by the agency, consideration should be given to whether the agency requires the individual to appear in-person for other reasons (e.g. to determine the individual’s entitlement to the particular service).

    Agencies need to assess the benefits of seeing an individual in person. Factors for consideration include:
    • costs and benefits (e.g. whether the compliance costs such as agency infrastructure and travel for the individual is unreasonably high in relation to the benefits gained)
    • whether an alternative process will achieve the same result, such as having a trusted referee verify the individual’s identity by signing the back of their photo
    • whether in-person contact can fulfil objectives additional to meeting EOI requirements (a number of services already require face-to-face contact with individuals). Strengths and limitations of in-person verification
    In-person verification provides potential ‘barriers’ for a person undertaking identity fraud. This is particularly likely to be the case where an individual is attempting to use a stolen identity.

    However, there are also limitations to the effectiveness of in-person verification. Although research has found that photographic facial recognition is not 100% accurate, its success rate is already high enough to add a significant degree of confidence to the EOI process, as long as it is used in combination with other forms of EOI. What is unclear, however, is whether face-to-face contact with an individual is any more successful in meeting Objective C (presenting person links to identity) than use of other processes, such as the use of trusted referees.

    3.4.13 Dealing with discrepancies
    This section outlines secondary EOI processes that should be used where a discrepancy is detected in the EOI documentation provided by an individual or trusted referee. The process is similar to that required in situations where discrepancies are identified during in-person verification processes (see 3.4.12).
    • Staff should first seek an explanation from the applicant for the service, unless it is clearly apparent that it is a fraudulent matter (in which case, the matter should be forwarded directly to investigations staff).
    • If the applicant’s explanation is not satisfactory, applications should be investigated further (this may be referred to investigations staff).
    • Discrepancies between documents/records regarding names or dates/places of birth should be resolved before continuing with a service. Agencies should not issue any documents that could be used subsequently to establish identity without resolving such discrepancies, as this may allow an individual to operate fraudulently under more than one identity.
    • Agencies should refer applicants back to the issuing authority to seek amendment and/or replacement of a document if it is incorrect.
    • Agencies should consult with their internal teams who deal with investigating discrepancies about what information from a document is useful to retain for investigative purposes (e.g. retain a photocopy versus record date of issuance and serial number).
    • Where documents are viewed with suspicion as possibly being fraudulent, agencies should not return such documents to an applicant until the individual’s identity has been fully established, unless it is unlawful for the agency to do so. Loss of evidence in such a way could significantly jeopardise any action an agency may wish to take against a fraudulent applicant.

    3.4.14 Investigative interviewing processes
    Investigative interviewing offers a higher degree of confidence than the in-person verification process outlined in section 3.4.12. An investigative interview involves the interviewer collecting identity-related information about an individual prior to the interview and preparing questions that the person claiming that particular identity should reasonably be expected to answer correctly.

    Because of the cost (both for the agency and the individual) and the level of agency staff training involved, investigative interviews should only be used where other EOI processes have not achieved the required level of confidence in the individual’s identity.

    3.4.15 Handling individual exceptions
    In some cases, individuals will be unable to meet the requirements of EOI processes. For these cases, agencies need to have exception-handling protocols in place. What these protocols involve will be determined by the agency, in relation to the particular service and customer base.

    Where possible, exception processes should be as functionally equivalent as possible to a service’s standard EOI processes. If a service requires a Moderate EOI Confidence Level process, the agency should attempt to meet the objectives that need to be met to satisfy the Moderate Level process by requiring alternative forms of EOI from the individual. For example, where an individual is unable to provide the required documentation to meet Objective A (i.e. evidence that their claimed identity exists) due to accidental destruction of all their personal documentation, the agency should contact the issuing agency of those documents, with the consent of the individual, to verify the existence of the claimed identity.

    3.4.16 Privacy requirements
    The Privacy Act 1993 covers the collection, disclosure and use of personal information. In designing and implementing an EOI process, agencies shall ensure that the process implemented is consistent with the Privacy Act 1993. Any consideration of the Act should be on the basis of agencies’ specific legal and privacy advice (see 3.4.22). The information that follows is not intended to substitute for that professional advice but is included to provide agencies with some preliminary guidance on the issues that may need to be considered. The Office of the Privacy Commissioner has developed a Privacy Impact Assessment Handbook to assist agencies in examining proposals that involve the collection, use or disclosure of personal information (available from: Key considerations
      Consideration of privacy issues is integral to the design and implementation of any EOI process. Key considerations that agencies should build into their EOI business processes include the following:
      • Agencies are responsible for ensuring, in cases where the customer gives consent to the access of data, that this is supported by adequate proof of that consent through a physical or digital signature (where allowed under the Electronic Transactions Act 2002).
      • Agencies are responsible for ensuring that individuals have access to and can correct any personal information held about them by the agency.
      • Some of the information collected from individuals during the EOI process will be necessary for agency records (e.g. name and address); other information may be provided to confirm a person’s identity but does not need to be retained by the agency after the EOI process has been completed. Agencies are responsible for ensuring that they do not keep information that is irrelevant or for which there is no legitimate purpose for retention. Agencies need to consider whether there is a legitimate business reason for retaining identity-related information collected from individuals or whether it is sufficient simply to record that the information was sighted.
      • Agencies are responsible for ensuring that information they collect for the purpose of verifying an individual’s identity is not used for any other purpose. However, in practice, personal information may be collected for several purposes at the same time. For example, an agency may collect evidence of an individual’s name, address, and date and place of birth in order to verify the individual’s identity, determine their eligibility for the service applied for, and ensure they can contact the individual in future. Agencies that are using the information for more than one purpose shall ensure that each purpose for which the information is being collected is clearly explained to the individual (e.g. purposes could be outlined on the application form for the relevant service).
      • Agencies are responsible for ensuring that information is not disclosed to another person or agency – with limited exceptions. If the EOI process includes the use of trusted referees as a way to verify identity, agencies should ensure that personal information is not unnecessarily disclosed to the trusted referee if the latter is contacted to verify identity information provided by the individual.
      • Agencies need to ensure no information systems (electronic or human-based) are vulnerable to inappropriate access by others (i.e. they are secure). For example agencies need to ensure identity information held on a database is not vulnerable to unauthorised access, or that frontline staff do not disclose identity-related information without sufficient checks to confirm the identity of the person requesting it. Agencies should periodically review the degree to which their systems and processes achieve the level of security that they have been designed to achieve – adjustments to systems and processes should be made where required.
      • Agencies are responsible for ensuring that they obtain the appropriate level of an applicant’s consent consistent with their need to acquire the data relevant to the determination of the applicant’s identity.

      For information on privacy considerations, refer to privacy officers or legal advisers in the first instance. Agencies may also wish to consult the Privacy Commissioner’s website (

      The Privacy Act 1993 is available from Collection of identity-related information from individuals
        At the heart of the Privacy Act 1993 are the notions of transparency and autonomy. Transparency is a precursor to autonomy. People cannot exert any control over the accuracy of their data or use of it until they know when data is being collected, who will have access to it and how it will be used. Transparency is also central to building customer trust.

        The Privacy Act requires agencies to advise individuals about the following:
        • why identity-related evidence is being collected
        • whether the information is required by law and, if so, the particular law
        • whether the supply is voluntary
        • the consequences if the required information is not supplied (e.g. that service provision may be withheld from the individual).

        The rationale for meeting objectives within an EOI process, and the documents/records that meet particular EOI objectives, could usefully be provided to the public to aid transparency of the process. For example, bankcards or utility bills provide confirmation that the person uses that identity in their daily life, and may be used to confirm the person’s current address. Agencies that request provision of these documents should provide advice to their customers that this is why these documents have been requested.
          3.4.17 Risk profiling
          Where appropriate, agencies may use risk profiling as a tool/approach to further mitigate identity-related risk in addition to the EOI process requirements specified in this Standard. Any risk profiling approach considered for adoption by the agency should be considered from a human rights perspective. The agency should liaise with its legal advisors in the first instance, particularly in regard to any human rights issues that may arise from use of a particular profiling tool.

          Risk profiling involves using information collected by an agency about previous cases where identity fraud (or other types of crime) was detected and from other sources (such as other government agencies, overseas counterparts and other intelligence sources), to highlight characteristics that are more likely to involve false identities.

          Agencies that use risk profiling may need to develop risk profiles that can be used as part of the process for establishing an individual’s identity. A risk profile highlights aspects about an individual that may indicate an increased risk of their perpetrating identity fraud. Where an individual application or the particular service fits within a risk profile, an agency may undertake additional processes to further verify the individual’s identity. For example, additional processes could include contacting trusted referees directly to validate information supplied by that referee, requiring the customer to attend an investigative interview, etc). The type of additional processes an agency chooses to undertake will need to be established as part of the overall EOI process design.

          Risk profiles should be updated to ensure their ongoing currency – relevant incident and/or intelligence information will provide valuable input to the refinement of agency risk profiles. As such, accountability mechanisms within agencies will be required to ensure updating happens in a timely manner.
            3.4.18 Data quality issues
            Accuracy of identity data is of key importance for any EOI process that an agency operates. EOI processes, once implemented, should be periodically audited for accuracy of identity information produced. Where unacceptable inaccuracies are found, the cause of the inaccuracies should be identified and resolved wherever possible.

            As a general rule, the greater the risk associated with inaccuracies in the identity data, the greater the effort that should be expended to improve and maintain the accuracy of the identity data held. This will also help to ensure that agency practices are compliant with Information Privacy Principle 8 of the Privacy Act 1993.

            3.4.19 Agents/persons acting on behalf of individuals
            EOI processes should be designed on the basis that personal information will be collected from the individual concerned when that individual applies for a government service. Agencies that receive service applications from agents or caregivers who are acting on behalf of the individual need to have processes in place to ensure that the agency/caregiver has authority to act for the recipient and that any personal information is provided with the individual’s consent or under some lawful authority (e.g. power of attorney or order issued by the Family Court under the Protection of Personal and Property Rights Act 1988).

            Where an agent is a named individual, agencies should consider whether they should verify that the agent is the named agent of the customer. This is recommended for services with moderate to high levels of identity-related risk.

            3.4.20 Step 3 – Ongoing operation of EOI processes

            Sections 3.4.21 to 3.4.25 provide guidance on areas that agencies shall consider prior to EOI processes being made operational.

            3.4.21 Internal controls
            Internal controls are an agency’s first line of defence in safeguarding assets, and in the prevention and detection of errors and fraud. Poor internal controls can jeopardise the effectiveness of any EOI process.

            Agencies should analyse their EOI process to determine the points at which internal controls need to be implemented to prevent process failure. EOI internal control activities are any policies, procedures, techniques, and mechanisms that minimise the risk that EOI processes will not meet their objectives. They include a diverse range of activities, such as:
            • controls over information processing
            • physical control over vulnerable assets
            • segregation of duties
            • access restrictions to, and accountability for, resources and records.

            There is a range and variety of EOI control activities that should be adopted by agencies carrying out EOI processes. An agency’s internal controls should be flexible enough to allow control activities to be tailored to fit particular contexts. The specific control activities used by a given agency may be different from those used by other agencies, due to a number of factors. These factors could include specific threats faced by the agency and risks incurred, differences in agency objectives, size and complexity of the agency, operational environment, sensitivity and value of data, and requirements for system reliability, availability, and performance.

   Operational considerations
            An agency’s human resource planning should allow for adequate EOI checking to be undertaken by staff. Agencies need to ensure that the workload given to staff is manageable. If staff members are unduly pressured for time or to meet targets there is a risk that their vigilance in identifying discrepancy cases may diminish. Complaints and errors should be analysed to determine their cause, so that remedies can then be applied appropriately and in a timely manner (see 3.4.24).

            Agencies should also ensure that they have adequate controls in place to prevent staff members perpetrating internal fraud, which can undermine the integrity of an agency’s EOI processes.

   Staff training
            Staff training should be comprehensive to ensure staff have an adequate understanding of the particular service’s EOI requirements and of the potential consequences should they fail to follow proper procedures. Specific areas where training is likely to be required include (but are not limited to):
            • document recognition and resources to assist with document recognition (see and
            • in-person verification processes (see 3.4.12)
            • the Privacy Act 1993 (see 3.4.16)
            • dealing with cases where individuals cannot meet EOI requirements.

   Physical control over vulnerable assets
            An agency should establish physical control to secure, limit access to, and safeguard vulnerable assets such as documents or records that might be vulnerable to risk of loss or unauthorised use. Physical files and records should be tracked in such a way that an audit trail clearly indicates where and with whom the files are located. Audit trails in computer systems should show records of all users, access information, the time and date of access, and before and after images of any changes.

   Segregation of duties
            Key duties and responsibilities need to be divided or segregated among different people to reduce the risks of error and internal fraud. This should include separating the responsibilities for authorising services, processing and recording them, reviewing the services, and handling any related assets. No one individual should control all key aspects of a service’s delivery. This is especially important when issuing any record that may be potentially used as evidence of identity for subsequent services.

   Accurate and timely recording of services
            Service delivery should be promptly recorded and processed to maintain its relevance and value to the control of operations and for latter evaluations. This applies to the entire process or life cycle of a service from the initiation and authorisation through to its final classification in summary records.

   Access restrictions to and accountability for identity-related records
            Access to resources and records should be limited to authorised individuals, and accountability for their custody and use should be assigned and maintained. Periodic reviews and monitoring of data are necessary to help reduce the risk of errors, fraud, misuse, or unauthorised alteration.
     Appropriate documentation of service delivery and internal controls
              Internal controls and all services need to be clearly documented, and the documentation should be readily available for examination. All documentation and records should be properly managed and maintained. Internal control monitoring should assess the quality of performance over time and ensure that the findings of audits and other reviews are promptly resolved.

              3.4.22 Legal considerations
              In identifying and implementing any changes to their business processes to meet the requirements of this Standard, agencies will need to identify any legal issues that might need to be addressed and ensure that they are dealt with appropriately. Examples of such issues include:
            • the need to make amendments to any Act or Regulation concerning the information to be collected from the applicant or the processes that applicants should undergo in order to receive a particular service from the agency
            • the need to obtain and act on legal advice associated with any changes in business processes, such as the introduction of more stringent identity verification checks that require the collection and sharing of greater amounts of personal information, and the implications of the Privacy Act 1993 on the range of strategies that the agency might consider to enable the carrying out of these verification checks.
                3.4.23 Transition of business processes
                The extent to which agencies will need to alter their current EOI processes to comply with this Standard will vary widely between agencies and, in some cases, also between different business groups within agencies.

                Agencies should identify the optimum transition strategy, given the particular type and extent of changes that need to be implemented by their organisation. Key considerations in planning the transition will include whether:
              • any changes to existing Acts or Statutory Regulations are required to implement the necessary changes to EOI processes
              • the required changes to business processes require action by other agencies in order to give effect to these new business processes (for example, the introduction of automated information matching with that agency). If so, the transition plan needs to address these dependencies.
              • the changes to be implemented will impact on the performance of the functions of another agency or service. If so, the changes should be discussed with the relevant agency or service unit before they are made.
              • there are any other issues that could constrain or prevent the agency from implementing the required changes. If there are, these issues need to be investigated and addressed before the agency finalises its transition plan, so that implementation of the changes can occur effectively.

                3.4.24 Complaints handling
                It is possible that changes to EOI processes, to ensure compliance with the EOI Standard, will result in changes to the number and/or type of complaints from individuals transacting with the agency. For example, individuals may consider the EOI requirements made of them by the agency to be overly intrusive, unnecessary, and so on. Each agency should make appropriate resource provision for this possibility when designing and testing these new EOI processes.

                Although most agencies are likely to have appropriate mechanisms already in place for handling complaints from customers, the need for expertise to process/investigate some types of complaints, such as complaints concerning potential breaches of the Privacy Act 1993, may be greater with the introduction of EOI processes required to comply with this Standard.

                Agencies should also have procedures in place for dealing with complaints about situations where they have incorrectly recorded identity-related information about an individual. Under the Privacy Act 1993, agencies should correct any personal information they hold about the individual. Alternatively, the agency can include a statement of changes sought by the individual and the reasons why it was not deemed appropriate to make the changes.

                If an agency fails to put in place appropriate administrative procedures for dealing with errors in identity details, or does not deal with any such complaints according to its procedures, this may give rise to an investigation by an Ombudsman, pursuant to the Ombudsmen Act 1975.

                3.4.25 Communication protocols between agencies
                Agencies should ensure that they monitor and evaluate the performance of their EOI processes and act in a timely manner on any issues that may need to be communicated to other agencies. Agencies should ensure, however, that any communication between agencies is in compliance with the Privacy Act 1993.

                Examples of such situations include:
              • the discovery of an EOI document that is issued by another agency and which is found to be:
                  • fraudulently obtained
                  • falsified in some way, or
                  • fraudulently used
              • the discovery of evidence (such as credible information) indicating that one or more EOI document issued by another agency has been, is likely to have been or is likely to be, either fraudulently obtained, falsified in some way or fraudulently used
              • the discovery of personal information regarding the activity of any person or persons who are found to have committed, or who are suspected to have committed, or who are suspected will commit, any illegal activity in relation to identity or the use of identity-related documentation.

                3.4.26 Checklist for Phase 2 Design and Operation
                  Checklist for Design and Operation

                  Checklist for . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . (service name)
                  StepTick when completed
                  1EOI Confidence Level determined (based on Service Risk Category)
                  2Design an EOI process, or modify existing EOI process, appropriate to EOI Confidence Level:
                  ‘Gap’ analysis completed (where existing EOI processes)
                  Minimum evidential requirements for EOI Confidence Level met
                  Transition of business processes planned
                  3Implementation of EOI process (either new or modified process)
                  Internal controls implemented
                  Legal aspects (including privacy) signed-off
                  Complaints handling implemented
                  Communication protocols with other agencies implemented
                  4Monitor, evaluate, and review on.......................................
                  3.5 Service delivery phase

                    The service delivery phase is not within the scope of this Standard, with the exception of any EOI process that is required before the service can be delivered to an individual. Detail about EOI processes is covered in 3.4.
                    3.6 Monitoring and evaluation phase
                    This section provides guidance on the monitoring and evaluation processes when establishing the identity of individuals6. It should assist agencies to develop a formal EOI process monitoring and evaluation plan.

                    The purpose of the monitoring and evaluation phase is to ensure that once implemented, each agency’s EOI business processes and associated outcomes remain consistent with their objectives.

                    6Services that require evidence of identity will have been determined during the risk assessment phase (see 3.3).
                      3.6.1 Continual improvement of EOI processes
                      Agencies need to modify their EOI processes where monitoring and evaluation results indicate that EOI process objectives are not being satisfactorily met and are consequently exposing the agency to unacceptable identity-related risk. This is an iterative process, as is shown within the ongoing cycle of monitoring, evaluation and process improvement in Figure 7.

                      In carrying out EOI process evaluation, agencies need to consider systematically what is wanted (i.e. their objectives) in relation to their EOI processes, and what might adversely influence the achievement of those objectives (i.e. the EOI process risks). Just as the context and nature of the risks dictate the types of risk analysis and risk evaluation carried out, context also dictates the design and implementation of appropriate monitoring and evaluation procedures.

                      Events (such as the use of forged EOI documents) that occur over time also allow an agency to improve its understanding of the likelihood and consequences of the risks, and to influence the ongoing maintenance of appropriate business processes to address them. In essence, monitoring processes provide this information and evaluation processes analyse it in order to identify any required improvements. For example, risk profiles (see 3.4.17) may be modified on the basis of information collected as part of monitoring processes in place of an EOI process.
                                Figure 7 – Monitoring and evaluation cycle

                        3.6.2 Monitoring and evaluation approaches
                        Identity-related risks vary between services. Monitoring and evaluation processes should, therefore, be tailored to the individual contexts of each agency.

                        This Standard does not prescribe maximum or minimum intervals between monitoring and evaluation cycles. These decisions will be the responsibility of each individual agency. However, agencies should ensure they document the basis for their approach to monitoring and evaluation and should maintain up-to-date documentation for auditing and quality assurance purposes. It is recommended that agencies review their EOI process monitoring and evaluation practices at least every two years.

                        3.6.3 Step 1 – Develop monitoring and evaluation plan
                        A monitoring and evaluation plan should be completed prior to an EOI process becoming operational. This plan should be completed as part of the design and operation phase. If this is not practical, information about monitoring and evaluation processes should be incorporated into the business and risk management documentation that relates to the particular service.

               Monitoring processes and performance indicators
                        Selecting appropriate indicators to measure the effectiveness of EOI processes is extremely important. ‘Performance indicators’ need to inform assessments about the degree to which the EOI processes meet the agency’s EOI-related objectives.

                        An agency will need to factor in the following considerations when choosing its performance indicators:
                        • Cost to the agency
                        • Ability to collect the required data/information
                        • Reliability of the performance indicator.

                        Table 16 provides a list of types of performance indicators that could be used for monitoring EOI processes. It is not an exhaustive list. Agencies should select performance indicators most relevant to their specific risks and desired outcomes, ensuring that they only monitor performance against a manageable number of indicators.

                        Table 16 – Performance indicators
                        IndicatorExample of measure
                        Emergence of any new EOI process risksFor example, analysis of discovered fraudulent activities to determine whether new modes of operation are being used to perpetrate identity fraud in relation to agency services.
                        Quality of administrative activitiesFor example, measurement of the proportion of processing errors found within EOI processes. An example error could be that individuals’ identities were established without the required EOI documentation being confirmed by a staff member.
                        Compliance with the Privacy ActFor example, the proportion of EOI processes (for identifying individuals) that are found (from an audit or similar) to have been in breach of the Privacy Act due to the actions of staff involved with the service.
                        Staff training and performanceFor example, measurement of the results of tests of staff regarding their ability to correctly identify authentic and non-authentic documents of the type used for EOI processes they administer.
                        Cost and effort associated with EOI processesFor example, measurement of the amount of money, time or other measure invested in the design and operation of EOI processes compared with the outcomes of those processes in relation to identity fraud rates or customer satisfaction.
                        Feedback from other agenciesFor example, analysis of the number of instances where other agencies have reported the use by individuals of identity-related documents, issued by the agency, that contain errors or that are stolen or counterfeit.
                        Alignment between EOI processes and objectivesFor example, measurement of any increase in the number of false identities detected per annum through implementation of an EOI process designed to better deter false identities.
                          The examples of performance indicators contained in Table 16 highlight the importance of efforts to identify specific cause and effect relationships between the achievement of business objectives and the indicators being measured. Where strong cause and effect relationships do exist, changes in the results of data collection will indicate corresponding changes in the achievement of business objectives. Each agency’s choice of performance indicators should enable the agency to remain informed about the degree to which its business objectives are being met.
                   Collection of data/information
                            Various methods can be used to collect monitoring data/information. These range from simply gathering feedback or descriptions of success or failure, to systematically gathering qualitative and/or quantitative data for statistical analysis. Collection methods include:
                          • routine checks of EOI process steps
                          • audit of EOI processes
                          • maintenance of a risk register for EOI processes
                          • maintenance of a database recording details of EOI process errors or failures
                          • costing information about EOI processes
                          • collection of customer complaints information regarding EOI processes, and any other customer feedback
                          • surveys of customer satisfaction.

                   Appropriateness of monitoring
                            The types and amount of monitoring chosen should provide agencies with information in a time frame and format that allows decisions about the suitability of the EOI processes to be assessed.

                            In addition, in the event of any unacceptable departure from, or mismatch between, these EOI processes and objectives, the monitoring processes used should allow process changes to be designed and implemented before significant problems arise.

                            3.6.4 Evaluation processes
                            The following aspects of evaluation shall be taken in account in plan development.
                     Designing evaluation processes
                              As a general rule, an agency’s EOI processes should be changed when the criteria for success of those processes are not met and the expense and/or effort required to improve the outcomes is justified. Evaluation processes allow these assessments of adequacy and the identification of appropriate improvements to be made.

                              In carrying out evaluation processes, agencies shall document the:
                            • rationale for all EOI business processes
                            • key EOI process objectives to be achieved and the context within which the evaluation is conducted
                            • performance indicators used (see 3.6.3) as a basis for the evaluation
                            • results (against those performance indicators) that the agency considers represent a contribution to outcomes – successful or otherwise.

                              Evaluation processes should be designed to inform decisions properly about the appropriateness of the EOI processes, so that those processes can be amended as appropriate. Evaluation processes should therefore be tailored to the specific situation in which they are being carried out. These processes should be designed in conjunction with the design of monitoring processes for agency EOI processes.

                              The design of evaluation processes is particularly important if specific interventions are to be assessed. For example, if a new business process is to be introduced in order to reduce the number of processing errors, the effect of the new process on the number of errors will need to be measured to gauge the success of the new process. Information will also need to be retained about the previous process, so that improvements can be measured. Where such changes to process are expected to affect another agency, the changes should be agreed to by all affected agencies prior to implementation.

                              It is very important that agencies identify and document the factors that will be taken into account during the evaluation of EOI processes. This transparency assists any external review of the appropriateness of the EOI processes and helps the agency to implement and maintain appropriate EOI processes.

                     Issues for evaluation
                              Table 17 provides examples of the types of issues that an agency may evaluate in relation to its EOI processes. This is not a complete list and the issues chosen for evaluation will need to take into account the agency context and the objectives within which particular services operate.
                                    Table 17 – Issues for evaluation
                                    Solutions to address identity-related risks For example, evaluation of whether new EOI checks on applications for welfare benefits resulted in an increase in the discovered rate of identity-related benefit fraud.
                                    Implications of ongoing initiatives, such as communication with agencies about downstream effects to existing operational proceduresFor example, the consideration, through discussions with other agencies, of possible measures to counter EOI process concerns that have been raised. The introduction of any new EOI checks should be weighed in light of the expected benefits they would bring to the affected agencies and the expected costs and operational changes that would need to be incurred with the new EOI checks (i.e. cost/benefit analysis). Performance indicators relating to those EOI checks (such as the number of discovered false identity events per year) would need to be identified and the resulting information periodically evaluated.
                                    Possible solutions to EOI process problems identified, such as breaches of the Privacy ActFor example, evaluation of the effectiveness of staff training courses aimed at improving staff’s compliance with the Privacy Act – as evidenced by a reduction in the number of breaches of the Act by staff.
                                    Adequacy of staff training and performanceFor example, the performance of staff as measured by regular performance assessments of the quality of their EOI process-related activities (such as quality of document verification checks and customer liaison activities).
                                    Possible improvements to EOI processes, such as efficiency gainsFor example, an agency might identify patterns of EOI-related concerns (such as attempts to use non-authentic EOI documentation). The agency may then channel future EOI processing to selected staff members who have been trained more comprehensively. In doing so, the agency might be aiming to reduce its processing costs while maintaining or even improving its level of success in meeting its business objectives. The monitoring of those business outcomes provides the agency with the information it needs to evaluate whether such initiatives are achieving their desired outcome.
                                Where evaluation results in information that may be of use to other agencies or to the Custodian of the Standard, and which can appropriately be shared, agencies are encouraged to share the results of their evaluations – for example, where an agency’s evaluation activities identify changes in the nature of identity fraud methods.
                                  3.6.5 Step 2 – Ongoing monitoring and evaluation
                                  Once an EOI process becomes operational, the monitoring and evaluation processes should commence.
                           Frequency of monitoring and evaluation activities
                                    Monitoring and evaluation will be undertaken at different frequencies, depending on the particular context within which a service exists. Monitoring and evaluation can be undertaken:
                                    • continuously – In some situations, such as in rapidly changing environments, it may be appropriate to monitor the adequacy of EOI processes more or less continuously (e.g. in cases where a particular service results in the issue of identity documentation that can be used subsequently).
                                    • periodically (on a discrete-interval basis) – This approach is likely to be appropriate for evaluation of most services.
                                    • episodically (event dependent) – This is likely to be the most appropriate approach after the completion of major changes to business processes.

                                    In all situations, the underlying rationale for the choice of frequency is the need to keep pace with the rate of change in the data or information that is being measured, so that any unacceptable deviations from desired performance of the EOI processes (unacceptable consequences) are avoided wherever possible, or resolved where this has not been possible.

                                    The frequency of monitoring and evaluation needs to be influenced by both the rate at which the identity-related risks can change and the extent to which any changes are important. In many cases, a change in one of these factors also affects the other.

                                    The intervals between evaluation cycles can be increased when the circumstances being evaluated have not materially changed from the circumstances that were evaluated last. Accordingly, each agency should revise its evaluation processes in light of experience gained from previous monitoring and evaluation cycles.

                                    NOTE –
                                    Episodic evaluation can be either in addition to or instead of periodic evaluation, depending on the extent to which outcomes may deviate from business objectives.

                           Changing monitoring processes
                                    The type of monitoring should be changed if the current monitoring does not allow an assessment of the appropriateness and effectiveness of the EOI processes relative to the EOI objectives to be assessed with the degree of timeliness and adequacy that is wanted.

                                    More monitoring should be undertaken when additional monitoring of the same type is expected to yield additional information that justifies the additional effort. This issue of justification relates to how valuable the additional monitoring is expected to be relative to the expense or effort of conducting it.

                                    Changes to monitoring regimes often involve changes to both the types of monitoring and the overall amount of monitoring that is undertaken, particularly until the agency establishes a good understanding of its exposure to identity-related risk and the extent to which its EOI processes address it. Once these positions have been established, agencies will be able to adjust their monitoring and evaluation activities more efficiently to maintain the identity-related risks associated with their services at an acceptable level.

                                    3.6.6 Step 3 – Amend EOI processes
                                    Where evaluation processes indicate that EOI processes are not sufficiently mitigating identity-related risk or meeting objectives, consideration shall be given to amending EOI processes.

                                    Any amendments to an agency’s EOI process shall be subject to the same consideration as the initial design (see 3.4.7). In addition, any amended process should be fully tested before becoming part of ongoing operation.

                                    Next page: Appendices

                                    Return to top of page
                                    Links for help with this site

                                  • Site map



                                    Contact us

                                    Email us


                                    Return to top of page

                                    Last updated: 06/12/2005