Audit of notetaking

Roles, responsibilities and accountabilities

26. Roles and responsibilities define what stakeholders should do to perform their assigned duties. Accountability is the means that ensures responsibilities are being executed and stakeholders are answerable for their actions.

27. The audit assessed whether roles, responsibilities and accountabilities for the oversight of notetaking were clear and occurring at appropriate levels per policy requirements.

28. The following graphic outlines the oversight responsibilities as described in the notetaking policy.

Figure 1: Oversight responsibilities in the notetaking policy

Text description

Port level

• BSOs document activities they perform on duty.
• Superintendents conduct reviews of officer notebooks biannually using a checklist (BSF858) and also perform ad-hoc compliance checks ("spot checks") periodically.
• Chiefs periodically review the completed checklists (BSF858) and their local tracking mechanism, to monitor review scores and ensure all BSOs are reviewed twice annually. Data is then provided as requested.

Regional level

Directors and Regional Directors General monitor their local tracking tool and review scores, to ensure all BSOs are reviewed twice annually and that scores are within acceptable levels. Data is then provided as requested.

National level

National Headquarters – Travellers Branch monitor national performance related to notebook reviews and regional compliance as well as develop the notetaking policy and provide guidance to regional personnel.

29. Roles and responsibilities were clearly outlined in the notetaking policy. In the absence of well defined accountabilities, some roles and responsibilities were not being fully executed at most levels (such as: reviewing oversight results / monitoring performance and ensuring compliance with the notetaking policy).

The notetaking policy states that stakeholders (that is chiefs, directors and RDGs) are responsible for providing data related to notebook reviews and port compliance as requested. The policy also states that it is the responsibility of TB to monitor national performance related to notebook reviews, ensure regional compliance and provide guidance. TB was not requesting this data nor providing feedback in order to confirm or reinforce that stakeholders were properly fulfilling their oversight responsibilities per policy.

The level of awareness of responsibilities per the notetaking policy varied. Those further removed from port of entry operations seemed less aware of their specific responsibilities related to notetaking oversight.

30. The notetaking policy is developed and maintained by NHQ – TB. The principles contained in the policy apply generally across the agency, in all operational areas. There are also program areas with specialized functions with additional notetaking requirements, which were not included as part of the policy. Some of these specific policies or requirements were referenced (either listed or hyperlinked) in the notetaking policy. However, the list was incomplete, not easily accessible and some policies were outdated.

The agency would benefit from all program areas confirming their notetaking-related guidance, where required. Incorporating or linking additional notetaking requirements for all specialized functions and operational areas in one policy would ensure the agency has a one-stop policy solution for notebooks and notetaking that applies throughout the agency, in all operational areas.

31. Risk and impact:

Risk

There may be a lack of accountabilities in place to confirm or ensure that responsibilities are being executed.

Impact

If stakeholders are not held accountable for their responsibilities per policy, the agency will not have the sufficient oversight in place to confirm compliance with the policy and improve notetaking practices.

Refer to Recommendation 1.

Oversight and monitoring compliance

32. Performing oversight and monitoring compliance with policy provides insight into whether notetaking practices are meeting program requirements and expectations. It also fosters learning through highlighting good or bad practices and raises awareness which can lead to positive change in behaviours and culture.

33. As per the notetaking policy, the main notetaking oversight and monitoring mechanisms are formal biannual reviews, and ad-hoc compliance checks (spot checks). Results of oversight activities are to be reviewed by regional management and NHQ – TB.

34. The audit tested a sample of biannual review checklists (BSF858) to determine if the oversight of notebooks were being conducted and also assessed if the results were being rolled up from the POEs up to NHQ. The process of conducting ad-hoc compliance checks was also reviewed.

35. Oversight requirements per policy:

• Biannual reviews should occur six months apart, as practicable, to ensure officers have an opportunity to respond to any areas requiring improvement.
  • Superintendents review a minimum of three (3) notebook entries at each biannual review, at least one (1) of which must involve a notable event^{Footnote 5}, if available. Completed Officer Notebook Review Checklists (BSF858) are kept in the officer's personnel file.
  • Chiefs, Directors and RDGs review the results to ensure notebooks are reviewed twice a year and that officer scores are within acceptable levels. NHQ – TB monitors national performance related to notebook reviews and regional compliance.
• Spot checks should include a cursory scan of an officer's notebook to ensure an officer is following the notebook and notetaking policy and procedures. Spot checks can be used to quickly identify areas for improvement outside of the formal biannual review process.
  • Spot checks do not need to be tracked in the port of entry's local notebook tracking tool. When a superintendent conducts a spot check of an officer's notebook, the superintendent is to date and initial the notebook entries reviewed.

36. The biannual review of notebooks (using the BSF858) was conducted, in some form, by all regions:

• 11/12 POEs in our sample were conducting the biannual review
• testing identified common inconsistencies/errors in approximately 58% of all biannual checklists that were reviewed (that is, not documenting the dates of entries reviewed, missing signatures, etc.)
• testing identified inconsistencies in the retention of checklists (that is, in some cases, completed checklists couldn't be located to support our testing, or results were entered directly in the tracker)

37. Roll-up of oversight results to the district and/or regional level was done inconsistently and to a limited extent in most regions.

Testing demonstrated that roll-up was being done at a regional level in the Pacific Region and the Northern Ontario Region including analytics, comparison of results between multiple periods and communicating any gaps.

38. Roll-up of oversight results and monitoring of national performance related to notebook reviews and regional compliance were not completed by NHQ – TB.

The policy defines responsibility to monitor the results of the notebook oversight. Monitoring was not prioritized by NHQ – TB and the oversight accountabilities were not fulfilled.

39. Ad-hoc compliance checks (spot checks) were not common practice. As per the notetaking policy, spot checks do not need to be tracked^{Footnote 6}. Therefore, it would be challenging to confirm compliance, even if Superintendents had attested to doing them.

40. Risk and impact:

Risk

Oversight and monitoring may not be occurring in compliance with policy.

Impact

POE management, regional management, and supporting HQ programs may not have insight into whether notetaking practices are meeting policy expectations and cannot take corrective actions.

Refer to Recommendation 2.

Sufficiency of current oversight

41. Oversight practices contribute to the improvement of the notetaking process by identifying and sharing gaps and best practices. The principles of notetaking contained in the policy apply throughout the agency, in all operational areas (refer to examples listed below). The manner in which notes are recorded may vary depending upon the specific environment in which an officer is working, where there may be specialized functions with additional notetaking requirements. While officers in specialized areas may refer to the principles in the notetaking policy, they must follow the policies associated to their role. Other considerations could impact notetaking such as port size, volume, schedule and time of the year.

42. The audit assessed whether current oversight and monitoring can be relied on to confirm proper notetaking and the sufficiency of notes.

Examples of operational areas:

Modes

• Air
• Highway
• Rail
• Marine
• Postal

Streams

• Commercial
• Traveller
• Immigration
• Enforcement

43. The audit conducted interviews with frontline officers to gain their perspective on the sufficiency of oversight from their experience. Overall, no concerns were raised by most of the interviewees about the degree of current oversight. The result of facilitated officer feedback sessions indicated that most officers expressed that current oversight has helped improve notetaking practices.

44. Feedback from frontline officers on the oversight they currently experience:

• What degree of oversight over your notetaking do you experience? [select one]
  (Based on 74 responses from frontline officers.)
  • Enough: 45%
  • Not enough: 39%
  • Too much: 16%
• Which of the following do you feel about current oversight? [select any that resonate]
  (Based on 100 responses from frontline officers.)
  • It helps remind me to take notes: 31%
  • It helps me to improve my notetaking: 21%
  • I don't see the value in it: 16%
  • It is not helpful at all: 13%
  • I learn about good notetaking practices: 11%
  • it takes too much time: 8%

Officers were taking more notes/using up more notebooks as a result of the new policy/notebook reviews.

Many officers and superintendents expressed concern that excessive oversight could foster a perceived lack-of-trust between officers and management and/or discourage notetaking.

from feedback/sentiments gathered through interviews and group sessions

Refer to Appendix D for details on facilitated sessions with frontline officers.

Factors impacting notetaking

52. The agency expects that officers exercise proper notetaking practices and take sufficient notes. Understanding factors impacting notetaking practices is the first step in determining why officers' notetaking practices may not produce notes that can be relied upon. According to the notetaking policy, an officer's notes must follow specific requirements during the course of their daily shift (refer to example below).

53. The audit assessed factors impacting officers exercising proper notetaking practices and taking sufficient notes. Through interviews, group discussions and audit testing, the audit team gathered and assessed feedback on impediments to proper notetaking.

Daily notetaking structure:

• as per the policy, there are specific requirements for start of shift, during shift, and end of shift notetaking
• notes are required for any incidents, occurrences, enforcement actions, or notable observations, including but not limited to 22 notable events listed in the policy

General notetaking requirements:

• per policy, officers should ensure the following in all notetaking:
  • be clear, concise, legible, understandable, accurate, complete, and sequential
  • be factual, unbiased, and provide an account of the events officers have witnessed or participated in, as well as any decisions made by officers
  • thoroughly describe the details of the occurrence and answer: who, what, when, where, why and how
  • contain professional language, unless documenting verbatim comments

54. Frontline staff expressed or demonstrated a lack of clarity about:

• what needs to be documented and how, such as:
  • lack of clarity on application of general notetaking requirements and guidance included in the main policy vs. other policies and guidelines associated with specialized functions
• where information should be notated/documented, such as:
  • how or when notes and narratives that are recorded in various mediums (for example, notebooks, systems, stat decks, and will say) can be relied upon as documentary evidence for investigations and court proceedings
  • lack of clarity between where and/or when notes are required in a notebook, in a system or both (for example, raised by immigration and postal officers)

55. Discrepancy in the perception of the sufficiency and/or use of notes between frontline and NHQ:

• for the most part, frontline officers did not consider their notes to be lacking; however, feedback from senior management, program areas, previous audits and reviews had highlighted concerns with insufficient notes
• some officers felt that notes are solely a personal tool to help refresh their memory, rather than providing support to the agency in the form of a record of actions taken to provide evidence, and a justification of decisions made

56. Overall throughout the agency, there was no evidence that reocurring notetaking concerns were documented and analyzed for trends in order to identify and address gaps (excluding analysis of results regarding biannual reviews in a few regions and review of Digital Device Examinations notes).

The lack of quantification/tracking of notetaking gaps limits the agency's understanding of the extent of the issues surrounding improper notetaking.

Tracking would identify areas where there are gaps in current guidance/processes and subsequently guide improvements. For example, two regions collected and consolidated data from their districts to analyze trends, identify gaps and determine recommendations which led to an action plan.

57. Interviews indicated that notetaking related gaps such as quality and consistency of notes are often communicated to program areas only after cases go to court.

58. Lack of awareness was often identified during audit interviews and discussions as a factor impacting notetaking. Many stakeholders expressed a desire for increased awareness around proper notetaking practices and what holds up in court or is required for internal/external investigations.

• The results of the audit's root cause analysis also highlighted a "lack of awareness" as a major factor contributing to the insufficiency of notes
• Stakeholders felt that buy-in for notetaking can be improved through better overall awareness of the "so-what?"/"why?"

59. Some of the reasons why a lack of awareness can hinder good notetaking practices are as follows:

• officers may never have experienced the impact of insufficient notes (for example, if they are called to testify in court or provide evidence long after an incident has occurred)
• officers may be less likely to prioritize notes if notetaking is not encouraged by POE management or colleagues
• officers may not realize the importance of making time to document notes
• officers may not understand the implications of delaying or forgetting notetaking
• officers may not be clear on why notetaking should be prioritized
• officers may make their own judgement around perceived duplication or redundancy and limit their notetaking if rationale is not clear

60. Refer to Appendix C for details on root cause analysis for this finding.

61. Risk and impact:

Risk

Frontline staff may experience a limited awareness of the importance of notetaking and a lack of clarity around notetaking expectations.

Impact

Notetaking practices are not executed as expected/required, and notes are insufficient to meet their required purpose.

Introduction

62. The following graphic outlines the notebook lifecycle as described in the updated notetaking policy.

Figure 2: Notebook lifecycle

Text description

Notebooks need to be accessible (Notetaking Policy ENF Part 8, Chapter 1, 14-19).

1. Blank notebooks provided to POE
2. Notebook issued to Officer
3. In-use by Officer
   • Stored during long-term leave
   • Lost stolen or damaged (refer to step 2)
4. Returned upon departure (POE or agency) or returned upon completion (refer to step 2)
5. Notebooks returned upon departure and completed notebooks are stored by POE
6. Destroyed following retention period

Storage of in-use notebooks

67. As per Treasury Board Secretariat Guidelines, notebooks are a controlled asset and must be stored on-site in a locked container or other container appropriate for storing Protected B information. When notebooks are stored securely on-site, per the policy requirements, they are safeguarded and can be quickly accessed when needed.

68. The audit assessed whether notebooks currently in-use are stored securely on-site, in a locked container that is accessible per the notetaking policy.

69. All POEs interviewed or visited attested to officer notebooks being accessible.

• Many POEs required storage of the in-use notebook in officer Dasco lockers (with other CBSA controlled assets such as firearms).
• Some POE environments were not setup for all officers to use Dascos for notebook storage. For example, at one POE in our sample, not all employees were armed/assigned Dascos and at other POEs there could be limited space in arming rooms.
• POE management attested that notebooks were maintained on-site (that is in Dascos or personal lockers with protective vest and duty belt). When on-site at POEs that required storage in Dascos, audit was able to access approximately 68% [27 of 40 notebooks] of notebooks selected for off-shift officers.
  • Where notebooks were not in the Dasco, the next expected place of storage was the officer's personal locker.
  • The audit did not access the contents of personal lockers. All POEs who allowed storage in the personal locker maintained that notebooks were accessible if required.
  • Given that the audit did not access the personal lockers, we could not confirm that the notebooks were in fact stored on-site.
  • Without confirming with officers if they store their in-use notebook in their Dasco or personal locker, POEs were not able to quickly identify the current location of a notebook.

70. Per interviews, POEs had processes in place to ensure access to in-use notebooks of employees on long-term leave.

• When on-site for testing, the audit team was able to access approximately 57% [8 of 14] of requested notebooks for selected employees on leave. This result was not representative (as the sample was small and limited to POEs that were visited) and it was influenced by the storage practices of each POE.
• Where the notebooks for this sample could not be found in the Dascos, there were other places that POE management mentioned they could be stored (that is personal lockers, with POE security team or administrative staff).

71. Ports did not have any concerns with lost, stolen or damaged notebooks, and mentioned very rare instances of such occurrences. A few examples that POEs provided pre-dated the implementation of the new policy.

POE staff indicated that lost, stolen or damaged notebooks would be noted in the notebook tracking tool and that a security incident form would be completed and submitted. One instance of a damaged notebook, documented in the tracker, was observed.

72. Although the POEs indicated that they do not have any major concerns about accessibility of in-use notebooks, based on previous audits and the results of our limited testing, there are indications that the process of storing in-use notebooks is still maturing.

Retention of used notebooks

73. Proper retention of used notebooks per policy ensures that notebooks are accessible when needed, and can be relied on to support officers and the agency.

74. The audit assessed whether used notebooks are collected and stored per the policy and can be accessed as needed.

75. Most POEs were still in the process of formalizing notebook retention and the approach toward retention varied. For the most part, used notebooks were stored in a secure room or cabinet with access limited to management or administrative staff.

Retention was generally handled by POE administrative staff (that is logging and storage of completed notebooks).

The return of old notebooks was not required or fully reinforced at some sampled POEs.

76. The audit was able to access approximately 88% of requested samples [44/50] while on-site.

While compliance with accessibility of retained notebooks was high (approximately 88%), several POEs that we visited mentioned that retention practices may have been put in place / improved in anticipation of our arrival. Therefore, compliance may not be as high elsewhere.

77. Management consideration:

• to improve the communication process, and encourage and monitor progress in this area
• to consider compliance checks in the future via the Operational Program Assessment function
• to consider obtaining feedback from programs/NHQ (that is if notes/notebooks are consistently unavailable) to determine if any future compliance checks are needed

Tracking of notebooks

78. As per the notetaking policy, notebooks must be tracked by ports of entry in their local tracking tools. Regions must be able to account for every notebook.

79. The audit assessed whether notebooks are tracked as per the notetaking policy^{Footnote 7}.

80. The notetaking policy includes some general requirements for tracking notebooks. However, the policy does not provide sufficient detail on how the tracking should occur at every stage from issuance to destruction of notebooks.

81. There was wide variation in the manner that POEs were tracking issued and retained notebooks (manual/paper, electronic/excel or a mix of both). This was not done consistently for the 7 POEs that we visited and tracking did not include the complete notebook lifecycle.

82. Most POEs maintained a manual / written log to track notebooks as they were issued to officers (including officer name, date of issuance, signature of supervisor and officer). This log was often stored with the blank notebooks.

83. POEs then maintained a separate log of returned notebooks (most commonly in excel) in various formats and included varying details (such as date returned and coverage period of entries within notebook).

84. It was not common practice to document the expected retention period. Documenting the retention period was not explicitly a requirement per the notetaking policy, but would add value to the tracker and help facilitate proper information management practices. This would also be helpful where the retention period resets to the beginning if the notebook is accessed after being stored.

85. There would be value to the agency in standardizing a template for a notebook tracking tool, which would facilitate security checks and help to easily locate notebooks throughout their lifecycle.

86. Risk and impact:

Risk

Improper tracking reduces visibility into the location of notebooks and whether they have been lost or stolen.

Impact

Impedes the agency's ability to locate notebooks in a timely manner to support administrative or legal proceedings.