Evaluation of eManifest

Final Report

July 2024

Program Evaluation Division, Internal Audit and Program Evaluation Directorate

Table of contents

• Overview of eManifest
• Evaluation scope, approach and methods
• Overall findings and recommendations
• Management Response and Action Plan (MRAP)
• Appendices
  • A. Detailed findings
  • B. Endnotes
  • C. Key and supplementary eManifest Information
  • D. Evaluation methodology and limitations
  • E. Acronyms

Overview of the eManifest initiative

The eManifest initiative was implemented to modernize and streamline the Canada Border Services Agency’s (CBSA or "the Agency") commercial process. It expanded on the groundwork set by the Agency’s Advance Commercial Information (ACI) initiative and completed the transition from a traditional, paper-based commercial process to an automated one.[1]

eManifest was delivered through a project that launched in 2006-07 and closed in 2018-19. It involved making program, policy, and legislative changes, in stages over time, to legally mandate Trade Chain Partners (TCP) to submit commercial information to the CBSA, electronically and prior to their arrival in Canada. The project also involved developing Information Technology (IT) system components and functionalities to enable electronic data transmission, warehousing, and retrieval for risk assessing commercial shipments in all modes of transportation.

Prior to eManifest implementation, TCPs (i.e., carriers) were able to submit electronic pre-arrival information in Marine, Air and Rail modes, but not in Highway mode.[2] eManifest implementation filled the gap in the Highway mode and, in 2021, made electronic pre-arrival commercial data submission mandatory for carriers and Freight Forwarders (FF) in all modes. A timeline of events follows. Key definitions can be found in Appendix C.

In 2018-19, the eManifest initiative transitioned from a project to a functional component of the commercial process and is now under the functional authority of the Commercial and Trade Branch (CTB). In its steady state, eManifest is expected to deliver on the following results:

1. Improved risk assessment (as pre-arrival, electronic commercial data allows the CBSA to identify and target high-risk shipments)
2. Faster, more efficient frontline processing for commercial trade
3. Reduced administrative burden on businesses

These expected results align with the eManifest project’s intended benefits (refer to Appendix D for original intended benefit statements).

Summary: eManifest key events timeline

2004 & 2005
ACI Phases I & II: Electronic pre-arrival data mandated for Air and Marine modes for carriers (not FFs)

2006
First eManifest Treasury Board (TB) submission

2015
eManifest milestone: Electronic pre-arrival data mandated for Rail and Highway modes for carriers (not FFs)

2016
Second eManifest TB submission

2019
eManifest Project closure

December 2021
eManifest milestone: Electronic pre-arrival data mandated for FFs in all modes

Development of eManifest IT components, enabling electronic data exchange in highway mode [3]

2011
eManifest Portal available to end-users

2015
Commercial Threat Assessment System (CTAS) available to end-users

2019
Commercial Passage (CPSG) available to front counter (limited functionality)

2023 and ongoing
Milestone: CBSA now receives electronic pre-arrival data from all required parties across all modes

Note: Given the importance of the December 2021 milestone, the evaluation will often present the impacts of eManifest through changes observed pre and post this date, although changes were implemented over time up to that date.

eManifest capabilities and key players

eManifest supports the following capabilities of the commercial process:

• Pre-arrival data: TCPs submit pre-arrival electronic data to the CBSA using the Electronic Data Interchange (EDI) or through a secure online reporting method called the eManifest Portal.
• Data warehousing: The CBSA collects and stores this information across multiple IT systems, both newly developed and legacy.
• Risk assessment: The National Targeting Centre (NTC) retrieves the pre-arrival data and risk assesses it using a combination of automated and manual analytical tools. Following review, the goods are moved forward in the commercial continuum, either for further processing or to be released into Canada.
• After processing, the commercial shipment is either released or examined, and enforcement actions are carried out if illegitimate goods are found.

Key CBSA stakeholders (eManifest – steady state)

Office of Primary Interest (OPI): CTB

• Transformation, Planning and Projects Directorate (TPPD)
  • Responsible for fund management and the transition of eManifest to steady-state
• Commercial Program Directorate
  • Benefit owner and Functional Authority of steady-state eManifest (policy and program support)

End-users: Intelligence and Enforcement Branch (IEB) and Regions

• Ports of Entry and Intelligence and Enforcement Operations (Regions)
  • Process commercial cargo
  • Conduct cargo tracing
  • Support risk assessment
• NTC, IEB
  • Conducts risk assessment
  • Supports development of automated risk assessment rules

Information, Science and Technology Branch (ISTB):

• Develops and maintains information technology systems
• Provides technical support to system end-users

Evaluation scope and methods

Evaluation Scope:

• Relevance: Continued utility of eManifest in relation to the CBSA’s overall suite of IT-enabled services
• Achievement of results:
  • Data acquisition and administrative non-compliance to enable risk assessment
  • Extent to which expected results have been achieved
  • Impacts of eManifest (positive and negative, intended and unintended) on the CBSA and its stakeholders, including with respect to resource allocation

Out of scope:

• eManifest's funding and expenditures, as CTB conducted a reassessment of eManifest’s ongoing funding requirements concurrent to this evaluation.
• eManifest project management: eManifest as a project was only examined with respect to how elements of project delivery impacted steady-state operations.
• TCPs were not consulted as part of this evaluation. As eManifest was only fully operationalized in 2021, it is too early to assess its impact on TCPs. Ensuring TCP views are gathered in the future will be taken into account when planning future evaluation activities.

Expected results assessed:

1. Improved risk assessment
2. Faster, more efficient frontline processing for commercial trade
3. Reduced administrative burden on businesses (limited assessment)

Lines of evidence used:

Operational and financial data, headquarters (HQ) consultations, regional questionnaire, operational scenarios, document review, and project performance metrics review. (See Appendix D for more information on the evaluation methodology and its limitations.)

Evaluation coverage:

The evaluation covers the five fiscal years from 2018-19 to 2022-23.

Evaluation considerations and limitations

General considerations:

• While eManifest implementation affected all modes, it had the most significant impact on the Highway mode, as risk assessment in Highway mode was not previously conducted.
• eManifest contributions to commercial processing results cannot be isolated from other contributing factors which may impact results (such as increases in commercial volumes).
• The commercial program is a complex process that involves various components and sub-processes. Within this continuum, eManifest is just one functional component among many others.

Limitations and considerations on assessing efficiency:

• The evaluation was not able to assess resource efficiency, as the Agency was not using specific Work Breakdown Structure (WBS) codes to track steady-state eManifest expenditures in isolation from commercial processing activities. [4]
• At the time of the evaluation, the CTB was conducting a parallel, substantive assessment of eManifest funding which is expected to complement the evaluation findings related to resource use.
• Process efficiencies are contingent on fully achieving risk assessment results.

Data considerations and limitations:

• Performance data is not collected/stored in a way that isolates effects of a single component or sub-process of the Commercial Program.
• For the purposes of this evaluation, electronic pre-arrival commercial data across all four modes was considered to be eManifest data, regardless of the IT system used to transmit or store the data.
• eManifest has been implemented for only a few years now – electronic pre-arrival data was mandated for all TCPs (carriers and FFs) across all modes in December 2021. This limited the evaluation in identifying data trends to assess the effectiveness or efficiency of eManifest.
• Since electronic submission of pre-arrival commercial data in Marine and Air modes was mandated for some TCPs under the ACI initiative prior to eManifest, the evaluation focused on assessing results achieved and process efficiencies gained specifically in the Highway mode, as it is in this mode where eManifest results are the most evident.

Overall findings

eManifest implementation is aligned with the CBSA and the Government of Canada’s digitalization and modernization efforts, and the shift to electronic data submission was necessary to undertake. While more time will be required to determine eManifest’s true impacts, the following contributions and areas for improvement were identified:

• eManifest implementation increased the volume of electronic pre-arrival commercial data used for risk assessment across all modes, and put in place data quality controls. This data is leveraged by the NTC to conduct risk assessment and issue targets in the Highway mode, a capability that did not previously exist.
• eManifest contributed additional benefits to the Agency, including: enabling the Agency to recover lost revenue, improving commercial process efficiency and electronic communication with TCPs. However, additional time and data are needed to fully assess the extent of these results.
• Unclear roles and responsibilities for steady-state eManifest, and a fragmented understanding of eManifest among Agency stakeholders, limit its effectiveness.
• Functionality gaps in the systems developed as part of eManifest resulted in continued end-user reliance on aging legacy systems and inefficient workarounds that may not maximize the benefits of using electronic pre-arrival data for risk assessment and targeting, particularly in the Marine and Air modes.

Despite some challenges, eManifest continues to be useful to the Agency as it provides the basis for ongoing development of capabilities and functionalities needed for efficient and effective commercial processing. Two recommendations are presented to fill the existing gaps and improve realization of expected results.

Recommendation 1 and associated findings

There is a fragmented understanding of eManifest and unclear roles and responsibilities for results achievement and performance tracking

Depending on Agency stakeholders’ role and interaction with eManifest, their definition of eManifest varied. This could contribute to a lack of consensus across the Agency on what aspects of eManifest are working well and what areas require improvements.

Additionally, the evaluation found that the roles and responsibilities for eManifest as a functional component of the Commercial Program have not been updated under the Agency's Functional Management Model (FMM), leading to gaps in governance and accountability for results, performance measurement and reporting. The Agency continues to use the key performance indicator (KPI) developed for the eManifest project; steady-state KPIs have not been developed.

A mechanism to monitor and report on eManifest’s ongoing results is required. eManifest KPIs for all modes are needed to determine if risk assessment and targeting efforts using eManifest data are effectively contributing to the Agency’s goals.

Recommendation 1:

In order to ensure consistent understanding of eManifest and its results, the Vice President (VP) of CTB should develop and communicate a performance measurement strategy for eManifest, either by embedding it within the existing Commercial Program Performance Measurement Framework or as a standalone framework. The strategy should include:

• A clear definition of eManifest;
• Clearly defined roles and responsibilities for eManifest as a functional component within commercial processing, including performance measurement and reporting; and,
• KPIs aimed at assessing eManifest's contributions to gained efficiencies and to the effectiveness of risk assessment across all modes.

Recommendation 2 and associated findings

Functionality gaps within the newly developed eManifest systems exist

The implementation of eManifest addressed risk assessment gaps in the Highway and Rail modes by adding system functionality where it did not previously exist. The CBSA implemented a complex IT architecture, adding functionalities to existing legacy systems and developing new systems.

While newly developed systems were originally intended to allow for risk assessment and targeting in all modes, the end-user risk assessment interface functionality in Air and Marine modes was deferred and is yet to be delivered. As a result, the NTC continues to rely on legacy systems for risk assessment and targeting in Marine and Air modes, using inefficient workaround solutions. Another system limitation due to project scope change is the inability to assess the effectiveness of risk assessment and targeting efforts, therefore not being able to ‘close the loop’ on commercial examination results.[5]

Similar functionality gaps were previously identified within the Agency but were not fully addressed, and there continue to be challenges for end-users of eManifest systems. CTB and ISTB noted that there are future plans to develop additional components and functionalities within newly developed systems; however, there is currently no timeline for this work to be completed.

Recommendation 2:

The VP of CTB, in collaboration with the VPs of IEB and ISTB, should develop and communicate an actionable plan with clear timelines to add end-user functionality to the eManifest systems. Consideration should be given to functionality that:

• enables risk assessment and targeting in the Air and Marine modes; and
• allows the Agency to track the results of risk assessment and targeting on commercial examination results in all modes.

The plan should consider Agency-wide information system needs, priorities, and alternatives.

Management Response and Action Plan (MRAP)

Recommendation 1: Develop and communicate a performance measurement strategy for eManifest, including a clear definition of eManifest and clearly defined roles and responsibilities.

Management response: The VP of CTB agrees with the recommendation to develop and communicate a performance measurement strategy for eManifest. The related CTB consultations to be led by the eManifest Task Force and scheduled to be completed within 9 months of Canada Assessment and Revenue Management (CARM) Release 3 (R3) with relevant stakeholders, including the regions, ISTB and IEB will lead to: the development and adoption of a clear and detailed definition of eManifest, which will include related roles and responsibilities, as well as the development of an eManifest performance measurement strategy with KPIs. The performance measurement strategy will be developed to align to the development of eManifest performance data and reporting capability, to ensure results against KPIs can be tracked and measured.

This exercise will result in a clear governance structure for eManifest providing oversight and guidance on all aspects of the initiative, with consideration given to integration with CARM governance given schedule and systems interdependencies. These elements will be components of the actionable plan mentioned in recommendation 2.

*CARM R3 is currently anticipated for October 2024, so action item completion is anticipated for July 31, 2025.

Recommendation 2: Develop and communicate an actionable plan with clear timelines to add end-user functionality to the eManifest systems.

Management response:

• The VP of CTB agrees with the recommendation to develop and communicate an actionable plan to add end-user functionality to the eManifest systems, with the note that given the interdependencies between eManifest and CARM, development of this actionable plan will begin following CARM R3 in order to ensure that anticipated eManifest functionality releases are aligned to any identified contingencies.
• Following CARM R3, the CTB will develop an eManifest functionality deployment plan, in consultation with stakeholders, including the regions, IEB, ISTB, SPB, and the Human Resources Branch (HRB) and seek consensus on the way forward. The deployment plan will provide clear guidance and actionable items in the development and implementation of end-user functionality for risk assessment and targeting in all modes including Air and Marine. The plan will also explore options for reporting tools to track the results of risk assessment and targeting on commercial examination in all modes.
• While the deployment plan cannot commit to clear timelines (specific dates) for functionality releases that are multiple years in the future due to the interdependencies between eManifest and CARM, the plan will indicate specific dates for the eManifest releases in the near term as well as timelines conditional on CARM production and release dates for later planned eManifest releases. For this reason, the deployment plan will be reviewed following each CARM release and the specific dates of subsequent eManifest releases will be identified at these points in time.

*CARM R3 is currently anticipated for October 2024, so action item completion is anticipated for July 31, 2025.

Appendices

Appendix A: Detailed findings

Findings Overview

Relevance

1. Fragmented understanding
   • Finding 1: There is a fragmented understanding of eManifest among Agency stakeholders which could be hindering collaboration and effectiveness.
2. Roles and responsibilities
   • Finding 2: Roles and responsibilities for eManifest and for its results as a functional component of the Commercial Program are not clear.

Achievement of Results

3. Improved risk assessment
   • Data availability
     • Finding 3: eManifest increased the availability of electronic pre-arrival commercial data for risk assessment, and put in place data quality controls.
   • IT functionality
     • Finding 4: The eManifest initiative added IT functionalities to enable risk assessment using pre-arrival data in the Highway and Rail modes.
   • IT functionality limitations
     • Finding 5: IT systems developed under the eManifest initiative lack end-user functionalities for risk assessment in Marine and Air modes.
4. Faster, more efficient commercial processing
   • Finding 6: eManifest contributes to a faster, more efficient frontline commercial process in the Highway mode by reducing referrals to the front counter, and contributing to release recommendations.
5. Reduced burden on TCPs
   • Finding 7: eManifest resulted in some benefits for TCPs, however additional time is needed for the reduced burden benefit to be fully realized.
6. Unintended benefits
   • Finding 8: eManifest enables the Agency to recover lost revenue by formalizing the process of cargo tracing to ensure payment of overdue duties and taxes.

Detailed Findings

Relevance

• Fragmented understanding
• Roles and responsibilities

FINDING 1 (fragmented understanding): There is a fragmented understanding of eManifest among Agency stakeholders which could be hindering collaboration and effectiveness.

There are a number of CBSA stakeholders with various roles related to eManifest:

• ISTB: developed and continues to develop and enhance the IT components of eManifest; provides system support to internal and external clients.
• CTB, Commercial Program Directorate (CPD): manages the policy and program-related components of eManifest as the benefit owner and the functional authority.
• CTB, TPPD: manages funding for continued implementation and is responsible for the continued transition of eManifest from project to steady-state.
• IEB, NTC: use eManifest data for risk assessment and targeting functions.
• Regions: use eManifest data and the NTC’s data analysis for processing commercial goods entering Canada.

Through multiple stakeholder consultations and document review, the evaluation found discrepancies in stakeholders’ understanding of what constitutes ‘eManifest’, as well as differing perceptions of its relevance, effectiveness, and efficiency. Communication gaps among eManifest stakeholders were also noted. These factors could be hindering the progress of eManifest as an operational and functional component of the commercial process, as stakeholders cannot agree on what is working well and what requires improvement.

Depending on the stakeholder’s role and interaction with eManifest, they can see it as:

• An ongoing project/initiative
• A project separate from ACI
• A standalone IT system or application; some equate it to the eManifest Portal, while others to the CPSP or CTAS systems
• A tool for commercial processing
• A commercial pre-arrival dataset
• Data used for risk assessment in Highway and Rail modes only

Various factors contributed to the fragmented understanding of eManifest in its steady state, including:

• Continued development on eManifest systems, post-project closure
• End-user system workarounds, implemented due to system functionality gaps
• Lack of eManifest-specific financial codes to track expenditures
• Benefits Management practices not fully applied during project delivery (as per FCMB stakeholders’ perspectives)
• Lack of steady-state performance measurement as a functional component of the Commercial Program
• Uneven regional training or resourcing to maximize eManifest use

FINDING 2 (Roles and responsibilities): Roles and responsibilities for eManifest and for its results as a functional component of the Commercial Program are not clear.

Unclear roles and responsibilities for eManifest may date to its transition from project to steady-state:

When the eManifest project closed in 2018-19, the Agency was initiating an organizational renewal, which would lead to the adoption of the FMM in 2019. Responsibility for eManifest was transitioned shortly before major changes to the CBSA’s organizational structure, meaning that eManifest responsibilities were delegated to CBSA branches that no longer exist today.

The evaluation found no evidence that an updated governance structure for an operational, steady-state eManifest was developed following the adoption of the FMM and the creation of CTB. As such, no single stakeholder within the CBSA is currently ensuring eManifest implementation is maximized to improve the effectiveness and efficiency of the Agency’s commercial process, including tracking and reporting on performance results.

During the eManifest’s project phase, roles and responsibilities were not always clear:

• There were initially three program sponsors for eManifest: the VPs of CBSA’s Enforcement, Operations, and Admissibility Branches. This shared responsibility diffused accountability.
• The project was re-structured in 2014, with the VP of Programs Branch identified as sponsor. This led to more clarity for decision-making, but led to stakeholders in Operations Branch feeling they were not adequately consulted on key decisions (e.g., to de-scope the project and defer key risk assessment functionality).
• Many stakeholders within the CBSA were unsure who to contact for information about the project.

Source: eManifest Post Implementation Review – Lessons Learned (Interis|BDO for the CBSA, May 2018)

Performance metrics do not capture the expected results of steady-state eManifest:

Following project closure, eManifest performance has not been monitored as a functional component of the Commercial Program. Steady-state KPIs have not been developed.

Rather, eManifest performance continues to be assessed against the outcomes and KPIs initially established for the eManifest Project (refer to Appendix C). For instance, existing metrics focus on measuring results achieved in Highway and Rail modes only, while pre-arrival commercial data is used for risk assessment across all modes.

The Agency could leverage certain indicators from its Departmental Results Framework to assess eManifest’s performance, including:

• Percentage of high-risk commercial goods targeted by the NTC that are examined at the border
• Percentage of random commercial examinations that produced a result
• Percentage of commercial examinations that result in non-compliance findings against a trader
• Percentage of penalties applied against traders for continued non-compliance

Achievement of expected result 1: Improved risk assessment

• Data availability
• IT functionality
• IT functionality limitations
• Spotlight: Highway mode results

FINDING 3 (Data availability): eManifest increased the availability of electronic pre-arrival commercial data for risk assessment, and put in place data quality controls.

Increased data availability:

One of the key achievements of eManifest was the introduction of legislation mandating carriers to submit electronic pre-arrival commercial information in the Highway and Rail modes, and FFs to submit the information in all modes. As a result, the volume of pre-arrival data submitted electronically increased since eManifest's implementation. A breakdown by mode shows that by 2021-22, almost 100% of pre-arrival data was being submitted electronically in all modes:

*eManifest project closure

**Electronic pre-arrival data mandated for carriers and FFs in all modes

Data on the number of pre-arrival submissions from 2018-19 to 2022-23 show an increase of electronic submissions and a decrease in paper submissions each year, resulting in 99.8% of pre-arrival data being submitted electronically through the eManifest service options by 2022-23:

Data quality controls enabled by the electronic environment:

Through the digitalization of the data submission process and the standardization of requirements for TCPs, eManifest introduced quality control features like automatic field validations, data cleansing (the process of correcting errors and inconsistencies in a dataset to improve data quality), and other IT functions (i.e., automated identity and address resolution).

Additionally, eManifest introduced a legislated pre-arrival electronic reporting requirement, which enabled the CBSA to issue Administrative Monetary Penalties (AMPs) in cases when there are data quality concerns, including when pre-arrival information is submitted that is untrue, inaccurate or incomplete (AMP codes: C382, C387, C381).

In Highway mode, TCPs are only required to submit electronic pre-arrival data one hour prior to the conveyance’s arrival in Canada. The NTC highlighted this as a challenge, as it is impossible for them to analyze all of the data before the conveyance’s arrival in Canada. So, while eManifest increased pre-arrival commercial data availability, it did not in all cases increase the data utility for risk assessment.

FINDING 4 (IT functionality): The eManifest initiative added IT functionalities to enable risk assessment using pre-arrival data in the Highway and Rail modes.

According to the 2016 eManifest TB Submission, the initiative was intended to develop multiple program and information technology components to ensure 100% of commercial goods in all modes could be risk-assessed in advance of their arrival to Canada. In order to accomplish this, the CBSA implemented a complex IT architecture of more than 19 back-end systems, end-user systems, and other IT products over 20 releases since 2008. Functionalities were added to existing (legacy) CBSA systems, and new systems were developed.

Legacy systems leveraged by the eManifest initiative:

• Customs Electronic Commerce Platform (CECP), includes:
  • Query and Data Viewing Utility (QDV)
  • Rogue Data Corporation Central (RDC Central)
  • Trading Partner Application (TPA)
• Accelerated Commercial Release Operations Support System (ACROSS) for data acquisition
• Tactical Information Targeting Analysis and Notification System (TITAN) Air
• TITAN Marine

New systems developed as part of the eManifest initiative:

• eManifest Portal
• Portal Account Administrator
• Master Data Management
• External Client Communications (ExCC), includes the External Client Notice Profile (ECNP)
• Commercial Document Entry Management (CDEM)
• Automated Risk Determination (ARD)
• CTAS
• Risk Assessment Program Maintenance (RAPM)
• CPSG

CTAS and CPSG are two of the new systems developed as part of eManifest, and both are now used by the NTC for pre-arrival risk assessment and targeting in Highway and Rail modes, to a limited extent. Prior to their development, risk assessment and targeting using pre-arrival electronic data in Highway and Rail modes was not being conducted by the NTC.

FINDING 5 (IT functionality limitations): IT systems developed under the eManifest initiative lack end-user functionalities for risk assessment in Marine and Air modes.

The eManifest project was originally intended to enable pre-arrival commercial risk assessment and targeting in all modes. However, end-user interface functionality in the Air and Marine modes was deferred for completion and is yet to be delivered. Due to this deferral, the NTC conducts its pre-arrival commercial risk assessment in Air and Marine modes using TITAN, a legacy system. Though CTB and ISTB stakeholders noted plans to develop these functionalities in CTAS and CPSG (eManifest systems), a specific timeline for their completion has not been established. While TITAN will continue to be supported until new functionalities are fully implemented, NTC stakeholders raised concerns regarding TITAN’s reliability, which are further discussed in the next page.[6]

The table below presents the functionalities across the pre-arrival data risk assessment continuum, summarizing all current capabilities made available through the newly developed eManifest IT components. As demonstrated, eManifest succeeded in acquiring, storing, and applying automated risk assessment to pre-arrival data in all modes. However, eManifest supports the NTC’s risk assessment capacity in the Highway and Rail modes only. This capacity is currently being piloted in three Ports of Entry (POE).[7] Moreover, functionalities related to communicating referrals to the front-line and closing the loop on exam results are exclusively available for the Highway mode. The NTC also uses the internally developed Customers Referral Inspection Manager (CRIM) system, which relies on eManifest data stored within the Electronic Data Warehouse. Additional details on functionality are provided in Appendix C.

Inefficient risk assessment resulting from limited functionalities

In the absence of CTAS and CPSG end-user functionalities, TITAN continues to be used by the NTC to risk assess for Marine and Air modes. As it is a legacy system, TITAN was not designed to efficiently process eManifest data in the format received. Therefore, the NTC has implemented workaround solutions to conduct risk assessment and targeting activities, but noted that these workarounds are neither sustainable nor efficient. Examples of inefficiencies stakeholders provided include:

• Additional time and system querying is required to access information warehoused in the newly developed eManifest tools.
• Analytics workaround solutions have had to be developed to address reporting and automated risk assessment gaps.
• A new Commercial Risk Development (CRD) team was established by the NTC to modernize commercial targeting and address gaps left by currently undelivered eManifest components. This team is focused on the creation, testing and implementation of risk indicators, the operationalization of automated products to bring efficiencies to risk assessments, and the collection of business requirements for an effective commercial targeting system. The team is comprised of 18 individuals from six NTC units, who conduct this unfunded work in addition to their regular roles.
• Additional resources allocated for providing ongoing business requirements for CTAS functionality related to highway, air, and upcoming marine operations; such as the development of automated risk assessment rules.

Additionally, the NTC indicated that risk assessment in the Highway mode using CTAS and CPSG functionalities are being piloted in three POEs, with some emerging challenges.[8]

Inability to assess the effectiveness of risk assessment

Reporting functionality is limited across all systems, preventing the CBSA from ‘closing the loop’ on Air, Marine and Rail exam results (i.e., to indicate whether NTC referrals for examination resulted in seizures or not). This limits the Agency’s ability to determine the effectiveness of current risk assessment and targeting efforts within the Commercial Program as a whole, including eManifest as a component of the process.

De-scoping and deferring elements without sufficient consultation

A post-implementation review of the eManifest project highlighted a lack of adequate stakeholder engagement in decisions related to the de-scoping of key capabilities from newly developed systems:

“The project was re-structured in 2014, with the Programs Branch VP identified as sponsor. This led to more clarity for decision making, however, key stakeholders in Operations Branch feel they were not adequately consulted on key decisions, e.g., to de-scope the project and defer key risk assessment functionality.”

Source: eManifest Post Implementation Review – Lessons Learned (Interis|BDO for the CBSA, May 2018)

Due to deferral decisions, when the eManifest project closed in 2018-19, it delivered new systems that did not include functionality for risk assessment in Marine and Air modes.

Project funding and key decisions about scope up to project closure:

2006-07: Preliminary Project Approval received from TB for $414.8M

2007-08: Effective Project Approval (EPA) received from TB for $415.1M

2015-16: Re-baselining, de-scoping of two initial capabilities and extension of project completion to September 2017

2017-18: Frozen allotment of $72.1M (ongoing funds) released by TB until the end of the 2020-21

2018-19: eManifest project closure and release of funding allotment by TB for an additional 3-year period (until end of 2023-24)

Similar functionality gaps were previously identified within the Agency. For instance, in 2022 a list of needed eManifest deliverables was produced in response to identified gaps; however, end-user functionality gaps have yet to be fully addressed.

Stakeholder perceptions collected for this evaluation and the CTB’s substantive funding assessment highlighted the need to continue to develop functionality for risk assessment in the Air and Marine modes.

Spotlight: Positive initial risk assessment results in Highway mode

As previously mentioned, the implementation of eManifest addressed a significant gap in data acquisition and risk assessment capabilities for the Highway and Rail modes. While the development of CTAS, the system designed for NTC data risk assessment, is still underway across all modes, it is most advanced and functional for Highway mode capabilities.

The evaluation analyzed NTC data for the Highway mode over the past four years to identify any trends. However, the following factors limited the evaluation's ability to draw conclusions:

• Since the NTC did not conduct risk assessment or targeting using pre-arrival electronic data in the Highway mode before eManifest, assessing the improvement of enforcement results against the previous scenario (risk assessment and enforcement done at the regional level) is not possible, due to variations in the process and data.
• The submission of pre-arrival electronic data became mandatory for FFs in December 2021, providing less than two years of data for the evaluation to observe trends.
• The pandemic years have had an impact on the Agency's commercial operations, resource allocation, and resulting trends.

The NTC shared some business requirements that are not currently being met by CTAS in the Highway mode, for which inefficient workarounds are applied, including:

• Inability to adequately conduct searches using the available parameters
• Distribution (workload management) tool does not meet business needs and requirements
• Limited reporting features to allow for performance assessment

For more information see Appendix C.

Rail Targeting is a new activity under the NTC that is currently being piloted on a small scale before implementation, and there is not enough data within the evaluation’s scope to analyze the impact of eManifest implementation in this mode.

The data from 2019-20 to 2022-23 shows that the resultant rate has improved over time in the Highway mode. However, given the limitations to data analysis mentioned on the previous page, it is too early to conclude this increase is due to the increased availability of electronic pre-arrival data.

The NTC's ability to conduct risk assessment and targeting in the Highway mode since the implementation of eManifest is a benefit for the Agency. However, due to previously highlighted factors, further time is required to fully evaluate the scale of this success.

Moreover, it will be important to compare NTC results from different modes to understand how the availability of IT functionality in newly-developed systems affects the Agency's enforcement capabilities.

The NTC’s resultant rate in the Highway mode, compared to volumes of electronic pre-arrival data over time.[9]

Figure 1: NTC’s Resultant Rate, Compared to Volumes of Electronic Pre-arrival Data Over Time

Image description

*Resultant rate is calculated using data from the NTC’s Highway Targeting Reports, and is calculated as ‘Direct results’ over ‘Targets issued’.

Figure 1: NTC’s Resultant Rate, Compared to Volumes of Electronic Pre-arrival Data Over Time

Fiscal Year	Highway electronic pre-arrival data	Resultant rate
2019-20	Under 14 million	1.34%
2020-21	Over 14 million	3.06%
2021-22	Approximately 16 million	3.10%
2022-23	Over 16 million	3.65%

Achievement of expected result 2: Faster, more efficient commercial processing

FINDING 6: eManifest contributes to a faster, more efficient frontline commercial process in the Highway mode by reducing referrals to the front counter, and contributing to release recommendations.

Fewer referrals to front counter in Highway mode

Prior to the eManifest initiative: Commercial cargo arriving at the border underwent processing at the Primary Inspection Line (PIL) with a portion of shipments redirected to the front counter to complete their primary processing. This was typically due to certain POEs being limited to scan only five Pre-Arrival Review System (PARS) codes at the PIL, resulting in referrals made to the front counter once that limit was reached.

Following the implementation of eManifest: Front-line staff process the majority of commercial shipments directly at the PIL, with an average 9% of shipments referred to the front counter. The table below shows a decrease over time of Highway conveyances referred from the PIL to the front counter. This results in gained process efficiency for the Agency as fewer trucks need to wait in line for processing at the front counter.

* Counts for service options SO976

Regional respondents generally felt eManifest implementation contributed to process efficiencies. Many respondents highlighted its role in streamlining cargo clearance, reducing border wait times, and enhancing customs procedure management via better tracking.

However, a few respondents also highlighted some challenges to fully realizing the intended efficiencies. For example, regional respondents indicated eManifest resulted in increased TCP data entry errors, requiring more time from the front-line to manually fix them. They also felt that continued reliance on the legacy ACROSS system is inefficient.

Contributing to release recommendations

Between 2018-19 and 2022-23, when an importer submitted a release request electronically at least one hour before arrival at the POE, a release recommendation was available upon arrival in 99.4% of cases. This is another way in which eManifest has streamlined commercial processing in the Highway mode. eManifest provides electronic pre-arrival data that can be analyzed and used to make release recommendations, before a shipment arrives in Canada.[10]

The evaluation looked at four scenarios to assess the potential impact of eManifest on the time and number of steps needed to process commercial goods in the highway mode. The scenarios presented in the table below estimate the total time and number of steps required to process commercial goods with eManifest and prior to eManifest implementation (the counterfactual). The estimates in each scenario are based on the real experiences of Southern Ontario Region’s (SOR) frontline staff who worked on commercial processing before and after eManifest implementation.[11]

As illustrated in the table below, in the three scenarios in which a recommended release is on file when the conveyances arrive at the border, eManifest facilitates the release of goods into Canada by reducing the number of steps and time needed for processing. In the fourth scenario, in which a recommended release was not on file, eManifest had no impact on the number of steps or time needed for processing.

*Times were provided by SOR’s frontline staff based on operational experience and vetted by the Transporter and Cargo Control Programs Unit, CTB.

Achievement of expected result 3: Reduced administrative burden on TCPs

FINDING 7: eManifest resulted in some benefits for TCPs, however additional time is needed for the reduced burden benefit to be fully realized.

Data harmonization with international requirements

In 2019, the eManifest initiative successfully harmonized the CBSA’s data requirements with World Customs Organization (WCO) and the United States (US) Customs and Border Protection requirements. Data harmonization reduces the administrative burden for TCPs by allowing them to utilize electronic processes and systems already implemented to comply with US requirements.

The benefit of reduced administrative burden on businesses was driven by the value proposition to “Harmonize data reporting requirements and minimize costs for external clients who have invested in systems that comply with the US ACE project.” By eManifest project closure (Benefit Realization Report, Gate 7, 2019) this value was deemed fully realized.

Counterfactual: Had data harmonization not happened, TCPs would likely need to provide one set of data for commercial shipments to the US and a second set of data for shipments to Canada.

Increased communication with TCPs

Communication has increased due to eManifest functionality which enables the Agency to proactively provide electronic updates and feedback to TCPs at various stages of the commercial process, in all modes.

By increasing proactive communication with TCPs, eManifest implementation reduces the number of times TCPs must reach out to the CBSA to request status updates, thus further reducing the time required for TCPs to track their shipments through the CBSA’s commercial process.

Analysis on the number of electronic notices issued to TCPs over time shows that they are increasingly receiving proactive notices from the CBSA: [12]

Figure 2: Number of EDI Applications and Notices Over Time

Image description

Figure 1: NTC’s Resultant Rate, Compared to Volumes of Electronic Pre-arrival Data Over Time

Fiscal Year	Number of EDI applications	Number of notices
2019-20	1,802	200,146,556
2020-21	2,073	210,351,987
2021-22	2,598	248,381,199
2022-23	2,759	257,107,787

Impacts on TCPs during the transition period:

The shift from a paper-based commercial process to an electronic one, and the introduction of new mandatory pre-arrival commercial reporting requirements, created an initial administrative demand on TCPs adapting to the new process. In 2021, when new requirements came into effect, the number of calls by TCPs to the CBSA’s centralized 24 hour helpline increased by 200% (see table).[13]

During this transition period, the CBSA implemented temporary measures to lessen the burden on TCPs, including increasing outreach and education efforts. Post-2021, the number of calls to the CBSA helpline decreased, suggesting that TCPs required less assistance after the initial transition.

However, it is important to note that there is not enough helpline data available at this time to show a continued trend, and data does not include calls by TCPs directly to the Regions.

*Estimated based on 4,735 calls received from April to December 2020

**Projected based on 5,710 calls received from January to July 2023

Limitation: The evaluation did not survey TCPs to collect their perspectives regarding the impact of eManifest implementation on their processes.

Achievement of expected result 4: Unintended benefits

FINDING 8: eManifest enables the Agency to recover lost revenue by formalizing the process of cargo tracing to ensure payment of overdue duties and taxes.

“Cargo tracing” refers to the process of tracking and verifying the movement of shipments to ensure that all goods entering the country are accounted for. Cargo tracing allows for the identification of duties and taxes owed on goods being released.

“Lost revenue” refers to unpaid duties and taxes on goods that have not been traced.

An unintended benefit of eManifest implementation is a reduction in lost revenue on commercial shipments due to incomplete tracing. Prior to eManifest implementation, cargo was verified at its port of entry through reviewing samples of the paper cargo control documents received. Samples represented: 10% of large airport shipments, 50% of small airport shipments, and 20% for other cargo.

The introduction of eManifest transformed the process, as the availability of electronic pre-arrival commercial data allowed for:

• The elimination of the sampling process. While there are some exemptions, most cargo is fully traced.
• An overdue cargo tracing procedure that more effectively verifies whether duties and taxes were paid on cargo entering Canada.

From August 2020 to December 2023, the CBSA traced an average of 2,122 monthly outstanding overdue releases to ensure payments were made. Stakeholders stated that eManifest played a critical role in recovering lost revenue from these accounts.

Note: The evaluation did not have the data to quantify the value of the revenue recovered during this time period.

CONCLUSION – eManifest continues to be relevant and useful to the Agency

What did the evaluation find?

The findings of this evaluation highlight the importance of the eManifest initiative to the CBSA’s commercial process. Through the eManifest initiative, the Agency conducted legislative and process changes required to digitalize the commercial process. The initiative also resulted in the development of a complex IT architecture, including upgrades to back-end systems, development of new end-user systems, and production of other IT products which have enabled risk assessment using electronic pre-arrival commercial data. These changes are showing some positive initial results.

eManifest implementation filled a risk assessment functionality gap in the Highway mode; however, additional efforts are necessary to continue developing functionality for Air and Marine modes. Similar gaps were identified previously but have not been fully addressed. While the intention to implement these functionalities as part of eManifest stabilisation was raised during the evaluation, there is currently no set date for these upgrades. Without a commitment to timelines in which to address these gaps, there is a risk the upgrades continue to be delayed.

Continuing to rely on aging, legacy systems like ACROSS and TITAN, is no longer a sustainable option for the Agency due to their inefficiency when processing high volumes of electronic commercial information. At present, there are no other active initiatives aimed at addressing these gaps. Therefore, eManifest stands as the path forward to enhance the Agency's risk assessment capacity in all modes and to advance Canadian security priorities.

What needs to change?

The findings support the need for continued strategic investment in eManifest to address existing IT functionality gaps.

Performance measurement and clear accountabilities are also needed to ensure positive results in all modes.

Appendix B: Endnotes

1. The eManifest initiative continued from the ACI and is sometimes referred to by some Agency stakeholders as ACI Phase III. The eManifest initiative leveraged the work completed by the ACI initiative and formalized the reporting requirement in all modes.
2. Prior to eManifest, Highway processing was in paper format, but rail was done through an old standard of EDI. eManifest harmonized and standardized the data elements among modes.
3. Internal government document.
4. Costing and Analytical Model (CAM) tool used by FCMB to code expenditures to programs. In 2022-23, FCMB added dedicated eManifest CAM functions to determine if retaining these functions would not result in significant discrepancies between expected and actual spending. Since this data is limited and experimental, and the evaluation opted to not use it for analysis. Due to the limited and experimental nature of this data, the evaluation chose not to use it for analysis.
5. 'Closing the loop' means being able to indicate the exam results - resultant or non-resultant.
6. While TITAN will continue to be supported, NTC stakeholders shared their continued concern with IT functionality to support risk assessment, including:
   • System capacity to view House Bills in TITAN beyond 2012.
   • No updates to the system risk indicators in TITAN (the NTC ceased updates to risk indicators while working on the eManifest project).
   • User review and decision functionality, which is only partially available in new systems for Rail.
   • User review and decisions for highway, which is available but only currently being piloted to 3 POEs.
   • Limited reporting capabilities.
7. Even though user review and decision functionality is partially available in new eManifest systems for the Rail mode, this has not yet been operationalized. A small scale Rail pilot began on February 5, 2024. Users are able to view ACI and make referral decisions through CTAS, but communicating the decisions for referral or negation to CPSG and closing the loop cannot be made.
8. For example, according to the NTC, CTAS in the Highway mode lacks the ability for coding specific risk indicators and the ability to complete comprehensive reporting.
9. Commercial volumes in this analysis refer to the total electronic eManifest service options data.
10. Not all releases are submitted prior to arrival. Stakeholders (CTB and NTC) indicated that approximately between 20% to 50% of release requests are submitted before arrival depending on the mode. Release requests are submitted by importers and are not mandated pre-arrival by law like reporting requirements (submitted by carriers and FFs).
11. Scenarios show situations in the Highway in SOR only. Therefore, they do not represent eManifest-related commercial processing in all scenarios, locations, modes, or regions. For example, if there is a keying error on any submission, it will hold up the release of the cargo. As a result, the time for some scenarios could vary from several minutes to hours, to days. This depends on knowledge of the carrier and type of access on third party portal. Often times, carriers submit new ACI or cargo document causing the originals to end up on a Primary worklist or overdue cargo list, causing additional resources to trace it. This can involve multiple Border Services Officers (BSO) and did not occur pre-eManifest.
12. Electronic notices sent to TCPs now include: Arrive Notices; Authorized To Deliver Notices; CBSA-Hold Notice; Completeness Notices; Deconsolidation Notices; Document Not on File Notices; Release Notices; and Reported Notices.
13. This data may include helpline tickets for other applications like the Single Window Initiative (SWI) and are not exclusively about eManifest-related issues.

Appendix C: Supplementary eManifest Information

Key definitions and information:

• Advance Commercial Information Program (ACI): A set of prescribed electronically transmitted pre-arrival cargo and conveyance data elements sent to the CBSA within prescribed timeframes, for the purpose of facilitating the process of commercial goods and risk assessing threats to health, safety and security prior to the arrival of the shipment in Canada.
• Administrative Monetary Penalties System (AMPs): A system whereby the CBSA issues monetary penalties to commercial clients for violating the CBSA’s trade and border legislation. The purpose of AMPs is to provide the Agency with a means to deter non-compliance by its clients and to ensure a consistent application of legislation and border regulation.
• Cargo: A term used to describe a collection of goods or shipment. It consists of a grouping of related goods. The cargo is detailed on the bill of lading, waybill, the manifest and/or a cargo control document.
• Electronic Data Interchange (EDI): A method to electronically transmit import or export data and accounting documents to the CBSA.
• eManifest Portal: A secure data transmission option developed by the CBSA that allows the trade community to electronically transmit their pre-arrival data through the Internet.
• First Port of Arrival (FPOA): The port of entry in Canada where a commercial conveyance first arrives from a foreign country.
• House Bill: A cargo control document for shipments that have, or will be, deconsolidated from another cargo control document.
• Shipment:
  • A shipment for which a carrier is responsible is one that consists of:
    • a specified good or collection of specified goods that is listed in a single bill of lading, waybill or other similar document that is issued by the carrier and that relates to the carriage of those goods or
    • a specified good that is an empty cargo container that is not for sale that is transported by the carrier but that is not listed in a bill of lading, waybill or other similar document
  • A shipment for which an FF is responsible is one that consists of a specified good or collection of specified goods that is listed in a single bill of lading, waybill or other similar document that is issued by the FF and that relates to the carriage of those goods
• Trade Chain Partner (TCP): An enterprise that is directly involved in the importation or cross-border movement of goods imported or transported by an importer. In the context of eManifest, two types of TCPs are implicated:
  • Carrier: A person who, in accordance with the Transportation of Goods Regulation, is authorized to transport goods or to cause goods to be transported.

Carriers were mandated to provide ACI to the CBSA as follows:

Freight Forwarder: A person who, on behalf of one or more owners, importers, shippers or consignees of goods, causes specified goods to be transported by one or more carriers. Freight Forwarders submit house bills.

FFs were mandated to provide ACI to the CBSA as follows:

The NTC’s Highway Targeting Unit’s perspective shared with the evaluation on current CTAS limitations in the Highway mode

Inability to adequately conduct searches using the available parameters: Search criteria is limited and often times out, preventing Targeting Officers (TO) from being able to identify shipments of risk based on search preferences and combination of data elements. This prevents TOs from being able to perform adequate analysis of incoming shipments. TOs have a minimum of 1 hour to conduct assessment of commercial highway cargo prior to its arrival, and any delays in searching data impede our abilities to perform comprehensive reviews pre-arrival.

“To mitigate this, the NTC relies heavily on automated products produced by our Targeting Data Analytics (TDA) team to provide us with a “Highway Summary,” an Excel-based document populated every 15 minutes that provides eManifest ACI and release information relating to every un-arrived commercial highway shipment submitted to the CBSA with an Estimated Date and Time of Arrival (EDTA) of -24 / + 5 hours to current time. This Excel document can be filtered and sorted based on specific data points, risk indicators and TO preference to identify shipments of risk for assessment and referral.”

“Reporting features: Are very basic and lack the necessary information to complete reporting suites for unit performance and statistics. Supervisor functions are also limited to run reports for TO performance including performance management agreement reviews. We have been told on multiple occasions that NTC had been consulted in the past regarding reporting requirements and CTAS supervisor functions were built to accommodate those previous requests and were “built as intended.” While this is true, our needs for reporting have changed significantly from the time that we were initially consulted and our requests for enhanced reporting features continue to be met with resistance. Requirements for reporting suites to accurately reflect unit performance and efficacy should include the following (based on current manual reporting procedures):

• Total number of targets issued
• Total number of targets intercepted and not intercepted, as well as percentages
• Total number of targets examined and not examined, as well as percentages
• Number of direct and indirect results, as well as percentages
• Percentage of Total Results”

Inability to refer on Integrated Import Declaration (IID): CTAS referral functionality will only allow referrals on the Cargo Control Number (CCN) or Conveyance Reference Number (CRN) entities.

• Current ACROSS functions allow for the referral on any submitted document (CCN, CRN, IID or eHB), however referral on IID is not permitted in CTAS. Consolidated or Less than Load (LTL) loads require referrals on the IID when available, as requested by the POEs, as referring on the CCN adds unnecessary steps for BSOs and delays to release at the FPOA.

Distribution (workload management) tool does not meet business needs/requirements:

• The NTC requires a distribution tool within CTAS to perform like the current distribution tool CRIM – shipments are queued by priority (i.e., agency mandate of national security and public safety, contraband concern and EDTA) for assessment based on automated risk rules and scenarios deemed to be of high risk and presented to the TO for review, however should the conveyance arrive prior to decisions made, the conveyance will be allowed to proceed past FPOA and will be removed from the distribution tool once logged ‘arrived’ by the POE.
• Inability to code relationship based risk indicators for identification of high risk shipments. ‘Scorecard’ functionality in CTAS is currently very limited and will only provide matches to basic indicators (e.g., ICES Lookouts, Targeting Commercial Intelligence (TCI), RAPM Lookouts, previous ICES enforcement, shipment to residence), but will not allow matching to more complex indicators based on relationship values (i.e., irregular port volume where a carrier is deviated from standard practices by crossing at a port where fewer than 5% of their crossings have occurred). This inability to code these indicators into CTAS force reliance on other tools such as the TDA Highway Summary to provide TOs with data to filter for risk assessment purposes.
• To work around this, the NTC relies on the TDA Highway Summary to present risk associated to commercial highway shipments. The excel document displays risk indicator “hits” to inform TOs of the instance of known risk associated to a shipment. Risk indicators have been determined based on analysis of previous enforcement actions and consultation with TCI and regional intelligence and operation staff. Recently, a CRD team was created to further develop risk indicators and implement a scoring system within the Highway Summary which would improve efficiencies in identifying potential high risk shipments using a colour coding system.

eManifest / IT support: Although we were advised that support is 24/7, support for technical issues or outages is not always immediate. Additionally, changes/enhancements may be requested but development and implementation require coding, testing and production releases which take considerable time. If an issue is identified, targeting operations requires timely resolution for efficiency within the unit. As commercial risk evolves, high risk indicators change and we require a way to adapt and refine coded risk indicators to identify high risk shipments in real time. From our understanding, CTAS will not be able to do this.

Benefit Realization Plan KPIs

The Benefit Realization Plan (BRP) lists the following KPIs for the eManifest initiative:

KPIs for which the Agency continues to report on:

• Highway conveyances referred from PIL to front counter as a proportion of all conveyances that arrive at PIL
• Ratio of PIL-officer referrals to secondary which were resultant: all PIL-officer referrals to secondary
• Average processing time – submission to acceptance
• Ratio of rail and highway shipments and/or conveyances referred to secondary, which were resultant
• Time take to “random” (RA) from data acceptance to RA decision

KPIs deemed “met” - reporting ceased:

• % of highway shipments received electronically prior to arrival at border
• # of highway carriers involved in eManifest
• % of rail shipments received electronically prior to arrival at border
• # of FFs involved in eManifest; % of FF
• % of Agency based paper trade documents;
• Data refresh (how frequently)
• Length of time data kept for analytical purposes
• Risk indicators deployed with eManifest in total
• CBSA is operating with the same standards in EDI as WCO (eManifest context)

Appendix D: Evaluation Methodology and Limitations

Lines of evidence, data, and limitations

Lines of evidence (methods)

The evaluation used multiple lines of evidence, including:

• Regional questionnaire: 40 responses (including some group responses) completed by various regional staff (frontline and regional program area), were received from all Regions, including different modes.
• HQ consultations: Multiple engagements throughout the evaluation period with various internal stakeholder in HQ.
• Review of Financial Data: CAM and Corporate Administrative System (CAS) data provided by FCMB, using commercial processing codes.
• Review of Operational Data:
  • Statistics on EDI applications, system outages, and ISTB’s external client support helpline
  • Number and substance of eManifest notices that the CBSA sent to TCPs
  • Volumes of pre-arrival cargo documents & eHBs submitted through eManifest Portal or EDI from all requests received (paper-based)
  • Number of referrals, exams and exam results by region and mode
  • Number of AMPs issued related to eManifest by type
  • Recovered revenue data from Ambassador Bridge, SOR
  • NTC Fiscal Year-to-Date, Highway Targeting Reports for 2019-20 to 2022-23
• Case Studies: Four case studies depicting different typical eManifest processing situations in the Highway mode in SOR.
• Document and literature Review: review of eManifest-related documents

Limitations

• Some of the data (e.g., helpline data) was provided in calendar years and not fiscal years, limiting the ability to triangulate analyses.
• Scenarios used for the counterfactual assessment show typical situations in the Highway mode in Southern Ontario Region only. Therefore, they do not represent eManifest-related commercial processing in all scenarios, locations, modes, or regions.
• As data was reviewed from 2018-19 to 2022-23, trend analysis are caveated by the impact from the Covid-19 pandemic years (2020-21 and 2021-22).

The intended results of eManifest assessed by the evaluation and the BRP’s benefits

The BRP for the eManifest initiative identified three intended benefits of eManifest at project closure (2018-19). Rather than develop a logic model (which is typically done for evaluating programs), the evaluation modified the original benefit statements to align with eManifest’s current role as a functional component of the Commercial Program. The alignment between the evaluation’s benefit statements and those included in the eManifest BRP is as follows:

Appendix E: Acronyms

ACE

Automated Commercial Environment

ACI

Advance Commercial Information

ACROSS

Accelerated Commercial Release Operations Support System

AMPs

Administrative Monetary Penalties

ARD

Automated Risk Determination

BRP

Benefit Realization Plan

BSO

Border Services Officer

CAM

Costing and Analytical Model

CBSA

Canada Border Services Agency

CCN

Cargo Control Number

CDEM

Commercial Document Entity Management

CECP

Customs Electronic Commerce Platform

CPD

Commercial Program Directorate

CPSG

Commercial Passage

CRD

Commercial Risk Development

CRIM

Customs Referral Inspection Manager

CRN

Conveyance Reference Number

CTAS

Commercial Threat Assessment System

CTB

Commercial and Trade Branch

EDI

Electronic Data Interchange

EDTA

Estimated Date and Time of Arrival

eHB

Electronic House Bill

EPA

Effective Project Approval

ExCC

External Client Communications

FCMB

Financial and Corporate Management Branch

FF

Freight Forwarder

FPOA

First Port of Arrival

HQ

Headquarters

IID

Integrated Import Declaration

ISTB

Information, Science and Technology Branch

KPI

Key Performance Indicator

LTL

Less than Load

NTC

National Targeting Centre

OPI

Office of Primary Interest

PIL

Primary Inspection Line

POE

Port of Entry

QDV

Query and Data Viewing Utility

RAPM

Risk Assessment Program Maintenance

RCNP

Residual Compliance Notification Process

RDC

Rogue Data Corporation

SOR

Southern Ontario Region

SWI

Single Window Initiative

TB

Treasury Board

TDA

Targeting Data Analytics

TCI

Targeting Commercial Intelligence

TCP

Trade Chain Partner

TITAN

Tactical Information Targeting Analysis and Notification System

TO

Targeting officer

TPA

Trading Partner Administrator

TPPD

Transformation, Planning and Projects Directorate

US CBP

U.S. Customs and Border Protection

WCO

World Customs Organization