Success Stories

4

U.S. Customs and Border Protection Cargo Systems Program Directorate (CSPD) Agile Development, Operations, & Maintenance

The CBP CSPD Production application portfolio includes 105 applications which directly support the CBP mission of enforcing trade laws and securing the borders while facilitating legitimate trade. This means processing over 66,000 conveyances daily, collecting over $30B annually, and meeting the needs of diverse stakeholders including the CBP Offices of Field Operations and International Trade, the public trade community, Participating Government Agencies (PGAs), and foreign governments.

Learn More
3

U.S. Customs and Border Protection Passenger Systems Program Directorate (PSPD) Traveler Processing & Vetting Service (TPVS)

The scope of this effort includes a full range of software development life cycle services to support CBP’s mission to safeguard America’s borders and enhance the nation's global economic competitiveness. TPVS supports a 24x7 mission handling millions of daily transactions to facilitate primary and secondary traveler processing at and between U.S. Ports of Entry and vetting services across and beyond DHS.

Learn More
3

U.S. Customs and Border Protection Passenger Systems Program Directorate (PSPD) Traveler Processing & Vetting Service (TPVS)

The scope of this effort includes a full range of software development life cycle services to support CBP’s mission to safeguard America’s borders and enhance the nation's global economic competitiveness. TPVS supports a 24x7 mission handling millions of daily transactions to facilitate primary and secondary traveler processing at and between U.S. Ports of Entry and vetting services across and beyond DHS.

Learn More
2

U.S. Antidumping and Countervailing Duties (AD/CVD) Portal
Modernization

The Department of Commerce (DoC) Office of Enforcement and Compliance collaborated with U.S. Customs and Border Protection (CBP) to initiate this project with the objective of streamlining the Antidumping and Countervailing Duties (AD/CVD) case messages and case reference data functions, resulting in a more user-friendly navigation for the trade community. This deployment of the enhanced AD/CVD module provides a more secure and transparent system and continues to contribute to the efforts by DoC and CBP to level the playing field for domestic industries injured by unfair trade practices.

Learn More
1

Digital Services Application Development & Design Support for Department of Homeland Security (DHS)

 

The purpose of this effort is to utilize the U.S Digital Service Playbook along with emerging technologies to identify and solve core user needs through the development and release to end users of a Digital Minimum Viable Products (MVPs) in the form of enhancements to U.S. Customs and Border Protection (CBP) applications. To achieve this goal, our team establishes and executes a framework for user-centered design, usability research, and agile implementation.

Learn More

U.S. Customs and Border Protection Cargo Systems Program Directorate (CSPD) Agile Development, Operations, & Maintenance

For Cargo Systems Program Division’s (CSPD) ACE program, we provide IT services to develop, migrate, and support cargo system applications, leveraging Agile development methodologies.  The Cargo Production applications provide direct support to the CSPD mission of enforcing trade laws and securing the borders while facilitating legitimate trade.  This means processing over 66,000 conveyances daily, collecting over $30B annually, and meeting the needs of diverse stakeholders including the CBP Offices of Field Operations and International Trade, the public trade community, Participating Government Agencies (PGAs), and foreign governments.  The CSPD application portfolio includes 105 applications. Our 17 Agile teams leverage advanced DevSecOps techniques to average over 50 non-invasive deployments each month.  This allows flexible mission responsiveness while ensuring the stability and quality of the critical production environment.

Operations and Maintenance, Performance Upgrades, and Enhancements

 We manage our Agile teams and individual sprints to balance the capacity for O&M activities and new development initiatives as prioritized by Product Owners and program leadership. We monitor and respond to Remedy tickets submitted on the CSPD applications. Our DevOps team tunes our system alerts from AppDynamics, Oracle Enterprise Manager (OEM), and Splunk to make them meaningful, providing a high level of coverage across ACE applications. We enhance automated test coverage for all applications using a dedicated team that drives test automation across all our Agile teams through the development of new scripts and mentoring developers and testers. The result is a robust set of functional regression tests run daily, with results automatically emailed to our Agile teams.

One of the most important areas of development and maintenance are interfaces. We have established and maintain over 45interfaces between ACE applications and external systems, including ATS. With over 30 years of experience modernizing CSPD ACE and PSPD TECS systems, our team has unique subject matter expertise across a broad set of CBP functional processes. Our expertise is critical to effective integration and has been leveraged to establish the existing interfaces between ACE and TECS to TASPD to share the data used in downstream targeting.

Cloud Migration

We have migrated 47 applications to the CBP AWS Cloud East (CACE) environment and 16 applications to CSPD’s Docker container environment. An additional 55 migrations to these environments are planned. We accomplish these migrations at scale using Agile team and program support engagement, risk mitigation strategies, and proactive communication.

New Developers are assigned to our migration tiger team, where their first assignment is to support the migration of an application out of WebLogic, mentored by our Program Technical Lead. Once they have gained the needed experience working with the technologies and design patterns, they join an Agile team to apply this knowledge and accelerate migrations.  We established a Pre-Production environment in CACE & Docker, certified to store Production data and scaled to handle representative Production volumes. We perform a two-part test in this environment before go-live. First, we complete performance testing of the migrated applications to verify performance under Production load.  We then perform data comparison testing to confirm that a set of messages processed by both the existing Production application, and the migrated version of the application, return the same results.

Performance Innovation

To optimize the performance of our Agile teams at CSPD, we established a robust DevSecOps team that is focused on Production availability and monitoring, expanding the use of our CI/CD pipeline, and supporting release management. Organizing these resources in a separate team allows us to improve availability and efficiency and accommodate changing priorities.  All Teams have fully migrated all ACE applications to automated CI/CD pipelines with code repositories in Stash and/or Jenkins.

A previous migration we completed was for the oldest set of ACE applications, built nearly 15 years ago, which still used Dimensions as their code repository and did not employ automation for deployments. Our team successfully migrated all the code from Dimensions to Stash, implemented automated build and deployments for all environments, and integrated the execution of automated unit tests into the build process.

Our DevOps team supports our release management process by creating Change Requests (CRs) for upcoming releases and managing the implementation of these CRs during our planned outage windows. This team has developed a custom utility to automate the creation of CRs by integrating JIRA and Remedy to automatically generate all necessary artifacts. This reduces CR creation time from 4 to 8 hours to minutes.

Project Management

Effective alignment to respond to evolving needs is the foundation of our ACE Project Management efforts. Our team has established a culture of collaboration, partnership, and investment with CBP leadership that is operationalized by several critical activities:

  • We conduct daily stand ups with the Deputy Executive Director (DXD) and Director of Cargo Systems Development to review any outstanding delivery risks or issues and ensure effective real-time alignment of our team, CSPD, and stakeholders.
  • We provide briefings to the CSPD Executive Director (XD) and his direct reports to provide an in-depth review of project status, upcoming delivery dates, improvement initiatives, and overall funding status.
  • We conduct meetings between OIT & Office of Trade leadership to define and prioritize our program backlog, ensuring alignment on priorities.
  • We ensure we have the right staff with the necessary skills at the right time. Through close alignment and planning with CSPD and program leaders, we anticipate potential staffing gaps in upcoming increments and staff in advance. Through our extensive experience at CBP we also understand what is necessary to ensure we provide clearable staff.

U.S. Customs and Border Protection Passenger Systems Program Directorate (PSPD) Traveler Processing & Vetting Service (TPVS)

The scope of this effort includes a full range of software development life cycle services to support CBP’s mission to safeguard America’s borders and enhance the nation's global economic competitiveness.  TPVS supports a 24x7 mission handling millions of daily transactions to facilitate primary and secondary traveler processing at and between U.S. Ports of Entry and vetting services across and beyond DHS.

Cloud Migration and Technology Modernization

Our team delivers TPVS Application Support services that align with CBP OIT’s modernization initiatives, including full migration to the Cloud by 2024.  To be successful, officers and agents need tailored, intuitive, and advanced capabilities to anticipate and combat emerging threats.  We provide innovative, mobile, resilient, available, reliable, and scalable technologies to propel CBP’s operational environment into the future.

Development and Enhancements

We provide application development, upgrades, updates, modifications, and enhancement services with a focus on a thorough and robust DevSecOps approach. Our enhancements help CBP leadership, agents, and officers meet ever-changing, complex business and technical challenges. This development includes both new applications and major changes to existing applications.

Operations & Maintenance Services

Our team develops and supports operations and maintenance (O&M) solutions, processes, and procedures vital to sustaining the suite of TPVS software applications at the highest levels of security, service, and availability.  Our O&M solutions and services allow CBP leadership to exceed cost, schedule, and performance objectives.  Our full range of O&M solutions ensures computer and TPVS applications operate efficiently, effectively, and securely to support mission requirements.

Project Management and Performance Metrics

We provide project management support services including oversight, control, and direction in team building, communications, time management, quality assurance, quality control, procedure development, risk management, configuration management, cost management, and software integration.  The metrics we provide allow visibility into the full software development life-cycle, and gives OIT the ability to baseline, discover trends, identify areas for improvement, and have up-to-date information on the size and scale of all TPVS applications and services.

U.S. Antidumping and Countervailing Duties (AD/CVD) Portal Modernization

Department of Commerce (DoC) Office of Enforcement and Compliance collaborated with U.S. Customs and Border Protection (CBP) to initiate this project with the objective of streamlining the Antidumping and Countervailing Duties (AD/CVD) case messages and case reference data functions, resulting in a more user-friendly navigation for the trade community. This deployment of the enhanced AD/CVD module provides a more secure and transparent system and continues to contribute to the efforts by DoC and CBP to level the playing field for domestic industries injured by unfair trade practices.

The scope for this effort included the redesign and development of the user-interface, data structure, usability, and security of the existing ACE Antidumping/Countervailing Duty (AD/CVD) Portal. In addition to enhancing the AD/CVD portal, our team was responsible for user provisioning and training key stakeholders.

A challenge with this work was that the majority of our key stakeholders had not worked with Agile before. Therefore, it was critical that we started our efforts with an education on the benefits of the Agile process to ensure we received buy-in from the start. To most effectively train the group on key principles and benefits of Agile, we invited key stakeholders to a workshop where we customized the training to the DHS agile process and leveraged our team’s knowledge of AD/CVD to provide content-accurate examples. The feedback immediately following the training was excellent and this enabled the team to hit the ground running.

With the stakeholders aligned to the agile process, our team began to utilize the digital service playbook by understanding what the stakeholders needed and addressed the whole experience using UI/UX user-centric stories. Certain key features in the AD/CVD single window redesign include improved user-centric design with fewer clicks for ease of use, enhanced organization of key data points for transparency, automation of manual processes to increase collaboration between stakeholders, exportable data, advance search capability and quick search filters. A premier feature of this enhancement is the utilization of CBP Amazon Web Service Cloud Environment (CACE), which includes a variety of security benefits. A portion of this effort also included the conversion of data from IBM DB2 to PostgreSQL. We leveraged DevSecOps heavily to focus our continuous integration (CI) and continuous delivery (CD) to ensure active testing and verification of code through automation testing, code coverage, analytics, and blue/green deployments.

Through these efforts, our team was able to deliver a modernized AD/CVD application, that provided more accessibility, a stronger user experience, reliability, maintainability and significant cost savings to the government. To read more about this success, please see the following CBP Press Release: “U.S. Customs and Border Protection and Department of Commerce Enhance ACE”.

Digital Services Application Development & Design Support for Department of Homeland Security (DHS)

The purpose of this effort is to utilize the U.S Digital Service Playbook along with emerging technologies to identify and solve core user needs through the development and release to end users of a Digital Minimum Viable Products (MVPs) in the form of enhancements to U.S. Customs and Border Protection (CBP) applications.  To achieve this goal, our team establishes and executes a framework for user-centered design, usability research, and agile implementation.

Key program goals for this project are to:

  • Leverage the CBP Amazon Web Services Cloud Environment (CACE) as much as possible in the design and delivery of functionality
  • Migrate to a common processing environment for functionality in current CSPD systems (ACE and ACS) as well supporting newly developed functionality using a common processing platform for all Cargo systems.
  • Support a component and service-based architecture operating in a distributed environment, leading to more efficient maintenance, extension, updates and upgrades to Cargo systems.
  • Use proven open-source technologies to the greatest extent possible to reduce acquisition and Operations and Maintenance (O&M) costs.

Our team collaborates with DHS to determine how problems will be solved through digitalization, utilizing user-centered design techniques and product management methods to create an initial product vision, product roadmap, and scope of the MVP.

We begin each portion of scope by exploring and pinpointing the needs of the people who will use the service, and the ways the service will fit into their lives. We continually test the products with end users to ensure delivery is focused on meeting their needs.  Key to our success has been the upfront time spent collaborating with current and prospective users of the service.  We leverage that feedback using a range of qualitative and quantitative research methods to determine people’s goals, needs, and behaviors.

Our team then develops mock-ups or prototypes of solutions to share with prospective users.  The feedback then fuels the backlog of prioritized user stories developed during the agile development process.  While the iterative development process is underway, there is a continuous cycle of testing and demonstrating the progress to stakeholders so that the feedback loop can continuously improve the output.

Agile ceremonies leveraged by our team include iteration retrospectives, release planning, backlog grooming, and other common activities associated with iterative design and agile methodologies.  These agile best practices lead to the release of high-quality MVPs to the end users.

Get Innovative Solutions for Your Complex IT Problems

Contact us for comprehensive IT solutions that can help improve your organization’s processes and efficiency. Our team is ready to help you out.