waste-movement-backend service

Internal backend service for DEFRA's Digital Waste Tracking Service. This service provides the data persistence layer for waste receipt movements, storing and managing waste tracking data in MongoDB.

Overview

The waste-movement-backend is an internal service that:

• Persists waste receipt movement data to MongoDB
• Manages waste input records with unique tracking IDs
• Maintains complete history of all changes to waste movements
• Tracks invalid submissions for audit purposes
• Provides organizational validation via API codes

This service is not directly accessible externally. It is called by the waste-movement-external-api service, which handles public-facing REST endpoints and authentication.

Architecture

The waste tracking system follows a 3-tier architecture:

External Clients
       ↓
waste-movement-external-api (Port 3001)
  - JWT authentication via AWS Cognito
  - Public REST endpoints
  - Generates tracking IDs via waste-tracking-id-backend
       ↓
waste-movement-backend (Internal)
  - Data persistence layer
  - MongoDB operations
  - History tracking
       ↓
MongoDB (Database)
  - waste-inputs (current state)
  - waste-inputs-history (audit trail)
  - invalid-submissions (failed submissions)

• Requirements
  • Node.js
• Local development
  • Setup
  • Development
  • Testing
  • Production
  • Npm scripts
  • Update dependencies
  • Formatting
    • Windows prettier issue
• API endpoints
• Development helpers
  • MongoDB Locks
  • Proxy
• Docker
  • Development image
  • Production image
  • Docker Compose
  • Dependabot
  • SonarCloud
• Licence
  • About the licence

Requirements

Node.js

Please install the Node.js version noted in engines in package.json and npm >= v12. You will find it easier to use the Node Version Manager nvm

To use the correct version of Node.js for this application, via nvm:

cd waste-movement-backend
nvm use

Local development

Setup

Install application dependencies:

npm install

Development

To run the application in development mode run:

npm run dev

Testing

To test the application run:

npm run test

Production

To mimic the application running in production mode locally run:

npm start

Npm scripts

All available Npm scripts can be seen in package.json. To view them in your command line run:

npm run

Update dependencies

To update dependencies use npm-check-updates:

The following script is a good start. Check out all the options on the npm-check-updates

ncu --interactive --format group

Formatting

Windows prettier issue

If you are having issues with formatting of line breaks on Windows update your global git config by running:

git config --global core.autocrlf false

API endpoints

Endpoint	Description
GET: /health	Health
GET: /example	Example API (remove as needed)
GET: /example/<id>	Example API (remove as needed)

Development helpers

MongoDB Locks

If you require a write lock for Mongo you can acquire it via server.locker or request.locker:

async function doStuff(server) {
  const lock = await server.locker.lock('unique-resource-name')

  if (!lock) {
    // Lock unavailable
    return
  }

  try {
    // do stuff
  } finally {
    await lock.free()
  }
}

Keep it small and atomic.

You may use using for the lock resource management. Note test coverage reports do not like that syntax.

async function doStuff(server) {
  await using lock = await server.locker.lock('unique-resource-name')

  if (!lock) {
    // Lock unavailable
    return
  }

  // do stuff

  // lock automatically released
}

Helper methods are also available in /src/helpers/mongo-lock.js.

Proxy

We are using forward-proxy which is set up by default. To make use of this: import { fetch } from 'undici' then because of the setGlobalDispatcher(new ProxyAgent(proxyUrl)) calls will use the ProxyAgent Dispatcher

If you are not using Wreck, Axios or Undici or a similar http that uses Request. Then you may have to provide the proxy dispatcher:

To add the dispatcher to your own client:

import { ProxyAgent } from 'undici'

return await fetch(url, {
  dispatcher: new ProxyAgent({
    uri: proxyUrl,
    keepAliveTimeout: 10,
    keepAliveMaxTimeout: 10
  })
})

Docker

Development image

Build:

docker build --target development --no-cache --tag waste-movement-backend:development .

Run:

docker run -e PORT=3001 -p 3001:3001 waste-movement-backend:development

Production image

Build:

docker build --no-cache --tag waste-movement-backend .

Run:

docker run -e PORT=3001 -p 3001:3001 waste-movement-backend

Docker Compose

A local environment with:

• MongoDB (required for this service)
• This service

docker compose up --build -d

The stack is defined once in digital-waste-tracking-uat (shared-compose/) and shared with the other service repos. compose.yml includes it and compose.override-shared.yml builds this service from your checkout. Its credentials are made-up local-only values, so no .env is needed; see .env.example for the optional overrides.

Dependabot

We have added an example dependabot configuration file to the repository. You can enable it by renaming the .github/example.dependabot.yml to .github/dependabot.yml

SonarCloud

Instructions for setting up SonarCloud can be found in sonar-project.properties

Licence

THIS INFORMATION IS LICENSED UNDER THE CONDITIONS OF THE OPEN GOVERNMENT LICENCE found at:

http://www.nationalarchives.gov.uk/doc/open-government-licence/version/3

The following attribution statement MUST be cited in your products and applications when using this information.

Contains public sector information licensed under the Open Government license v3

About the licence

The Open Government Licence (OGL) was developed by the Controller of Her Majesty's Stationery Office (HMSO) to enable information providers in the public sector to license the use and re-use of their information under a common open licence.

It is designed to encourage use and re-use of information freely and flexibly, with only a few conditions.