diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index d2d78065a..58f883ee2 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -22,6 +22,24 @@ jobs: node-version: [22.x, 24.x] mongodb-version: ['6.0', '7.0', '8.0'] + # PostgreSQL service container for the postgres integration tests. A + # single version (postgres:16) keeps the lane fast; a broader version + # matrix can follow later if needed. + services: + postgres: + image: postgres:16 + env: + POSTGRES_USER: postgres + POSTGRES_PASSWORD: postgres + POSTGRES_DB: git_proxy_test + ports: + - 5432:5432 + options: >- + --health-cmd "pg_isready -U postgres" + --health-interval 10s + --health-timeout 5s + --health-retries 5 + steps: - name: Harden Runner uses: step-security/harden-runner@05e31511f85b41b11d1cf0ef85d0992719546e2c # v2.21.0 @@ -71,6 +89,12 @@ jobs: GIT_PROXY_MONGO_CONNECTION_STRING: mongodb://localhost:27017/git-proxy-test run: npm run test:integration + - name: PostgreSQL Integration Tests + env: + RUN_POSTGRES_TESTS: 'true' + GIT_PROXY_POSTGRES_CONNECTION_STRING: postgresql://postgres:postgres@localhost:5432/git_proxy_test + run: npm run test:integration:postgres + - name: Upload test coverage report uses: codecov/codecov-action@fb8b3582c8e4def4969c97caa2f19720cb33a72f # v7.0.0 with: diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 6d09f1be5..9f0978866 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -12,6 +12,8 @@ For project governance, roles, and voting procedures, see the [Governance sectio - [Development Workflow](#development-workflow) - [Testing](#testing) - [Unit Tests](#unit-tests) + - [MongoDB Integration Tests](#mongodb-integration-tests) + - [PostgreSQL Integration Tests](#postgresql-integration-tests) - [End-to-End Tests](#end-to-end-tests) - [UI Tests (Cypress)](#ui-tests-cypress) - [Fuzz Tests](#fuzz-tests) @@ -87,7 +89,7 @@ git-proxy/ ├── src/ │ ├── proxy/ # Core proxy logic (action chain, processors) │ ├── service/ # Express app, API routes, authentication (Passport.js) -│ ├── db/ # Database abstraction (MongoDB + NeDB) +│ ├── db/ # Database abstraction (MongoDB, PostgreSQL, NeDB) │ ├── config/ # Configuration loading and generated types │ ├── ui/ # React dashboard (Material-UI) │ ├── plugin.ts # Plugin base classes (PushActionPlugin, PullActionPlugin) @@ -113,7 +115,7 @@ git-proxy/ - **Action chain**: Git push/fetch requests flow through a chain of processors in `src/proxy/chain.ts` - **Plugin system**: Extends the action chain with custom logic (see `src/plugin.ts`) -- **Dual database**: MongoDB for production state; [NeDB](https://github.com/seald/nedb) for local file-based development (`.data/` directory) +- **Pluggable database**: MongoDB or PostgreSQL for production state; [NeDB](https://github.com/seald/nedb) for local file-based development (`.data/` directory) - **Authentication**: Passport.js strategies (local, Active Directory, OpenID Connect) ## Development Workflow @@ -202,6 +204,31 @@ Configuration: [vitest.config.integration.ts](vitest.config.integration.ts) In CI, `RUN_MONGO_TESTS` is set automatically in the workflow that runs integration tests. +### PostgreSQL Integration Tests + +Some tests require a real PostgreSQL instance. These are guarded by the `RUN_POSTGRES_TESTS` environment variable and run separately from unit tests. + +```bash +# Start PostgreSQL with Docker +docker run -d --name postgres-test -p 5432:5432 \ + -e POSTGRES_USER=postgres \ + -e POSTGRES_PASSWORD=postgres \ + -e POSTGRES_DB=git_proxy_test \ + postgres:16 + +# Run PostgreSQL integration tests +npm run test:integration:postgres + +# Cleanup +docker stop postgres-test && docker rm postgres-test +``` + +Configuration: [vitest.config.integration.postgres.ts](vitest.config.integration.postgres.ts) + +Unlike the MongoDB lane, `RUN_POSTGRES_TESTS` and the connection string are set by the Vitest config itself, so no extra environment variables are required on the command line — you only need a PostgreSQL instance reachable at `postgresql://postgres:postgres@localhost:5432/git_proxy_test`. + +In CI, the PostgreSQL integration tests run against a `postgres:16` service container in the same workflow as the MongoDB integration tests. + ### End-to-End Tests E2E tests perform real git operations against a Dockerized environment. They use Vitest with a separate config. diff --git a/config.schema.json b/config.schema.json index a54d828d4..a9f9da668 100644 --- a/config.schema.json +++ b/config.schema.json @@ -568,6 +568,20 @@ "enabled": { "type": "boolean" } }, "required": ["type", "enabled"] + }, + { + "type": "object", + "name": "PostgreSQL Config", + "description": "Connection properties for PostgreSQL. The `connectionString` may also be supplied via the `GIT_PROXY_POSTGRES_CONNECTION_STRING` environment variable.", + "properties": { + "type": { "type": "string", "const": "postgres" }, + "enabled": { "type": "boolean" }, + "connectionString": { + "type": "string", + "description": "PostgreSQL client connection string, see [https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING). If omitted, `GIT_PROXY_POSTGRES_CONNECTION_STRING` is used as a fallback." + } + }, + "required": ["type", "enabled"] } ] }, diff --git a/eslint.config.mjs b/eslint.config.mjs index b074b622f..510ca046b 100644 --- a/eslint.config.mjs +++ b/eslint.config.mjs @@ -57,6 +57,8 @@ export default defineConfig( // vendored code we're not changing 'src/ui/assets/js/**', 'src/ui/assets/css/**', + // local claude worktrees / scratch + '.claude/**', ], }, diff --git a/package-lock.json b/package-lock.json index 7bdeca97b..382bc04c9 100644 --- a/package-lock.json +++ b/package-lock.json @@ -26,6 +26,7 @@ "clsx": "^2.1.1", "concurrently": "^9.2.4", "connect-mongo": "^6.0.0", + "connect-pg-simple": "^10.0.0", "cors": "^2.8.6", "diff2html": "^3.4.56", "env-paths": "^4.0.0", @@ -50,6 +51,7 @@ "passport": "^0.7.0", "passport-activedirectory": "^1.4.0", "passport-local": "^1.0.0", + "pg": "^8.20.0", "react": "^19.2.5", "react-dom": "^19.2.5", "react-is": "^19.0.0", @@ -74,6 +76,7 @@ "@eslint/json": "^2.0.0", "@tailwindcss/vite": "^4.2.2", "@types/activedirectory2": "^1.2.6", + "@types/connect-pg-simple": "^7.0.3", "@types/cors": "^2.8.19", "@types/express": "^5.0.6", "@types/express-http-proxy": "^1.6.7", @@ -85,6 +88,7 @@ "@types/node": "^22.19.7", "@types/passport": "^1.0.17", "@types/passport-local": "^1.0.38", + "@types/pg": "^8.20.0", "@types/react": "^19.2.14", "@types/react-dom": "^19.2.3", "@types/ssh2": "^1.15.5", @@ -2218,18 +2222,6 @@ "react-dom": "^16.8.0 || ^17.0.0" } }, - "node_modules/@finos/git-proxy/node_modules/@types/react": { - "version": "17.0.93", - "resolved": "https://registry.npmjs.org/@types/react/-/react-17.0.93.tgz", - "integrity": "sha512-KM4Ty/ZTLZupiYxZVAlP+InNJS3De6uBMdq0ePa6/04+eG9Y7ftnWfst1xTLQ5rwAhgHwQ4momt/O4KepdGBTw==", - "extraneous": true, - "license": "MIT", - "dependencies": { - "@types/prop-types": "*", - "@types/scheduler": "^0.16", - "csstype": "^3.2.2" - } - }, "node_modules/@finos/git-proxy/node_modules/dom-serializer": { "version": "0.2.2", "resolved": "https://registry.npmjs.org/dom-serializer/-/dom-serializer-0.2.2.tgz", @@ -3578,9 +3570,6 @@ "arm64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "MIT", "optional": true, "os": [ @@ -3598,9 +3587,6 @@ "arm64" ], "dev": true, - "libc": [ - "musl" - ], "license": "MIT", "optional": true, "os": [ @@ -3618,9 +3604,6 @@ "ppc64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "MIT", "optional": true, "os": [ @@ -3638,9 +3621,6 @@ "s390x" ], "dev": true, - "libc": [ - "glibc" - ], "license": "MIT", "optional": true, "os": [ @@ -3658,9 +3638,6 @@ "x64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "MIT", "optional": true, "os": [ @@ -3678,9 +3655,6 @@ "x64" ], "dev": true, - "libc": [ - "musl" - ], "license": "MIT", "optional": true, "os": [ @@ -4028,9 +4002,6 @@ "arm64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "MIT", "optional": true, "os": [ @@ -4048,9 +4019,6 @@ "arm64" ], "dev": true, - "libc": [ - "musl" - ], "license": "MIT", "optional": true, "os": [ @@ -4068,9 +4036,6 @@ "x64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "MIT", "optional": true, "os": [ @@ -4088,9 +4053,6 @@ "x64" ], "dev": true, - "libc": [ - "musl" - ], "license": "MIT", "optional": true, "os": [ @@ -4130,6 +4092,72 @@ "node": ">=14.0.0" } }, + "node_modules/@tailwindcss/oxide-wasm32-wasi/node_modules/@emnapi/core": { + "version": "1.11.1", + "dev": true, + "inBundle": true, + "license": "MIT", + "optional": true, + "dependencies": { + "@emnapi/wasi-threads": "1.2.2", + "tslib": "^2.4.0" + } + }, + "node_modules/@tailwindcss/oxide-wasm32-wasi/node_modules/@emnapi/runtime": { + "version": "1.11.1", + "dev": true, + "inBundle": true, + "license": "MIT", + "optional": true, + "dependencies": { + "tslib": "^2.4.0" + } + }, + "node_modules/@tailwindcss/oxide-wasm32-wasi/node_modules/@emnapi/wasi-threads": { + "version": "1.2.2", + "dev": true, + "inBundle": true, + "license": "MIT", + "optional": true, + "dependencies": { + "tslib": "^2.4.0" + } + }, + "node_modules/@tailwindcss/oxide-wasm32-wasi/node_modules/@napi-rs/wasm-runtime": { + "version": "1.1.4", + "dev": true, + "inBundle": true, + "license": "MIT", + "optional": true, + "dependencies": { + "@tybys/wasm-util": "^0.10.1" + }, + "funding": { + "type": "github", + "url": "https://github.com/sponsors/Brooooooklyn" + }, + "peerDependencies": { + "@emnapi/core": "^1.7.1", + "@emnapi/runtime": "^1.7.1" + } + }, + "node_modules/@tailwindcss/oxide-wasm32-wasi/node_modules/@tybys/wasm-util": { + "version": "0.10.2", + "dev": true, + "inBundle": true, + "license": "MIT", + "optional": true, + "dependencies": { + "tslib": "^2.4.0" + } + }, + "node_modules/@tailwindcss/oxide-wasm32-wasi/node_modules/tslib": { + "version": "2.8.1", + "dev": true, + "inBundle": true, + "license": "0BSD", + "optional": true + }, "node_modules/@tailwindcss/oxide-win32-arm64-msvc": { "version": "4.3.3", "resolved": "https://registry.npmjs.org/@tailwindcss/oxide-win32-arm64-msvc/-/oxide-win32-arm64-msvc-4.3.3.tgz", @@ -4303,17 +4331,6 @@ "assertion-error": "^2.0.1" } }, - "node_modules/@types/chai": { - "version": "5.2.3", - "resolved": "https://registry.npmjs.org/@types/chai/-/chai-5.2.3.tgz", - "integrity": "sha512-Mw558oeA9fFbv65/y4mHtXDs9bPnFMZAL/jxdPFUpOHHIXX91mcgEHbS5Lahr+pwZFR8A7GQleRWeI6cGFC2UA==", - "dev": true, - "license": "MIT", - "dependencies": { - "@types/deep-eql": "*", - "assertion-error": "^2.0.1" - } - }, "node_modules/@types/connect": { "version": "3.4.38", "resolved": "https://registry.npmjs.org/@types/connect/-/connect-3.4.38.tgz", @@ -4324,6 +4341,18 @@ "@types/node": "*" } }, + "node_modules/@types/connect-pg-simple": { + "version": "7.0.3", + "resolved": "https://registry.npmjs.org/@types/connect-pg-simple/-/connect-pg-simple-7.0.3.tgz", + "integrity": "sha512-NGCy9WBlW2bw+J/QlLnFZ9WjoGs6tMo3LAut6mY4kK+XHzue//lpNVpAvYRpIwM969vBRAM2Re0izUvV6kt+NA==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/express": "*", + "@types/express-session": "*", + "@types/pg": "*" + } + }, "node_modules/@types/conventional-commits-parser": { "version": "5.0.2", "resolved": "https://registry.npmjs.org/@types/conventional-commits-parser/-/conventional-commits-parser-5.0.2.tgz", @@ -4532,12 +4561,17 @@ "@types/passport": "*" } }, - "node_modules/@types/prop-types": { - "version": "15.7.15", - "resolved": "https://registry.npmjs.org/@types/prop-types/-/prop-types-15.7.15.tgz", - "integrity": "sha512-F6bEyamV9jKGAFBEmlQnesRPGOQqS2+Uwi0Em15xenOxHaf2hv6L8YCVn3rPdPJOiJfPiCnLIRyvwVaqMY3MIw==", - "extraneous": true, - "license": "MIT" + "node_modules/@types/pg": { + "version": "8.23.1", + "resolved": "https://registry.npmjs.org/@types/pg/-/pg-8.23.1.tgz", + "integrity": "sha512-fKVHpikPdg4GKks3JuLEhvwSyvwzF23hnabPy6DD8ljVbC7+6J5dQzdv4arV6jqq57djnMgs1HKBxX4P8aBI3A==", + "dev": true, + "license": "MIT", + "dependencies": { + "@types/node": "*", + "pg-protocol": "*", + "pg-types": "^2.2.0" + } }, "node_modules/@types/qs": { "version": "6.15.1", @@ -4581,13 +4615,6 @@ "@types/react": "*" } }, - "node_modules/@types/scheduler": { - "version": "0.16.8", - "resolved": "https://registry.npmjs.org/@types/scheduler/-/scheduler-0.16.8.tgz", - "integrity": "sha512-WZLiwShhwLRmeV6zH+GkbOFT6Z6VklCItrDioxUnv+u4Ll+8vKeFySoFyK/0ctcRpOmwAicELfmys1sDc/Rw+A==", - "extraneous": true, - "license": "MIT" - }, "node_modules/@types/send": { "version": "1.2.1", "resolved": "https://registry.npmjs.org/@types/send/-/send-1.2.1.tgz", @@ -6942,6 +6969,18 @@ "mongodb": ">=5.0.0" } }, + "node_modules/connect-pg-simple": { + "version": "10.0.0", + "resolved": "https://registry.npmjs.org/connect-pg-simple/-/connect-pg-simple-10.0.0.tgz", + "integrity": "sha512-pBGVazlqiMrackzCr0eKhn4LO5trJXsOX0nQoey9wCOayh80MYtThCbq8eoLsjpiWgiok/h+1/uti9/2/Una8A==", + "license": "MIT", + "dependencies": { + "pg": "^8.12.0" + }, + "engines": { + "node": "^18.18.0 || ^20.9.0 || >=22.0.0" + } + }, "node_modules/content-disposition": { "version": "1.1.0", "resolved": "https://registry.npmjs.org/content-disposition/-/content-disposition-1.1.0.tgz", @@ -11600,9 +11639,6 @@ "arm64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "MPL-2.0", "optional": true, "os": [ @@ -11624,9 +11660,6 @@ "arm64" ], "dev": true, - "libc": [ - "musl" - ], "license": "MPL-2.0", "optional": true, "os": [ @@ -11648,9 +11681,6 @@ "x64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "MPL-2.0", "optional": true, "os": [ @@ -11672,9 +11702,6 @@ "x64" ], "dev": true, - "libc": [ - "musl" - ], "license": "MPL-2.0", "optional": true, "os": [ @@ -13568,6 +13595,95 @@ "dev": true, "license": "MIT" }, + "node_modules/pg": { + "version": "8.23.0", + "resolved": "https://registry.npmjs.org/pg/-/pg-8.23.0.tgz", + "integrity": "sha512-Ip2EQCngowJLGOfCwkFhPXU7/ljlhn6Rxlmy4XYfL2Y+vyRM59+8uR2xqRWKdYmbXmxCFOAmKxBuSUCdF34qLg==", + "license": "MIT", + "dependencies": { + "pg-connection-string": "^2.14.0", + "pg-pool": "^3.14.0", + "pg-protocol": "^1.16.0", + "pg-types": "2.2.0", + "pgpass": "1.0.5" + }, + "engines": { + "node": ">= 16.0.0" + }, + "optionalDependencies": { + "pg-cloudflare": "^1.4.0" + }, + "peerDependencies": { + "pg-native": ">=3.0.1" + }, + "peerDependenciesMeta": { + "pg-native": { + "optional": true + } + } + }, + "node_modules/pg-cloudflare": { + "version": "1.4.0", + "resolved": "https://registry.npmjs.org/pg-cloudflare/-/pg-cloudflare-1.4.0.tgz", + "integrity": "sha512-Vo7z/6rrQYxpNRylp4Tlob2elzbh+N/MOQbxFVWCxS7oEx6jF53GTJFxK2WWpKuBRkmiin4Mt+xofFDjx09R0A==", + "license": "MIT", + "optional": true + }, + "node_modules/pg-connection-string": { + "version": "2.14.0", + "resolved": "https://registry.npmjs.org/pg-connection-string/-/pg-connection-string-2.14.0.tgz", + "integrity": "sha512-XwWDGcLRGCXAR8F/AM5bG7Q+A3Wm2s6QeEjlOKZLlH3UYcguiqCWKyWXVag5TLTIjR7oOJUY8kcADaZgWPyLeg==", + "license": "MIT" + }, + "node_modules/pg-int8": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/pg-int8/-/pg-int8-1.0.1.tgz", + "integrity": "sha512-WCtabS6t3c8SkpDBUlb1kjOs7l66xsGdKpIPZsg4wR+B3+u9UAum2odSsF9tnvxg80h4ZxLWMy4pRjOsFIqQpw==", + "license": "ISC", + "engines": { + "node": ">=4.0.0" + } + }, + "node_modules/pg-pool": { + "version": "3.14.0", + "resolved": "https://registry.npmjs.org/pg-pool/-/pg-pool-3.14.0.tgz", + "integrity": "sha512-gKtPkFdQPU3DksooVLi9LsjZxrsBUZIpa+7aVx+LV5pNh0KzP4Zleud2po+ConrxbuXGBJ6Hfer6hdgpIBpBaw==", + "license": "MIT", + "peerDependencies": { + "pg": ">=8.0" + } + }, + "node_modules/pg-protocol": { + "version": "1.16.0", + "resolved": "https://registry.npmjs.org/pg-protocol/-/pg-protocol-1.16.0.tgz", + "integrity": "sha512-sILXutLVjCLjcDuOmvhX5e2Z4cS5qG/6Bu3VkpFwdf/633ElGLpEh9bgmuI5I4sqKqkifQiGyiCcx1HdtrK7tg==", + "license": "MIT" + }, + "node_modules/pg-types": { + "version": "2.2.0", + "resolved": "https://registry.npmjs.org/pg-types/-/pg-types-2.2.0.tgz", + "integrity": "sha512-qTAAlrEsl8s4OiEQY69wDvcMIdQN6wdz5ojQiOy6YRMuynxenON0O5oCpJI6lshc6scgAY8qvJ2On/p+CXY0GA==", + "license": "MIT", + "dependencies": { + "pg-int8": "1.0.1", + "postgres-array": "~2.0.0", + "postgres-bytea": "~1.0.0", + "postgres-date": "~1.0.4", + "postgres-interval": "^1.1.0" + }, + "engines": { + "node": ">=4" + } + }, + "node_modules/pgpass": { + "version": "1.0.5", + "resolved": "https://registry.npmjs.org/pgpass/-/pgpass-1.0.5.tgz", + "integrity": "sha512-FdW9r/jQZhSeohs1Z3sI1yxFQNFvMcnmfuj4WBMUTxOrAyLMaTcE1aAMBiTlbMNaXvBCQuVi0R7hd8udDSP7ug==", + "license": "MIT", + "dependencies": { + "split2": "^4.1.0" + } + }, "node_modules/picocolors": { "version": "1.1.1", "resolved": "https://registry.npmjs.org/picocolors/-/picocolors-1.1.1.tgz", @@ -13731,6 +13847,45 @@ "node": "^10 || ^12 || >=14" } }, + "node_modules/postgres-array": { + "version": "2.0.0", + "resolved": "https://registry.npmjs.org/postgres-array/-/postgres-array-2.0.0.tgz", + "integrity": "sha512-VpZrUqU5A69eQyW2c5CA1jtLecCsN2U/bD6VilrFDWq5+5UIEVO7nazS3TEcHf1zuPYO/sqGvUvW62g86RXZuA==", + "license": "MIT", + "engines": { + "node": ">=4" + } + }, + "node_modules/postgres-bytea": { + "version": "1.0.1", + "resolved": "https://registry.npmjs.org/postgres-bytea/-/postgres-bytea-1.0.1.tgz", + "integrity": "sha512-5+5HqXnsZPE65IJZSMkZtURARZelel2oXUEO8rH83VS/hxH5vv1uHquPg5wZs8yMAfdv971IU+kcPUczi7NVBQ==", + "license": "MIT", + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/postgres-date": { + "version": "1.0.7", + "resolved": "https://registry.npmjs.org/postgres-date/-/postgres-date-1.0.7.tgz", + "integrity": "sha512-suDmjLVQg78nMK2UZ454hAG+OAW+HQPZ6n++TNDUX+L0+uUlLywnoxJKDou51Zm+zTCjrCl0Nq6J9C5hP9vK/Q==", + "license": "MIT", + "engines": { + "node": ">=0.10.0" + } + }, + "node_modules/postgres-interval": { + "version": "1.2.0", + "resolved": "https://registry.npmjs.org/postgres-interval/-/postgres-interval-1.2.0.tgz", + "integrity": "sha512-9ZhXKM/rw350N1ovuWHbGxnGh/SNJ4cnxHiM0rxE4VN41wsg8P8zWn9hv/buK00RP4WvlOyr/RBDiptyxVbkZQ==", + "license": "MIT", + "dependencies": { + "xtend": "^4.0.0" + }, + "engines": { + "node": ">=0.10.0" + } + }, "node_modules/precond": { "version": "0.2.3", "resolved": "https://registry.npmjs.org/precond/-/precond-0.2.3.tgz", @@ -15357,7 +15512,6 @@ "version": "4.2.0", "resolved": "https://registry.npmjs.org/split2/-/split2-4.2.0.tgz", "integrity": "sha512-UcjcJOWknrNkF6PLX83qcHM6KHgVKNkV62Y8a5uYDVv9ydGQVwAHMKqHdJje1VTWpljG0WYpCDhrCdAOYH4TWg==", - "dev": true, "license": "ISC", "engines": { "node": ">= 10.x" @@ -16957,8 +17111,10 @@ "version": "5.9.3", "resolved": "https://registry.npmjs.org/typescript/-/typescript-5.9.3.tgz", "integrity": "sha512-jl1vZzPDinLr9eUt3J/t7V6FgNEw9QjvBPdysz9KfQDD41fQrC2Y4vKQdiaUpFT4bXlb1RHhLpp8wtm6M5TgSw==", - "extraneous": true, + "dev": true, "license": "Apache-2.0", + "optional": true, + "peer": true, "bin": { "tsc": "bin/tsc", "tsserver": "bin/tsserver" @@ -17110,9 +17266,6 @@ "arm64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "MPL-2.0", "optional": true, "os": [ @@ -17134,9 +17287,6 @@ "arm64" ], "dev": true, - "libc": [ - "musl" - ], "license": "MPL-2.0", "optional": true, "os": [ @@ -17158,9 +17308,6 @@ "x64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "MPL-2.0", "optional": true, "os": [ @@ -17182,9 +17329,6 @@ "x64" ], "dev": true, - "libc": [ - "musl" - ], "license": "MPL-2.0", "optional": true, "os": [ @@ -17647,6 +17791,15 @@ } } }, + "node_modules/xtend": { + "version": "4.0.2", + "resolved": "https://registry.npmjs.org/xtend/-/xtend-4.0.2.tgz", + "integrity": "sha512-LKYU1iAXJXUgAXn9URjiu+MWhyUXHsvfp7mcuYm9dSUKK0/CjtrUwFAxD82/mCWbtLsGjFIad0wIsod4zrTAEQ==", + "license": "MIT", + "engines": { + "node": ">=0.4" + } + }, "node_modules/y18n": { "version": "5.0.8", "resolved": "https://registry.npmjs.org/y18n/-/y18n-5.0.8.tgz", diff --git a/package.json b/package.json index 19ba4bc8a..c4cd752a6 100644 --- a/package.json +++ b/package.json @@ -70,6 +70,7 @@ "test-coverage": "cross-env NODE_ENV=test vitest --run --dir ./test --coverage", "test-coverage-ci": "cross-env NODE_ENV=test vitest --run --dir ./test --coverage.enabled=true --coverage.reporter=lcovonly --coverage.reporter=text", "test:integration": "cross-env NODE_ENV=test vitest --run --config vitest.config.integration.ts", + "test:integration:postgres": "cross-env NODE_ENV=test vitest --run --config vitest.config.integration.postgres.ts", "test:watch": "cross-env NODE_ENV=test vitest --dir ./test --watch", "test:migrate": "cross-env NODE_ENV=test vitest --run --dir ./scripts/migrate/test", "prepare": "node ./scripts/prepare.js", @@ -120,6 +121,7 @@ "clsx": "^2.1.1", "concurrently": "^9.2.4", "connect-mongo": "^6.0.0", + "connect-pg-simple": "^10.0.0", "cors": "^2.8.6", "diff2html": "^3.4.56", "env-paths": "^4.0.0", @@ -144,6 +146,7 @@ "passport": "^0.7.0", "passport-activedirectory": "^1.4.0", "passport-local": "^1.0.0", + "pg": "^8.20.0", "react": "^19.2.5", "react-dom": "^19.2.5", "react-is": "^19.0.0", @@ -164,6 +167,7 @@ "@eslint/json": "^2.0.0", "@tailwindcss/vite": "^4.2.2", "@types/activedirectory2": "^1.2.6", + "@types/connect-pg-simple": "^7.0.3", "@types/cors": "^2.8.19", "@types/express": "^5.0.6", "@types/express-http-proxy": "^1.6.7", @@ -175,6 +179,7 @@ "@types/node": "^22.19.7", "@types/passport": "^1.0.17", "@types/passport-local": "^1.0.38", + "@types/pg": "^8.20.0", "@types/react": "^19.2.14", "@types/react-dom": "^19.2.3", "@types/ssh2": "^1.15.5", diff --git a/proxy.config.json b/proxy.config.json index 5848348c6..2e2f5793c 100644 --- a/proxy.config.json +++ b/proxy.config.json @@ -39,6 +39,11 @@ "ssl": true }, "enabled": false + }, + { + "type": "postgres", + "connectionString": "postgresql://localhost:5432/gitproxy", + "enabled": false } ], "authentication": [ diff --git a/src/config/env.ts b/src/config/env.ts index 503764ee1..aab264341 100644 --- a/src/config/env.ts +++ b/src/config/env.ts @@ -29,6 +29,7 @@ const { GIT_PROXY_HTTPS_UI_PORT, GIT_PROXY_COOKIE_SECRET, GIT_PROXY_MONGO_CONNECTION_STRING = 'mongodb://localhost:27017/git-proxy', + GIT_PROXY_POSTGRES_CONNECTION_STRING, } = process.env; export const serverConfig: ServerConfig = { @@ -39,4 +40,5 @@ export const serverConfig: ServerConfig = { GIT_PROXY_HTTPS_UI_PORT, GIT_PROXY_COOKIE_SECRET, GIT_PROXY_MONGO_CONNECTION_STRING, + GIT_PROXY_POSTGRES_CONNECTION_STRING, }; diff --git a/src/config/generated/config.ts b/src/config/generated/config.ts index 0bc0baf6a..d2c4dc469 100644 --- a/src/config/generated/config.ts +++ b/src/config/generated/config.ts @@ -538,11 +538,18 @@ export interface RateLimit { * or broken out in the options object * * Connection properties for an neDB file-based database + * + * Connection properties for PostgreSQL. The `connectionString` may also be supplied via the + * `GIT_PROXY_POSTGRES_CONNECTION_STRING` environment variable. */ export interface Database { /** * mongoDB Client connection string, see * [https://www.mongodb.com/docs/manual/reference/connection-string/](https://www.mongodb.com/docs/manual/reference/connection-string/) + * + * PostgreSQL client connection string, see + * [https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING](https://www.postgresql.org/docs/current/libpq-connect.html#LIBPQ-CONNSTRING). + * If omitted, `GIT_PROXY_POSTGRES_CONNECTION_STRING` is used as a fallback. */ connectionString?: string; enabled: boolean; @@ -580,6 +587,7 @@ export interface AuthMechanismProperties { export enum DatabaseType { FS = 'fs', Mongo = 'mongo', + Postgres = 'postgres', } /** @@ -1200,6 +1208,6 @@ const typeMap: any = { false, ), AuthenticationElementType: ['ActiveDirectory', 'jwt', 'local', 'openidconnect'], - DatabaseType: ['fs', 'mongo'], + DatabaseType: ['fs', 'mongo', 'postgres'], AuthType: ['basic', 'ntlm'], }; diff --git a/src/config/index.ts b/src/config/index.ts index 6f6ac90b4..16f5ef313 100644 --- a/src/config/index.ts +++ b/src/config/index.ts @@ -329,6 +329,10 @@ export const getDatabase = () => { if (db.type === 'mongo' && !db.connectionString) { db.connectionString = serverConfig.GIT_PROXY_MONGO_CONNECTION_STRING; } + // same fallback for postgres + if (db.type === 'postgres' && !db.connectionString) { + db.connectionString = serverConfig.GIT_PROXY_POSTGRES_CONNECTION_STRING; + } return db; } } diff --git a/src/config/types.ts b/src/config/types.ts index 5cf5fd60c..d2065f4d5 100644 --- a/src/config/types.ts +++ b/src/config/types.ts @@ -24,6 +24,7 @@ export type ServerConfig = { GIT_PROXY_HTTPS_UI_PORT: string | undefined; GIT_PROXY_COOKIE_SECRET: string | undefined; GIT_PROXY_MONGO_CONNECTION_STRING: string; + GIT_PROXY_POSTGRES_CONNECTION_STRING: string | undefined; }; interface GitAuth { diff --git a/src/db/index.ts b/src/db/index.ts index bfd2f5ef7..10fb8dbf5 100644 --- a/src/db/index.ts +++ b/src/db/index.ts @@ -30,8 +30,10 @@ import * as bcrypt from 'bcryptjs'; import * as config from '../config'; import * as mongo from './mongo'; import * as neDb from './file'; +import * as postgres from './postgres'; import { Action } from '../proxy/actions/Action'; import MongoDBStore from 'connect-mongo'; +import { Store } from 'express-session'; import { CompletedAttestation, Rejection } from '../proxy/processors/types'; import { processGitUrl } from '../proxy/routes/helper'; import { initializeFolders } from './file/helper'; @@ -56,6 +58,9 @@ const start = () => { console.log('Loading neDB database adaptor'); initializeFolders(); _sink = neDb; + } else if (config.getDatabase().type === 'postgres') { + console.log('Loading PostgreSQL database adaptor'); + _sink = postgres; } else { console.error(`Unsupported database type: ${config.getDatabase().type}`); process.exit(1); @@ -197,7 +202,9 @@ export const canUserCancelPush = async (id: string, user: string) => { }; export const runMigrations = (): Promise => applyMigrations(start(), migrations); -export const getSessionStore = (): MongoDBStore | undefined => start().getSessionStore(); +export const getSessionStore = (): MongoDBStore | Store | undefined => start().getSessionStore(); +export const ensureSessionStoreReady = (): Promise => + start().ensureSessionStoreReady?.() ?? Promise.resolve(); export const getPushes = (query: Partial): Promise => start().getPushes(query); export const getPushesForUserProfile = async (user: User): Promise => { const emailVariants = collectUserProfileEmailVariants(user); diff --git a/src/db/postgres/helper.ts b/src/db/postgres/helper.ts new file mode 100644 index 000000000..5e92df2ff --- /dev/null +++ b/src/db/postgres/helper.ts @@ -0,0 +1,120 @@ +/** + * Copyright 2026 GitProxy Contributors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { Pool, QueryResult, QueryResultRow } from 'pg'; +import session, { Store } from 'express-session'; +import connectPgSimple from 'connect-pg-simple'; + +import { getDatabase } from '../../config'; +import { runMigrations } from './schemaMigrations'; + +let _pool: Pool | null = null; +let _bootstrapPromise: Promise | null = null; + +const ensurePool = (): Pool => { + if (_pool) return _pool; + + const connectionString = getDatabase().connectionString; + if (!connectionString) { + throw new Error('Postgres connection string is not provided'); + } + + _pool = new Pool({ connectionString }); + return _pool; +}; + +/** + * Lazily resolves the pg Pool and runs any pending schema migrations exactly + * once per process. All adapter modules acquire the pool through this function + * so migrations complete before any query against `users` / `repos` / `pushes` + * is executed. + */ +export const connect = async (): Promise => { + const pool = ensurePool(); + if (!_bootstrapPromise) { + _bootstrapPromise = runMigrations(pool).catch((err) => { + // Reset so the next caller retries instead of being permanently latched + // onto a rejected promise. + _bootstrapPromise = null; + throw err; + }); + } + await _bootstrapPromise; + return pool; +}; + +export const query = async ( + text: string, + params?: ReadonlyArray, +): Promise> => { + const pool = await connect(); + return pool.query(text, params as unknown[] | undefined); +}; + +/** + * Reset the pool and bootstrap latch — exported for test cleanup. + */ +export const resetConnection = async (): Promise => { + if (_pool) { + await _pool.end(); + _pool = null; + } + _bootstrapPromise = null; +}; + +/** + * Build an express-session Store backed by Postgres via `connect-pg-simple`. + * + * IMPORTANT: this function MUST NOT silently return undefined when Postgres is + * the active sink — that would cause express-session to fall back to its + * default in-memory store, which loses sessions on every restart and is unsafe + * in any multi-process deployment. Throw loudly instead. + */ +export const getSessionStore = (): Store => { + const connectionString = getDatabase().connectionString; + if (!connectionString) { + throw new Error( + 'Postgres connection string is required for session storage (set it in `sink[].connectionString` or via GIT_PROXY_POSTGRES_CONNECTION_STRING)', + ); + } + + const pool = ensurePool(); + const PgStore = connectPgSimple(session); + return new PgStore({ + pool, + tableName: 'session', + createTableIfMissing: true, + }); +}; + +export const ensureSessionStoreReady = async (): Promise => { + const store = getSessionStore(); + + await new Promise((resolve, reject) => { + store.get('__git_proxy_session_startup_probe__', (err) => { + if (err) { + reject(err); + } else { + resolve(); + } + }); + }); + + const maybeClosableStore = store as Store & { close?: () => Promise }; + if (maybeClosableStore.close) { + await maybeClosableStore.close(); + } +}; diff --git a/src/db/postgres/index.ts b/src/db/postgres/index.ts new file mode 100644 index 000000000..8c903c555 --- /dev/null +++ b/src/db/postgres/index.ts @@ -0,0 +1,67 @@ +/** + * Copyright 2026 GitProxy Contributors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import * as helper from './helper'; +import * as migrations from './migrations'; +import * as pushes from './pushes'; +import * as repo from './repo'; +import * as users from './users'; + +export const { getSessionStore, ensureSessionStoreReady } = helper; + +export const { + getPushes, + getPushesForUserProfile, + getRepoPushRollupsByCanonicalUrl, + writeAudit, + getPush, + deletePush, + authorise, + cancel, + reject, +} = pushes; + +export const { deriveCreatedAt, getAppliedMigrations, recordMigration, unrecordMigration } = + migrations; + +export const { + getRepos, + getRepo, + getRepoByUrl, + getRepoById, + createRepo, + updateRepo, + addUserCanPush, + addUserCanAuthorise, + removeUserCanPush, + removeUserCanAuthorise, + deleteRepo, +} = repo; + +export const { + findUser, + findUserByEmail, + findUserByGitAccount, + findUserByOIDC, + findUserBySSHKey, + getUsers, + createUser, + deleteUser, + updateUser, + addPublicKey, + removePublicKey, + getPublicKeys, +} = users; diff --git a/src/db/postgres/migrations.ts b/src/db/postgres/migrations.ts new file mode 100644 index 000000000..905e5d5db --- /dev/null +++ b/src/db/postgres/migrations.ts @@ -0,0 +1,37 @@ +/** + * Copyright 2026 GitProxy Contributors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { query } from './helper'; + +/** + * PostgreSQL primary keys are random UUIDs (`gen_random_uuid()`), which carry no + * embedded creation time. Like the filesystem backend, this backend cannot + * recover a timestamp from an id, so callers fall back to their own default. + */ +export const deriveCreatedAt = (): string | undefined => undefined; + +export const getAppliedMigrations = async (): Promise => { + const result = await query<{ id: string }>(`SELECT id FROM migrations`); + return result.rows.map((row) => row.id); +}; + +export const recordMigration = async (id: string): Promise => { + await query(`INSERT INTO migrations (id) VALUES ($1) ON CONFLICT (id) DO NOTHING`, [id]); +}; + +export const unrecordMigration = async (id: string): Promise => { + await query(`DELETE FROM migrations WHERE id = $1`, [id]); +}; diff --git a/src/db/postgres/pushes.ts b/src/db/postgres/pushes.ts new file mode 100644 index 000000000..48481b3a8 --- /dev/null +++ b/src/db/postgres/pushes.ts @@ -0,0 +1,282 @@ +/** + * Copyright 2026 GitProxy Contributors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { activityPrimaryStatusFromFlags } from '../../activity/activityPrimaryStatus'; +import { canonicalRemoteUrl } from '../../activity/canonicalRemoteUrl'; +import { Action } from '../../proxy/actions'; +import { CompletedAttestation, Rejection } from '../../proxy/processors/types'; +import { toClass } from '../helper'; +import { + emptyRepoActivityTabCounts, + PushQuery, + RepoActivityTabCounts, + RepoPushRollupsByCanonicalUrl, +} from '../types'; +import { query } from './helper'; + +const defaultPushQuery: Partial = { + error: false, + blocked: true, + allowPush: false, + authorised: false, + type: 'push', +}; + +// Columns that mirror Action fields used to filter `getPushes` results. +// Anything not in this map is ignored — the API only filters by these. +const FILTER_COLUMNS: Record = { + error: 'error', + blocked: 'blocked', + allowPush: 'allow_push', + authorised: 'authorised', + canceled: 'canceled', + rejected: 'rejected', + type: 'type', +}; + +const rowToAction = (row: { data: unknown }): Action => + toClass(row.data, Action.prototype) as Action; + +function bumpCount( + m: Map, + canonicalKey: string, + tab: keyof RepoActivityTabCounts, +): void { + if (!canonicalKey) { + return; + } + let row = m.get(canonicalKey); + if (!row) { + row = emptyRepoActivityTabCounts(); + m.set(canonicalKey, row); + } + row[tab] += 1; +} + +function bumpMaxTimestampMs( + m: Map, + canonicalKey: string, + timestamp: unknown, +): void { + if (!canonicalKey) { + return; + } + // `timestamp` is a BIGINT column, which node-postgres returns as a string. + // Anything else (null, undefined, empty) is not a usable timestamp. + const ts = + typeof timestamp === 'number' + ? timestamp + : typeof timestamp === 'string' && timestamp.trim() !== '' + ? Number(timestamp) + : NaN; + if (!Number.isFinite(ts)) { + return; + } + const prev = m.get(canonicalKey); + if (prev === undefined || ts > prev) { + m.set(canonicalKey, ts); + } +} + +/** + * Scan all push rows: tab counts and max timestamps per canonical remote URL + * (matches the Activity UI). The URL lives inside the `data` JSONB payload, and + * canonicalization happens in Node so the result matches the mongo and fs + * backends exactly. + */ +export const getRepoPushRollupsByCanonicalUrl = + async (): Promise => { + const result = await query<{ + url: string | null; + error: boolean; + rejected: boolean; + canceled: boolean; + authorised: boolean; + blocked: boolean; + allow_push: boolean; + timestamp: string | number | null; + }>( + `SELECT data->>'url' AS url, error, rejected, canceled, authorised, blocked, + allow_push, timestamp + FROM pushes + WHERE type = 'push'`, + ); + + const tabCounts = new Map(); + const latestPendingReviewAtMs = new Map(); + const latestPushAtMs = new Map(); + + for (const row of result.rows) { + const url = typeof row.url === 'string' ? row.url : ''; + const key = canonicalRemoteUrl(url); + if (!key) { + continue; + } + const tab = activityPrimaryStatusFromFlags({ + error: row.error === true, + rejected: row.rejected === true, + canceled: row.canceled === true, + authorised: row.authorised === true, + blocked: row.blocked === true, + allowPush: row.allow_push === true, + }); + bumpCount(tabCounts, key, tab); + bumpMaxTimestampMs(latestPushAtMs, key, row.timestamp); + if (tab === 'pending') { + bumpMaxTimestampMs(latestPendingReviewAtMs, key, row.timestamp); + } + } + + return { tabCounts, latestPendingReviewAtMs, latestPushAtMs }; + }; + +/** + * Pushes shown on a user profile: those the user made (any known email variant) + * plus those they reviewed. Mirrors `buildUserProfilePushFilter`, which the + * mongo and fs backends feed to their query engines; the reviewer match is + * case-insensitive on the exact username. + */ +export const getPushesForUserProfile = async ( + emailVariants: string[], + profileUsername: string, +): Promise => { + const reviewerClause = `lower(data->'attestation'->'reviewer'->>'username') = lower($1)`; + const values: unknown[] = [profileUsername]; + let predicate = reviewerClause; + + if (emailVariants.length > 0) { + values.push(emailVariants); + predicate = `((data->>'userEmail') = ANY($${values.length}::text[]) OR ${reviewerClause})`; + } + + const result = await query<{ data: unknown }>( + `SELECT data FROM pushes WHERE type = 'push' AND ${predicate} ORDER BY timestamp DESC`, + values, + ); + return result.rows.map(rowToAction); +}; + +export const getPushes = async (q: Partial = defaultPushQuery): Promise => { + const clauses: string[] = []; + const values: unknown[] = []; + for (const [key, value] of Object.entries(q)) { + const column = FILTER_COLUMNS[key]; + if (!column || value === undefined) continue; + values.push(value); + clauses.push(`${column} = $${values.length}`); + } + + const where = clauses.length > 0 ? `WHERE ${clauses.join(' AND ')}` : ''; + const result = await query<{ data: unknown }>( + `SELECT data FROM pushes ${where} ORDER BY timestamp DESC`, + values, + ); + return result.rows.map(rowToAction); +}; + +export const getPush = async (id: string): Promise => { + const result = await query<{ data: unknown }>(`SELECT data FROM pushes WHERE id = $1`, [id]); + if (result.rowCount === 0) return null; + return rowToAction(result.rows[0]); +}; + +export const deletePush = async (id: string): Promise => { + await query(`DELETE FROM pushes WHERE id = $1`, [id]); +}; + +export const writeAudit = async (action: Action): Promise => { + if (typeof action.id !== 'string') { + throw new Error('Invalid id'); + } + + // Round-trip through JSON to drop class identity / mongo-specific _id fields + // before persisting (mirrors mongo's `JSON.parse(JSON.stringify(action))`). + const data = JSON.parse(JSON.stringify(action)); + delete data._id; + + await query( + `INSERT INTO pushes ( + id, timestamp, type, error, blocked, allow_push, + authorised, canceled, rejected, data + ) + VALUES ($1, $2, $3, $4, $5, $6, $7, $8, $9, $10::jsonb) + ON CONFLICT (id) DO UPDATE SET + timestamp = EXCLUDED.timestamp, + type = EXCLUDED.type, + error = EXCLUDED.error, + blocked = EXCLUDED.blocked, + allow_push = EXCLUDED.allow_push, + authorised = EXCLUDED.authorised, + canceled = EXCLUDED.canceled, + rejected = EXCLUDED.rejected, + data = EXCLUDED.data`, + [ + action.id, + action.timestamp ?? Date.now(), + action.type ?? null, + action.error ?? false, + action.blocked ?? false, + action.allowPush ?? false, + action.authorised ?? false, + action.canceled ?? false, + action.rejected ?? false, + JSON.stringify(data), + ], + ); +}; + +export const authorise = async ( + id: string, + attestation?: CompletedAttestation, +): Promise<{ message: string }> => { + const action = await getPush(id); + if (!action) { + throw new Error(`push ${id} not found`); + } + action.authorised = true; + action.canceled = false; + action.rejected = false; + action.attestation = attestation; + await writeAudit(action); + return { message: `authorised ${id}` }; +}; + +export const reject = async (id: string, rejection: Rejection): Promise<{ message: string }> => { + const action = await getPush(id); + if (!action) { + throw new Error(`push ${id} not found`); + } + action.authorised = false; + action.canceled = false; + action.rejected = true; + // Preserve the existing rejection-payload shape used by the fs/mongo + // backends — the issue calls this out explicitly as a must-fix. + action.rejection = rejection; + await writeAudit(action); + return { message: `reject ${id}` }; +}; + +export const cancel = async (id: string): Promise<{ message: string }> => { + const action = await getPush(id); + if (!action) { + throw new Error(`push ${id} not found`); + } + action.authorised = false; + action.canceled = true; + action.rejected = false; + await writeAudit(action); + return { message: `canceled ${id}` }; +}; diff --git a/src/db/postgres/repo.ts b/src/db/postgres/repo.ts new file mode 100644 index 000000000..93f412212 --- /dev/null +++ b/src/db/postgres/repo.ts @@ -0,0 +1,228 @@ +/** + * Copyright 2026 GitProxy Contributors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { Repo, RepoQuery } from '../types'; +import { query } from './helper'; + +interface RepoRow { + _id: string; + project: string; + name: string; + url: string; + users: { canPush: string[]; canAuthorise: string[] } | null; + date_created: string | null; + last_modified: string | null; +} + +const rowToRepo = (row: RepoRow): Repo => + new Repo( + row.project, + row.name, + row.url, + // Guard against null/legacy rows so callers always see arrays. + { + canPush: row.users?.canPush ?? [], + canAuthorise: row.users?.canAuthorise ?? [], + }, + row._id, + row.date_created ?? undefined, + row.last_modified ?? undefined, + ); + +const SELECT_COLUMNS = '_id, project, name, url, users, date_created, last_modified'; + +export const getRepos = async (q: Partial = {}): Promise => { + const clauses: string[] = []; + const values: unknown[] = []; + if (q.name) { + values.push(q.name.toLowerCase()); + clauses.push(`name = $${values.length}`); + } + if (q.project !== undefined) { + values.push(q.project); + clauses.push(`project = $${values.length}`); + } + if (q.url) { + values.push(q.url); + clauses.push(`url = $${values.length}`); + } + + const where = clauses.length > 0 ? `WHERE ${clauses.join(' AND ')}` : ''; + const result = await query(`SELECT ${SELECT_COLUMNS} FROM repos ${where}`, values); + return result.rows.map(rowToRepo); +}; + +export const getRepo = async (name: string): Promise => { + const result = await query(`SELECT ${SELECT_COLUMNS} FROM repos WHERE name = $1`, [ + name.toLowerCase(), + ]); + return result.rowCount === 0 ? null : rowToRepo(result.rows[0]); +}; + +export const getRepoByUrl = async (url: string): Promise => { + const result = await query(`SELECT ${SELECT_COLUMNS} FROM repos WHERE url = $1`, [url]); + return result.rowCount === 0 ? null : rowToRepo(result.rows[0]); +}; + +export const getRepoById = async (_id: string): Promise => { + const result = await query(`SELECT ${SELECT_COLUMNS} FROM repos WHERE _id = $1`, [_id]); + return result.rowCount === 0 ? null : rowToRepo(result.rows[0]); +}; + +export const createRepo = async (repo: Repo): Promise => { + const users = repo.users ?? { canPush: [], canAuthorise: [] }; + const now = new Date().toISOString(); + if (!repo.dateCreated) repo.dateCreated = now; + if (!repo.lastModified) repo.lastModified = now; + const result = await query<{ _id: string }>( + `INSERT INTO repos (project, name, url, users, date_created, last_modified) + VALUES ($1, $2, $3, $4::jsonb, $5, $6) + RETURNING _id`, + [ + repo.project ?? '', + repo.name, + repo.url, + JSON.stringify(users), + repo.dateCreated, + repo.lastModified, + ], + ); + repo._id = result.rows[0]._id; + repo.users = users; + return repo; +}; + +/** + * Apply a partial update to a repo row. Only the supplied fields are written, + * matching mongo's `$set` / `$unset` behaviour: a field explicitly set to + * `undefined` is reset to the column default rather than left untouched. + */ +export const updateRepo = async (repo: Partial): Promise => { + const { _id, ...fields } = repo; + if (!_id) { + throw new Error('updateRepo requires a repo _id'); + } + + const COLUMNS: Record = { + project: 'project', + name: 'name', + url: 'url', + users: 'users', + dateCreated: 'date_created', + lastModified: 'last_modified', + }; + + const sets: string[] = []; + const values: unknown[] = []; + for (const [key, value] of Object.entries(fields)) { + const column = COLUMNS[key]; + if (!column) continue; + if (value === undefined) { + sets.push(`${column} = DEFAULT`); + continue; + } + if (column === 'users') { + values.push(JSON.stringify(value)); + sets.push(`${column} = $${values.length}::jsonb`); + continue; + } + values.push(value); + sets.push(`${column} = $${values.length}`); + } + + if (sets.length === 0) { + throw new Error('updateRepo requires at least one field to update'); + } + + values.push(_id); + await query(`UPDATE repos SET ${sets.join(', ')} WHERE _id = $${values.length}`, values); +}; + +/** + * Append a user to one of the JSONB permission arrays. The query is a + * read-modify-write that deduplicates the value, then re-serialises the array + * so the stored shape matches the existing mongo/fs backends exactly. + */ +const addUserToRole = async ( + _id: string, + user: string, + role: 'canPush' | 'canAuthorise', +): Promise => { + const lowered = user.toLowerCase(); + await query( + `UPDATE repos + SET users = jsonb_set( + users, + $2::text[], + ( + SELECT to_jsonb( + ARRAY( + SELECT DISTINCT v + FROM jsonb_array_elements_text(coalesce(users->$3, '[]'::jsonb)) AS v + UNION + SELECT $4 + ) + ) + ) + ), + last_modified = $5 + WHERE _id = $1`, + [_id, `{${role}}`, role, lowered, new Date().toISOString()], + ); +}; + +const removeUserFromRole = async ( + _id: string, + user: string, + role: 'canPush' | 'canAuthorise', +): Promise => { + const lowered = user.toLowerCase(); + // The filter evaluates to `[]` if the last matching user is removed + await query( + `UPDATE repos + SET users = jsonb_set( + users, + $2::text[], + coalesce( + ( + SELECT to_jsonb(array_agg(v)) + FROM jsonb_array_elements_text(coalesce(users->$3, '[]'::jsonb)) AS v + WHERE v <> $4 + ), + '[]'::jsonb + ) + ), + last_modified = $5 + WHERE _id = $1`, + [_id, `{${role}}`, role, lowered, new Date().toISOString()], + ); +}; + +export const addUserCanPush = (_id: string, user: string): Promise => + addUserToRole(_id, user, 'canPush'); + +export const addUserCanAuthorise = (_id: string, user: string): Promise => + addUserToRole(_id, user, 'canAuthorise'); + +export const removeUserCanPush = (_id: string, user: string): Promise => + removeUserFromRole(_id, user, 'canPush'); + +export const removeUserCanAuthorise = (_id: string, user: string): Promise => + removeUserFromRole(_id, user, 'canAuthorise'); + +export const deleteRepo = async (_id: string): Promise => { + await query(`DELETE FROM repos WHERE _id = $1`, [_id]); +}; diff --git a/src/db/postgres/schemaMigrations.ts b/src/db/postgres/schemaMigrations.ts new file mode 100644 index 000000000..c92a8385e --- /dev/null +++ b/src/db/postgres/schemaMigrations.ts @@ -0,0 +1,173 @@ +/** + * Copyright 2026 GitProxy Contributors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { Pool } from 'pg'; + +/** + * A single, immutable schema change. Append new migrations with the next + * `version`; never edit or reorder entries that have already shipped, since + * deployed databases record which versions they have applied. + */ +export interface Migration { + version: number; + name: string; + sql: string; +} + +/** + * Ordered, append-only list of schema migrations. + * + * Version 1 is the initial schema. Because every statement uses + * `CREATE TABLE/INDEX IF NOT EXISTS`, databases that were already bootstrapped + * by the pre-migration code adopt the runner transparently: the statements are + * no-ops and version 1 is simply recorded as applied. + */ +export const MIGRATIONS: Migration[] = [ + { + version: 1, + name: 'initial_schema', + sql: ` + CREATE TABLE IF NOT EXISTS users ( + _id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + username TEXT NOT NULL UNIQUE, + email TEXT NOT NULL UNIQUE, + password TEXT, + git_account TEXT NOT NULL, + admin BOOLEAN NOT NULL DEFAULT FALSE, + oidc_id TEXT UNIQUE, + display_name TEXT, + title TEXT + ); + + CREATE TABLE IF NOT EXISTS repos ( + _id UUID PRIMARY KEY DEFAULT gen_random_uuid(), + project TEXT NOT NULL DEFAULT '', + name TEXT NOT NULL, + url TEXT NOT NULL UNIQUE, + users JSONB NOT NULL DEFAULT '{"canPush":[],"canAuthorise":[]}'::jsonb, + date_created TEXT, + last_modified TEXT + ); + ALTER TABLE repos ADD COLUMN IF NOT EXISTS date_created TEXT; + ALTER TABLE repos ADD COLUMN IF NOT EXISTS last_modified TEXT; + CREATE INDEX IF NOT EXISTS repos_name_idx ON repos (name); + + CREATE TABLE IF NOT EXISTS pushes ( + id TEXT PRIMARY KEY, + timestamp BIGINT NOT NULL, + type TEXT, + error BOOLEAN NOT NULL DEFAULT FALSE, + blocked BOOLEAN NOT NULL DEFAULT FALSE, + allow_push BOOLEAN NOT NULL DEFAULT FALSE, + authorised BOOLEAN NOT NULL DEFAULT FALSE, + canceled BOOLEAN NOT NULL DEFAULT FALSE, + rejected BOOLEAN NOT NULL DEFAULT FALSE, + data JSONB NOT NULL + ); + CREATE INDEX IF NOT EXISTS pushes_timestamp_idx ON pushes (timestamp DESC); +`, + }, + { + version: 2, + name: 'user_public_keys_and_optional_email', + sql: ` + ALTER TABLE users ADD COLUMN IF NOT EXISTS public_keys JSONB NOT NULL DEFAULT '[]'::jsonb; + ALTER TABLE users ALTER COLUMN email DROP NOT NULL; + ALTER TABLE users DROP CONSTRAINT IF EXISTS users_email_key; + -- Email uniqueness is best-effort, like the mongo/fs backends: a real + -- address can only be claimed once, but any number of users may have no + -- email (the AD "mail" attribute is optional, for instance). + CREATE UNIQUE INDEX IF NOT EXISTS users_email_unique + ON users (email) WHERE email IS NOT NULL AND email <> ''; +`, + }, + { + version: 3, + name: 'migration_bookkeeping', + sql: ` + -- Bookkeeping for the cross-backend migration framework in src/db/migrations. + -- That framework records logical migrations by string id through the Sink + -- hooks; this table is its postgres storage, and is separate from the + -- schema_migrations table that versions the DDL below. + CREATE TABLE IF NOT EXISTS migrations ( + id TEXT PRIMARY KEY + ); +`, + }, +]; + +const SCHEMA_MIGRATIONS_TABLE_SQL = ` + CREATE TABLE IF NOT EXISTS schema_migrations ( + version INTEGER PRIMARY KEY, + name TEXT NOT NULL, + applied_at TIMESTAMPTZ NOT NULL DEFAULT now() + ); +`; + +// Fixed, arbitrary advisory-lock key. Serialises migration runs across +// concurrently starting processes so each migration is applied exactly once. +const MIGRATION_ADVISORY_LOCK_KEY = 4815162342; + +/** + * Apply any not-yet-applied migrations in version order, inside a single + * transaction guarded by a transaction-scoped advisory lock. + * + * Safe to call on every process start: already-applied migrations are skipped, + * and concurrent callers block on the lock rather than racing. The lock is + * acquired before the `schema_migrations` table is touched so two processes + * booting against a brand-new database cannot both seed version 1. + * + * NOTE: all pending migrations run in one transaction, so a future statement + * that cannot run transactionally (e.g. `CREATE INDEX CONCURRENTLY`) will need + * dedicated handling — not required for the current schema. + */ +export const runMigrations = async (pool: Pool): Promise => { + const client = await pool.connect(); + try { + await client.query('BEGIN'); + // Transaction-scoped lock; auto-released on COMMIT/ROLLBACK. + await client.query('SELECT pg_advisory_xact_lock($1)', [MIGRATION_ADVISORY_LOCK_KEY]); + await client.query(SCHEMA_MIGRATIONS_TABLE_SQL); + + const { rows } = await client.query<{ version: number }>( + 'SELECT version FROM schema_migrations', + ); + const applied = new Set(rows.map((row) => row.version)); + + const pending = [...MIGRATIONS] + .sort((a, b) => a.version - b.version) + .filter((migration) => !applied.has(migration.version)); + + for (const migration of pending) { + await client.query(migration.sql); + await client.query('INSERT INTO schema_migrations (version, name) VALUES ($1, $2)', [ + migration.version, + migration.name, + ]); + } + + await client.query('COMMIT'); + } catch (err) { + try { + await client.query('ROLLBACK'); + } catch { + // best-effort rollback; the original error is rethrown below + } + throw err; + } finally { + client.release(); + } +}; diff --git a/src/db/postgres/users.ts b/src/db/postgres/users.ts new file mode 100644 index 000000000..59225d1a9 --- /dev/null +++ b/src/db/postgres/users.ts @@ -0,0 +1,255 @@ +/** + * Copyright 2026 GitProxy Contributors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { PublicKeyRecord, User, UserQuery } from '../types'; +import { DuplicateSSHKeyError } from '../../errors/DatabaseErrors'; +import { query } from './helper'; + +interface UserRow { + _id: string; + username: string; + email: string | null; + password: string | null; + git_account: string; + admin: boolean; + oidc_id: string | null; + public_keys: PublicKeyRecord[] | null; + display_name: string | null; + title: string | null; +} + +const rowToUser = (row: UserRow): User => { + const user = new User( + row.username, + row.password ?? '', + row.git_account, + row.email ?? '', + row.admin, + row.oidc_id, + row.public_keys ?? [], + row._id, + ); + user.password = row.password; + user.displayName = row.display_name; + user.title = row.title; + return user; +}; + +const SELECT_COLUMNS = + '_id, username, email, password, git_account, admin, oidc_id, public_keys, display_name, title'; + +export const findUser = async (username: string): Promise => { + const result = await query(`SELECT ${SELECT_COLUMNS} FROM users WHERE username = $1`, [ + username.toLowerCase(), + ]); + return result.rowCount === 0 ? null : rowToUser(result.rows[0]); +}; + +export const findUserByEmail = async (email: string): Promise => { + const result = await query(`SELECT ${SELECT_COLUMNS} FROM users WHERE email = $1`, [ + email.toLowerCase(), + ]); + return result.rowCount === 0 ? null : rowToUser(result.rows[0]); +}; + +export const findUserByGitAccount = async (gitAccount: string): Promise => { + const result = await query( + `SELECT ${SELECT_COLUMNS} FROM users WHERE git_account = $1`, + [gitAccount.toLowerCase()], + ); + return result.rowCount === 0 ? null : rowToUser(result.rows[0]); +}; + +export const findUserByOIDC = async (oidcId: string): Promise => { + const result = await query(`SELECT ${SELECT_COLUMNS} FROM users WHERE oidc_id = $1`, [ + oidcId, + ]); + return result.rowCount === 0 ? null : rowToUser(result.rows[0]); +}; + +export const getUsers = async (q: Partial = {}): Promise => { + const clauses: string[] = []; + const values: unknown[] = []; + if (q.username) { + values.push(q.username.toLowerCase()); + clauses.push(`username = $${values.length}`); + } + if (q.email) { + values.push(q.email.toLowerCase()); + clauses.push(`email = $${values.length}`); + } + + const where = clauses.length > 0 ? `WHERE ${clauses.join(' AND ')}` : ''; + // Match mongo's `.project({ password: 0 })` — omit password from list results. + const result = await query( + `SELECT _id, username, email, NULL::text AS password, git_account, admin, oidc_id, public_keys, display_name, title + FROM users ${where}`, + values, + ); + return result.rows.map(rowToUser); +}; + +export const createUser = async (user: User): Promise => { + await query( + `INSERT INTO users (username, email, password, git_account, admin, oidc_id, public_keys, display_name, title) + VALUES ($1, $2, $3, $4, $5, $6, $7::jsonb, $8, $9)`, + [ + user.username.toLowerCase(), + user.email.toLowerCase(), + user.password ?? null, + user.gitAccount, + user.admin, + user.oidcId ?? null, + JSON.stringify(user.publicKeys ?? []), + user.displayName ?? null, + user.title ?? null, + ], + ); +}; + +export const deleteUser = async (username: string): Promise => { + await query(`DELETE FROM users WHERE username = $1`, [username.toLowerCase()]); +}; + +/** + * Update an existing user, or insert a new one if no matching row exists. + * + * Mirrors the mongo adapter's upsert semantics: partial updates are merged + * onto an existing row (only supplied fields are written), and a missing row + * is created. Identity is by `_id` when provided, otherwise by `username`. + */ +export const updateUser = async (user: Partial): Promise => { + const username = user.username?.toLowerCase(); + const email = user.email?.toLowerCase(); + + // Track the supplied columns so both branches only ever write the fields + // the caller patched. + const columns: string[] = []; + const values: unknown[] = []; + const set = (column: string, value: unknown) => { + columns.push(column); + values.push(value); + }; + + if (username !== undefined) set('username', username); + if (email !== undefined) set('email', email); + if (user.password !== undefined) set('password', user.password); + if (user.gitAccount !== undefined) set('git_account', user.gitAccount); + if (user.admin !== undefined) set('admin', user.admin); + if (user.oidcId !== undefined) set('oidc_id', user.oidcId); + if (user.publicKeys !== undefined) set('public_keys', JSON.stringify(user.publicKeys)); + if (user.displayName !== undefined) set('display_name', user.displayName); + if (user.title !== undefined) set('title', user.title); + + // An empty SET list would be a SQL syntax error, so fail loudly rather than + // let callers (or future handlers copying this builder) hit that. + if (columns.length === 0) { + throw new Error('updateUser requires at least one field to update'); + } + + if (user._id) { + const sets = columns.map((column, i) => `${column} = $${i + 1}`); + values.push(user._id); + await query(`UPDATE users SET ${sets.join(', ')} WHERE _id = $${values.length}`, values); + return; + } + + if (!username) { + throw new Error('updateUser requires either _id or username'); + } + + // Upsert by username when no _id is supplied, matching mongo's behaviour. + // A single atomic statement (rather than UPDATE-then-INSERT) so a + // concurrent insert of the same username can't drop the update; on + // conflict only the supplied fields are merged onto the existing row. + const assignments = columns.map((column) => `${column} = EXCLUDED.${column}`); + await query( + `INSERT INTO users (username, email, password, git_account, admin, oidc_id, public_keys, display_name, title) + VALUES ($1, $2, $3, $4, $5, $6, $7::jsonb, $8, $9) + ON CONFLICT (username) DO UPDATE SET ${assignments.join(', ')}`, + [ + username, + email ?? null, + user.password ?? null, + user.gitAccount ?? '', + user.admin ?? false, + user.oidcId ?? null, + JSON.stringify(user.publicKeys ?? []), + user.displayName ?? null, + user.title ?? null, + ], + ); +}; + +export const findUserBySSHKey = async (sshKey: string): Promise => { + // JSONB containment: matches any element of public_keys with this exact key, + // equivalent to mongo's `{ 'publicKeys.key': sshKey }`. + const result = await query( + `SELECT ${SELECT_COLUMNS} FROM users WHERE public_keys @> $1::jsonb`, + [JSON.stringify([{ key: sshKey }])], + ); + return result.rowCount === 0 ? null : rowToUser(result.rows[0]); +}; + +export const addPublicKey = async (username: string, publicKey: PublicKeyRecord): Promise => { + const existingUser = await findUserBySSHKey(publicKey.key); + if (existingUser && existingUser.username.toLowerCase() !== username.toLowerCase()) { + throw new DuplicateSSHKeyError(existingUser.username); + } + + const user = await findUser(username); + if (!user) { + throw new Error('User not found'); + } + + const keyExists = user.publicKeys?.some( + (k) => k.key === publicKey.key || (k.fingerprint && k.fingerprint === publicKey.fingerprint), + ); + if (keyExists) { + throw new Error('SSH key already exists'); + } + + await query(`UPDATE users SET public_keys = public_keys || $2::jsonb WHERE username = $1`, [ + username.toLowerCase(), + JSON.stringify([publicKey]), + ]); +}; + +export const removePublicKey = async (username: string, fingerprint: string): Promise => { + // Filter the matching key out of the JSONB array; like mongo's `$pull`, this + // is a no-op when the user or fingerprint does not exist. + await query( + `UPDATE users + SET public_keys = coalesce( + ( + SELECT jsonb_agg(k) + FROM jsonb_array_elements(public_keys) AS k + WHERE (k->>'fingerprint') IS DISTINCT FROM $2 + ), + '[]'::jsonb + ) + WHERE username = $1`, + [username.toLowerCase(), fingerprint], + ); +}; + +export const getPublicKeys = async (username: string): Promise => { + const user = await findUser(username); + if (!user) { + throw new Error('User not found'); + } + return user.publicKeys || []; +}; diff --git a/src/db/types.ts b/src/db/types.ts index 10f35ba5d..99f185c9c 100644 --- a/src/db/types.ts +++ b/src/db/types.ts @@ -16,6 +16,7 @@ import { Action } from '../proxy/actions/Action'; import MongoDBStore from 'connect-mongo'; +import { Store } from 'express-session'; import { CompletedAttestation, Rejection } from '../proxy/processors/types'; export type PushQuery = { @@ -169,7 +170,8 @@ export interface PublicUser { } export interface Sink { - getSessionStore: () => MongoDBStore | undefined; + getSessionStore: () => MongoDBStore | Store | undefined; + ensureSessionStoreReady?: () => Promise; getRepoPushRollupsByCanonicalUrl: () => Promise; getPushes: (query: Partial) => Promise; getPushesForUserProfile: (emailVariants: string[], profileUsername: string) => Promise; diff --git a/src/service/index.ts b/src/service/index.ts index 831a1c0fb..893ceaf1e 100644 --- a/src/service/index.ts +++ b/src/service/index.ts @@ -134,6 +134,12 @@ const corsOptions: cors.CorsOptions = { * @param {Proxy} proxy A reference to the proxy, used to restart it when necessary. * @return {Promise} the express application */ +// Backend sink types that promise a persistent session store. If one of these +// is active and getSessionStore() returns undefined, express-session would +// silently fall back to MemoryStore — which loses sessions on restart and is +// unsafe in any multi-process deployment. Throw loudly instead. +const PERSISTENT_SESSION_BACKENDS = new Set(['mongo', 'postgres']); + async function createApp(proxy: Proxy): Promise { // configuration of passport is async // Before we can bind the routes - we need the passport strategy @@ -143,9 +149,20 @@ async function createApp(proxy: Proxy): Promise { app.set('trust proxy', 1); app.use(limiter); + const backendType = config.getDatabase().type; + if (PERSISTENT_SESSION_BACKENDS.has(backendType)) { + await db.ensureSessionStoreReady(); + } + const sessionStore = db.getSessionStore(); + if (PERSISTENT_SESSION_BACKENDS.has(backendType) && !sessionStore) { + throw new Error( + `Session store for backend "${backendType}" failed to initialize — refusing to fall back to MemoryStore`, + ); + } + app.use( session({ - store: db.getSessionStore(), + store: sessionStore, secret: config.getCookieSecret(), resave: false, saveUninitialized: false, diff --git a/test-integration.postgres.proxy.config.json b/test-integration.postgres.proxy.config.json new file mode 100644 index 000000000..3885d004f --- /dev/null +++ b/test-integration.postgres.proxy.config.json @@ -0,0 +1,20 @@ +{ + "cookieSecret": "integration-test-cookie-secret", + "sessionMaxAgeHours": 12, + "sink": [ + { + "type": "fs", + "enabled": false + }, + { + "type": "postgres", + "enabled": true + } + ], + "authentication": [ + { + "type": "local", + "enabled": true + } + ] +} diff --git a/test/db/postgres/helper.test.ts b/test/db/postgres/helper.test.ts new file mode 100644 index 000000000..a8061698f --- /dev/null +++ b/test/db/postgres/helper.test.ts @@ -0,0 +1,170 @@ +/** + * Copyright 2026 GitProxy Contributors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { describe, it, expect, beforeEach, vi } from 'vitest'; + +const mockPoolQuery = vi.fn(); +const mockPoolEnd = vi.fn(); +const mockPoolCtor = vi.fn(); +const mockPoolConnect = vi.fn(); +const mockClientQuery = vi.fn(); +const mockClientRelease = vi.fn(); + +vi.mock('pg', () => { + class Pool { + constructor(opts: unknown) { + mockPoolCtor(opts); + } + query = mockPoolQuery; + end = mockPoolEnd; + connect = mockPoolConnect; + } + return { Pool }; +}); + +// connect-pg-simple returns a constructor that accepts options including a +// `pool` instance. We don't exercise the real store — just want to capture the +// options the helper passes. +const mockStoreCtor = vi.fn(); +vi.mock('connect-pg-simple', () => ({ + default: () => + class FakePgStore { + constructor(opts: unknown) { + mockStoreCtor(opts); + } + get(_sid: string, cb: (err: Error | null) => void) { + mockPoolQuery('SELECT 1', []); + cb(null); + } + close() { + return Promise.resolve(); + } + }, +})); + +const getDatabaseMock = vi.fn(); +vi.mock('../../../src/config', () => ({ + getDatabase: getDatabaseMock, +})); + +describe('PostgreSQL - helper', async () => { + const { connect, query, resetConnection, getSessionStore, ensureSessionStoreReady } = + await import('../../../src/db/postgres/helper'); + + beforeEach(async () => { + vi.clearAllMocks(); + await resetConnection(); + mockPoolQuery.mockResolvedValue({ rowCount: 0, rows: [] }); + mockClientQuery.mockResolvedValue({ rowCount: 0, rows: [] }); + mockPoolConnect.mockResolvedValue({ query: mockClientQuery, release: mockClientRelease }); + }); + + describe('connect / migrations', () => { + it('runs migrations exactly once across many concurrent connects', async () => { + getDatabaseMock.mockReturnValue({ + type: 'postgres', + enabled: true, + connectionString: 'postgresql://localhost/x', + }); + + await Promise.all([connect(), connect(), connect()]); + + // Pool constructed once; a single client acquired to run migrations once. + expect(mockPoolCtor).toHaveBeenCalledTimes(1); + expect(mockPoolConnect).toHaveBeenCalledTimes(1); + + const sqls = mockClientQuery.mock.calls.map((call) => String(call[0])); + expect(sqls[0]).toBe('BEGIN'); + expect(sqls.some((sql) => /pg_advisory_xact_lock/.test(sql))).toBe(true); + expect(sqls.some((sql) => /CREATE TABLE IF NOT EXISTS schema_migrations/.test(sql))).toBe( + true, + ); + expect(sqls.some((sql) => /CREATE TABLE IF NOT EXISTS users/.test(sql))).toBe(true); + expect(sqls[sqls.length - 1]).toBe('COMMIT'); + }); + + it('retries migrations on the next call if they failed', async () => { + getDatabaseMock.mockReturnValue({ + type: 'postgres', + enabled: true, + connectionString: 'postgresql://localhost/x', + }); + + // First migration run rejects on its opening statement. + mockClientQuery.mockRejectedValueOnce(new Error('schema kaboom')); + + await expect(connect()).rejects.toThrow('schema kaboom'); + + // The latch is cleared on failure, so the next connect re-runs migrations + // rather than being permanently latched to the rejected promise. + await connect(); + expect(mockPoolConnect).toHaveBeenCalledTimes(2); + }); + + it('throws when the connection string is missing', async () => { + getDatabaseMock.mockReturnValue({ + type: 'postgres', + enabled: true, + connectionString: undefined, + }); + + await expect(query('SELECT 1')).rejects.toThrow('Postgres connection string is not provided'); + }); + }); + + describe('getSessionStore', () => { + it('throws when connection string is missing — no MemoryStore fallback', () => { + getDatabaseMock.mockReturnValue({ + type: 'postgres', + enabled: true, + connectionString: undefined, + }); + + expect(() => getSessionStore()).toThrow( + /Postgres connection string is required for session storage/, + ); + }); + + it('passes the shared pool to connect-pg-simple with createTableIfMissing', () => { + getDatabaseMock.mockReturnValue({ + type: 'postgres', + enabled: true, + connectionString: 'postgresql://localhost/x', + }); + + getSessionStore(); + + expect(mockStoreCtor).toHaveBeenCalledTimes(1); + const opts = mockStoreCtor.mock.calls[0][0] as Record; + expect(opts.tableName).toBe('session'); + expect(opts.createTableIfMissing).toBe(true); + expect(opts.pool).toBeDefined(); + }); + + it('touches the session store during readiness checks', async () => { + getDatabaseMock.mockReturnValue({ + type: 'postgres', + enabled: true, + connectionString: 'postgresql://localhost/x', + }); + + await ensureSessionStoreReady(); + + expect(mockStoreCtor).toHaveBeenCalledTimes(1); + expect(mockPoolQuery).toHaveBeenCalled(); + }); + }); +}); diff --git a/test/db/postgres/migrations.test.ts b/test/db/postgres/migrations.test.ts new file mode 100644 index 000000000..7d40a952a --- /dev/null +++ b/test/db/postgres/migrations.test.ts @@ -0,0 +1,78 @@ +/** + * Copyright 2026 GitProxy Contributors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { describe, it, expect, beforeEach, vi } from 'vitest'; + +const mockQuery = vi.fn(); + +vi.mock('../../../src/db/postgres/helper', () => ({ + query: mockQuery, +})); + +describe('PostgreSQL - Migrations', async () => { + const { deriveCreatedAt, getAppliedMigrations, recordMigration, unrecordMigration } = + await import('../../../src/db/postgres/migrations'); + + beforeEach(() => { + vi.clearAllMocks(); + }); + + describe('deriveCreatedAt', () => { + it('cannot recover a timestamp from a random UUID', () => { + // Same contract as the filesystem backend: callers fall back to their own default. + expect(deriveCreatedAt()).toBeUndefined(); + }); + }); + + describe('getAppliedMigrations', () => { + it('returns the recorded ids', async () => { + mockQuery.mockResolvedValue({ rowCount: 2, rows: [{ id: '001-a' }, { id: '002-b' }] }); + + await expect(getAppliedMigrations()).resolves.toEqual(['001-a', '002-b']); + }); + + it('returns an empty list on a fresh database', async () => { + mockQuery.mockResolvedValue({ rowCount: 0, rows: [] }); + + await expect(getAppliedMigrations()).resolves.toEqual([]); + }); + }); + + describe('recordMigration', () => { + it('is idempotent so an interrupted run can resume', async () => { + mockQuery.mockResolvedValue({ rowCount: 1, rows: [] }); + + await recordMigration('001-a'); + + const [sql, params] = mockQuery.mock.calls[0]; + expect(sql).toContain('INSERT INTO migrations'); + expect(sql).toContain('ON CONFLICT (id) DO NOTHING'); + expect(params).toEqual(['001-a']); + }); + }); + + describe('unrecordMigration', () => { + it('deletes only the given id', async () => { + mockQuery.mockResolvedValue({ rowCount: 1, rows: [] }); + + await unrecordMigration('001-a'); + + const [sql, params] = mockQuery.mock.calls[0]; + expect(sql).toContain('DELETE FROM migrations WHERE id = $1'); + expect(params).toEqual(['001-a']); + }); + }); +}); diff --git a/test/db/postgres/pushes.integration.test.ts b/test/db/postgres/pushes.integration.test.ts new file mode 100644 index 000000000..e3b5133f1 --- /dev/null +++ b/test/db/postgres/pushes.integration.test.ts @@ -0,0 +1,269 @@ +/** + * Copyright 2026 GitProxy Contributors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { describe, it, expect, beforeEach } from 'vitest'; +import { + writeAudit, + getPush, + getPushes, + deletePush, + authorise, + reject, + cancel, +} from '../../../src/db/postgres/pushes'; +import { Action } from '../../../src/proxy/actions'; + +const shouldRunPostgresTests = process.env.RUN_POSTGRES_TESTS === 'true'; + +describe.runIf(shouldRunPostgresTests)('PostgreSQL Pushes Integration Tests', () => { + const createTestAction = (overrides: Partial = {}): Action => { + const timestamp = Date.now(); + const action = new Action( + overrides.id || `test-push-${timestamp}`, + overrides.type || 'push', + overrides.method || 'POST', + overrides.timestamp || timestamp, + overrides.url || 'https://github.com/test/repo.git', + ); + + action.error = overrides.error ?? false; + action.blocked = overrides.blocked ?? true; + action.allowPush = overrides.allowPush ?? false; + action.authorised = overrides.authorised ?? false; + action.canceled = overrides.canceled ?? false; + action.rejected = overrides.rejected ?? false; + + return action; + }; + + describe('writeAudit', () => { + it('writes an action to the database', async () => { + const action = createTestAction({ id: 'write-audit-test' }); + await writeAudit(action); + + const retrieved = await getPush('write-audit-test'); + expect(retrieved).not.toBeNull(); + expect(retrieved?.id).toBe('write-audit-test'); + }); + + it('upserts an existing action', async () => { + const action = createTestAction({ id: 'upsert-test' }); + await writeAudit(action); + + action.blocked = false; + action.allowPush = true; + await writeAudit(action); + + const retrieved = await getPush('upsert-test'); + expect(retrieved?.blocked).toBe(false); + expect(retrieved?.allowPush).toBe(true); + }); + + it('throws Invalid id for non-string ids', async () => { + const action = createTestAction(); + action.id = 123 as unknown as string; + + await expect(writeAudit(action)).rejects.toThrow('Invalid id'); + }); + + it('strips _id from action before saving', async () => { + const action = createTestAction({ id: 'strip-id-test' }); + (action as any)._id = 'should-be-removed'; + + await writeAudit(action); + const retrieved = await getPush('strip-id-test'); + expect(retrieved).not.toBeNull(); + // _id should not leak back out — the action JSON contains only public fields + expect((retrieved as any)._id).toBeUndefined(); + expect(retrieved?.id).toBe('strip-id-test'); + }); + }); + + describe('getPush', () => { + it('retrieves a push by id', async () => { + const action = createTestAction({ id: 'get-push-test' }); + await writeAudit(action); + + const result = await getPush('get-push-test'); + expect(result?.id).toBe('get-push-test'); + expect(result?.type).toBe('push'); + }); + + it('returns null for a non-existent push', async () => { + expect(await getPush('non-existent')).toBeNull(); + }); + + it('returns an Action instance', async () => { + const action = createTestAction({ id: 'action-instance-test' }); + await writeAudit(action); + + const result = await getPush('action-instance-test'); + expect(Object.getPrototypeOf(result)).toBe(Action.prototype); + }); + }); + + describe('getPushes', () => { + beforeEach(async () => { + // Three pushes with deliberately increasing timestamps so we can verify + // DESC ordering deterministically. + await writeAudit( + createTestAction({ + id: 'push-a', + timestamp: 1000, + blocked: true, + authorised: false, + }), + ); + await writeAudit( + createTestAction({ + id: 'push-b', + timestamp: 2000, + blocked: true, + authorised: false, + }), + ); + await writeAudit( + createTestAction({ + id: 'push-authorised', + timestamp: 3000, + blocked: true, + authorised: true, + }), + ); + }); + + it('orders pushes by timestamp DESC', async () => { + const result = await getPushes({}); + const ids = result.map((p) => p.id); + expect(ids).toEqual(['push-authorised', 'push-b', 'push-a']); + }); + + it('filters by authorised flag', async () => { + const result = await getPushes({ authorised: true }); + const authorisedPush = result.find((p) => p.id === 'push-authorised'); + expect(authorisedPush).toBeDefined(); + expect(result.every((p) => p.authorised === true)).toBe(true); + }); + + it('does not leak _id', async () => { + const result = await getPushes({}); + result.forEach((push) => { + expect((push as any)._id).toBeUndefined(); + expect(push.id).toBeDefined(); + }); + }); + }); + + describe('deletePush', () => { + it('deletes a push by id', async () => { + const action = createTestAction({ id: 'delete-test' }); + await writeAudit(action); + await deletePush('delete-test'); + expect(await getPush('delete-test')).toBeNull(); + }); + + it('does not throw when deleting a non-existent push', async () => { + await expect(deletePush('non-existent')).resolves.not.toThrow(); + }); + }); + + describe('authorise', () => { + it('authorises a push and resets cancel/reject flags', async () => { + const action = createTestAction({ + id: 'authorise-test', + authorised: false, + canceled: true, + rejected: true, + }); + await writeAudit(action); + + const result = await authorise('authorise-test', { note: 'approved' } as never); + expect(result.message).toBe('authorised authorise-test'); + + const updated = await getPush('authorise-test'); + expect(updated?.authorised).toBe(true); + expect(updated?.canceled).toBe(false); + expect(updated?.rejected).toBe(false); + expect((updated as any)?.attestation).toEqual({ note: 'approved' }); + }); + + it('throws for a non-existent push', async () => { + await expect(authorise('non-existent', {} as never)).rejects.toThrow( + 'push non-existent not found', + ); + }); + }); + + describe('reject', () => { + it('rejects a push and persists the rejection payload', async () => { + const action = createTestAction({ + id: 'reject-test', + authorised: true, + canceled: true, + rejected: false, + }); + await writeAudit(action); + + const rejection = { + reason: 'policy violation', + timestamp: new Date('2026-05-11T00:00:00Z'), + reviewer: { username: 'r', reviewerEmail: 'r@example.com' }, + }; + + const result = await reject('reject-test', rejection as never); + expect(result.message).toBe('reject reject-test'); + + const updated = await getPush('reject-test'); + expect(updated?.authorised).toBe(false); + expect(updated?.canceled).toBe(false); + expect(updated?.rejected).toBe(true); + // Round-tripped through JSONB — `reason` and `reviewer` survive + // exactly; the `Date` round-trips as an ISO string in JSON. + expect((updated as any)?.rejection?.reason).toBe('policy violation'); + expect((updated as any)?.rejection?.reviewer).toEqual(rejection.reviewer); + }); + + it('throws for a non-existent push', async () => { + await expect(reject('non-existent', {} as never)).rejects.toThrow( + 'push non-existent not found', + ); + }); + }); + + describe('cancel', () => { + it('cancels a push and resets authorise/reject flags', async () => { + const action = createTestAction({ + id: 'cancel-test', + authorised: true, + canceled: false, + rejected: true, + }); + await writeAudit(action); + + const result = await cancel('cancel-test'); + expect(result.message).toBe('canceled cancel-test'); + + const updated = await getPush('cancel-test'); + expect(updated?.authorised).toBe(false); + expect(updated?.canceled).toBe(true); + expect(updated?.rejected).toBe(false); + }); + + it('throws for a non-existent push', async () => { + await expect(cancel('non-existent')).rejects.toThrow('push non-existent not found'); + }); + }); +}); diff --git a/test/db/postgres/pushes.test.ts b/test/db/postgres/pushes.test.ts new file mode 100644 index 000000000..1bcadd3bc --- /dev/null +++ b/test/db/postgres/pushes.test.ts @@ -0,0 +1,331 @@ +/** + * Copyright 2026 GitProxy Contributors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { describe, it, expect, beforeEach, vi } from 'vitest'; + +const mockQuery = vi.fn(); + +vi.mock('../../../src/db/postgres/helper', () => ({ + query: mockQuery, +})); + +describe('PostgreSQL - Pushes', async () => { + const { + reject, + getPushes, + getPush, + writeAudit, + authorise, + cancel, + deletePush, + getPushesForUserProfile, + getRepoPushRollupsByCanonicalUrl, + } = await import('../../../src/db/postgres/pushes'); + + beforeEach(() => { + vi.clearAllMocks(); + }); + + describe('getPushes', () => { + it('orders results by timestamp DESC', async () => { + mockQuery.mockResolvedValue({ rowCount: 0, rows: [] }); + + await getPushes({}); + + const [sql] = mockQuery.mock.calls[0]; + expect(sql).toMatch(/ORDER BY timestamp DESC/); + }); + + it('translates allowPush to the snake_case column', async () => { + mockQuery.mockResolvedValue({ rowCount: 0, rows: [] }); + + await getPushes({ allowPush: true }); + + const [sql, params] = mockQuery.mock.calls[0]; + expect(sql).toContain('allow_push = $1'); + expect(params).toEqual([true]); + }); + + it('ignores unknown filter keys', async () => { + mockQuery.mockResolvedValue({ rowCount: 0, rows: [] }); + + await getPushes({ id: 'x' } as never); + + const [sql, params] = mockQuery.mock.calls[0]; + expect(sql).not.toContain('WHERE'); + expect(params).toEqual([]); + }); + }); + + describe('getPush', () => { + it('returns null when no row matches', async () => { + mockQuery.mockResolvedValue({ rowCount: 0, rows: [] }); + expect(await getPush('missing')).toBeNull(); + }); + }); + + describe('writeAudit', () => { + it('throws Invalid id when id is not a string', async () => { + const action = { id: 42, timestamp: 1 } as unknown as Parameters[0]; + await expect(writeAudit(action)).rejects.toThrow('Invalid id'); + expect(mockQuery).not.toHaveBeenCalled(); + }); + + it('upserts via ON CONFLICT (id)', async () => { + mockQuery.mockResolvedValue({ rowCount: 1, rows: [] }); + + const action = { + id: 'push-1', + timestamp: 1234, + type: 'push', + error: false, + blocked: true, + allowPush: false, + authorised: false, + canceled: false, + rejected: false, + } as unknown as Parameters[0]; + + await writeAudit(action); + + const [sql] = mockQuery.mock.calls[0]; + expect(sql).toContain('ON CONFLICT (id) DO UPDATE'); + }); + }); + + describe('reject', () => { + it('persists rejection payload onto data JSONB', async () => { + const rejection = { + reason: 'fails policy', + timestamp: new Date('2026-05-11T00:00:00Z'), + reviewer: { username: 'r', reviewerEmail: 'r@example.com' }, + }; + + // First call: getPush → resolves to a row whose data is the action. + // Second call: writeAudit upsert. + mockQuery + .mockResolvedValueOnce({ + rowCount: 1, + rows: [{ data: { id: 'p1', authorised: false, canceled: false, rejected: false } }], + }) + .mockResolvedValueOnce({ rowCount: 1, rows: [] }); + + const result = await reject('p1', rejection as never); + + expect(result).toEqual({ message: 'reject p1' }); + + // The upsert call serializes the action (with rejection assigned) into + // the final query parameter as JSON text. + const upsertParams = mockQuery.mock.calls[1][1] as unknown[]; + const dataJson = JSON.parse(upsertParams[9] as string); + expect(dataJson).toMatchObject({ + id: 'p1', + rejected: true, + authorised: false, + canceled: false, + rejection: { reason: 'fails policy' }, + }); + }); + + it('throws if push is not found', async () => { + mockQuery.mockResolvedValue({ rowCount: 0, rows: [] }); + + await expect(reject('missing', {} as never)).rejects.toThrow('push missing not found'); + }); + }); + + describe('authorise', () => { + it('marks the push authorised and clears canceled/rejected', async () => { + mockQuery + .mockResolvedValueOnce({ + rowCount: 1, + rows: [{ data: { id: 'p1', authorised: false, canceled: true, rejected: true } }], + }) + .mockResolvedValueOnce({ rowCount: 1, rows: [] }); + + const result = await authorise('p1', { token: 't' } as never); + + expect(result).toEqual({ message: 'authorised p1' }); + const upsertParams = mockQuery.mock.calls[1][1] as unknown[]; + const dataJson = JSON.parse(upsertParams[9] as string); + expect(dataJson).toMatchObject({ + id: 'p1', + authorised: true, + canceled: false, + rejected: false, + }); + }); + + it('throws if push is not found', async () => { + mockQuery.mockResolvedValue({ rowCount: 0, rows: [] }); + await expect(authorise('missing')).rejects.toThrow('push missing not found'); + }); + }); + + describe('cancel', () => { + it('marks the push canceled and clears authorised/rejected', async () => { + mockQuery + .mockResolvedValueOnce({ + rowCount: 1, + rows: [{ data: { id: 'p1', authorised: true, canceled: false, rejected: false } }], + }) + .mockResolvedValueOnce({ rowCount: 1, rows: [] }); + + const result = await cancel('p1'); + + expect(result).toEqual({ message: 'canceled p1' }); + const upsertParams = mockQuery.mock.calls[1][1] as unknown[]; + const dataJson = JSON.parse(upsertParams[9] as string); + expect(dataJson).toMatchObject({ + id: 'p1', + canceled: true, + authorised: false, + rejected: false, + }); + }); + + it('throws if push is not found', async () => { + mockQuery.mockResolvedValue({ rowCount: 0, rows: [] }); + await expect(cancel('missing')).rejects.toThrow('push missing not found'); + }); + }); + + describe('deletePush', () => { + it('issues a DELETE by id', async () => { + mockQuery.mockResolvedValue({ rowCount: 1, rows: [] }); + await deletePush('p1'); + const [sql, params] = mockQuery.mock.calls[0]; + expect(sql).toContain('DELETE FROM pushes WHERE id = $1'); + expect(params).toEqual(['p1']); + }); + }); + + describe('getPushesForUserProfile', () => { + it('matches the reviewer case-insensitively when there are no emails', async () => { + mockQuery.mockResolvedValue({ rowCount: 0, rows: [] }); + + await getPushesForUserProfile([], 'Alice'); + + const [sql, params] = mockQuery.mock.calls[0]; + expect(sql).toContain("data->'attestation'->'reviewer'->>'username'"); + expect(sql).toMatch(/ORDER BY timestamp DESC/); + expect(sql).not.toContain('userEmail'); + expect(params).toEqual(['Alice']); + }); + + it('matches either the author email variants or the reviewer', async () => { + mockQuery.mockResolvedValue({ rowCount: 0, rows: [] }); + + await getPushesForUserProfile(['a@b.com', 'A@B.com'], 'alice'); + + const [sql, params] = mockQuery.mock.calls[0]; + expect(sql).toContain("(data->>'userEmail') = ANY($2::text[])"); + expect(sql).toContain(' OR '); + expect(params).toEqual(['alice', ['a@b.com', 'A@B.com']]); + }); + + it('returns Action instances', async () => { + mockQuery.mockResolvedValue({ + rowCount: 1, + rows: [{ data: { id: 'p1', url: 'https://github.com/a/b.git' } }], + }); + + const result = await getPushesForUserProfile([], 'alice'); + + expect(result).toHaveLength(1); + expect(result[0].id).toBe('p1'); + }); + }); + + describe('getRepoPushRollupsByCanonicalUrl', () => { + const row = (over: Record = {}) => ({ + url: 'https://github.com/finos/git-proxy.git', + error: false, + rejected: false, + canceled: false, + authorised: false, + blocked: true, + allow_push: false, + timestamp: 1000, + ...over, + }); + + it('only scans push rows', async () => { + mockQuery.mockResolvedValue({ rowCount: 0, rows: [] }); + + await getRepoPushRollupsByCanonicalUrl(); + + const [sql] = mockQuery.mock.calls[0]; + expect(sql).toContain("WHERE type = 'push'"); + }); + + it('counts pushes per canonical url and tracks the latest timestamps', async () => { + mockQuery.mockResolvedValue({ + rowCount: 2, + rows: [row({ timestamp: 1000 }), row({ timestamp: 5000 })], + }); + + const { tabCounts, latestPushAtMs, latestPendingReviewAtMs } = + await getRepoPushRollupsByCanonicalUrl(); + + const [key] = [...tabCounts.keys()]; + expect(tabCounts.get(key)?.pending).toBe(2); + expect(latestPushAtMs.get(key)).toBe(5000); + expect(latestPendingReviewAtMs.get(key)).toBe(5000); + }); + + it('separates approved pushes from pending ones', async () => { + mockQuery.mockResolvedValue({ + rowCount: 2, + rows: [row(), row({ authorised: true, blocked: false, timestamp: 9000 })], + }); + + const { tabCounts, latestPendingReviewAtMs } = await getRepoPushRollupsByCanonicalUrl(); + const [key] = [...tabCounts.keys()]; + + expect(tabCounts.get(key)?.pending).toBe(1); + expect(tabCounts.get(key)?.approved).toBe(1); + // the approved push must not advance the pending-review timestamp + expect(latestPendingReviewAtMs.get(key)).toBe(1000); + }); + + it('skips rows with an unusable url', async () => { + mockQuery.mockResolvedValue({ rowCount: 2, rows: [row({ url: null }), row({ url: '' })] }); + + const { tabCounts } = await getRepoPushRollupsByCanonicalUrl(); + + expect(tabCounts.size).toBe(0); + }); + + it('parses BIGINT timestamps returned as strings', async () => { + mockQuery.mockResolvedValue({ rowCount: 1, rows: [row({ timestamp: '4200' })] }); + + const { latestPushAtMs } = await getRepoPushRollupsByCanonicalUrl(); + const [key] = [...latestPushAtMs.keys()]; + + expect(latestPushAtMs.get(key)).toBe(4200); + }); + + it('ignores non-numeric timestamps', async () => { + mockQuery.mockResolvedValue({ rowCount: 1, rows: [row({ timestamp: null })] }); + + const { tabCounts, latestPushAtMs } = await getRepoPushRollupsByCanonicalUrl(); + + expect(tabCounts.size).toBe(1); + expect(latestPushAtMs.size).toBe(0); + }); + }); +}); diff --git a/test/db/postgres/repo.integration.test.ts b/test/db/postgres/repo.integration.test.ts new file mode 100644 index 000000000..f39261e7b --- /dev/null +++ b/test/db/postgres/repo.integration.test.ts @@ -0,0 +1,173 @@ +/** + * Copyright 2026 GitProxy Contributors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { describe, it, expect } from 'vitest'; +import { + createRepo, + getRepo, + getRepoByUrl, + getRepoById, + getRepos, + addUserCanPush, + addUserCanAuthorise, + removeUserCanPush, + removeUserCanAuthorise, + deleteRepo, +} from '../../../src/db/postgres/repo'; +import { Repo } from '../../../src/db/types'; + +const shouldRunPostgresTests = process.env.RUN_POSTGRES_TESTS === 'true'; + +const createTestRepo = (overrides: Partial = {}): Repo => { + const id = Date.now() + Math.floor(Math.random() * 10_000); + return new Repo( + overrides.project ?? 'test-project', + overrides.name ?? `repo-${id}`, + overrides.url ?? `https://github.com/test-project/repo-${id}.git`, + overrides.users ?? { canPush: [], canAuthorise: [] }, + ); +}; + +describe.runIf(shouldRunPostgresTests)('PostgreSQL Repo Integration Tests', () => { + describe('createRepo', () => { + it('persists the row and stamps a generated _id', async () => { + const repo = createTestRepo({ name: 'create-test', url: 'https://example.com/x.git' }); + const created = await createRepo(repo); + + expect(created._id).toBeDefined(); + expect(created._id).toMatch(/^[0-9a-f-]{36}$/i); + + const fromDb = await getRepoByUrl('https://example.com/x.git'); + expect(fromDb?.name).toBe('create-test'); + }); + }); + + describe('getRepo / getRepoByUrl / getRepoById', () => { + it('finds by name (lower-cased lookup)', async () => { + await createRepo(createTestRepo({ name: 'findme', url: 'https://example.com/findme.git' })); + const found = await getRepo('FINDME'); + expect(found?.name).toBe('findme'); + }); + + it('finds by url exactly', async () => { + const url = 'https://example.com/url-test.git'; + await createRepo(createTestRepo({ name: 'url-test', url })); + const found = await getRepoByUrl(url); + expect(found?.url).toBe(url); + }); + + it('finds by _id', async () => { + const created = await createRepo( + createTestRepo({ name: 'id-test', url: 'https://example.com/id-test.git' }), + ); + const fromDb = await getRepoById(created._id as string); + expect(fromDb?.url).toBe('https://example.com/id-test.git'); + }); + + it('returns null when nothing matches', async () => { + expect(await getRepo('does-not-exist')).toBeNull(); + expect(await getRepoByUrl('https://nope.example/x.git')).toBeNull(); + }); + }); + + describe('getRepos', () => { + it('returns the seeded repos', async () => { + await createRepo(createTestRepo({ name: 'list-1', url: 'https://example.com/l1.git' })); + await createRepo(createTestRepo({ name: 'list-2', url: 'https://example.com/l2.git' })); + + const all = await getRepos(); + const names = all.map((r) => r.name); + expect(names).toEqual(expect.arrayContaining(['list-1', 'list-2'])); + }); + }); + + describe('permission JSONB', () => { + it('starts with empty arrays', async () => { + const created = await createRepo( + createTestRepo({ name: 'perm-start', url: 'https://example.com/ps.git' }), + ); + const fromDb = await getRepoById(created._id as string); + expect(fromDb?.users.canPush).toEqual([]); + expect(fromDb?.users.canAuthorise).toEqual([]); + }); + + it('adds a user without duplication', async () => { + const created = await createRepo( + createTestRepo({ name: 'perm-add', url: 'https://example.com/pa.git' }), + ); + const id = created._id as string; + + await addUserCanPush(id, 'Alice'); + await addUserCanPush(id, 'alice'); // duplicate (after lower-casing) + + const fromDb = await getRepoById(id); + expect(fromDb?.users.canPush).toEqual(['alice']); + }); + + it('removes the last user, leaving an empty array (NOT null)', async () => { + const created = await createRepo( + createTestRepo({ name: 'perm-remove', url: 'https://example.com/pr.git' }), + ); + const id = created._id as string; + + await addUserCanPush(id, 'bob'); + await removeUserCanPush(id, 'bob'); + + const fromDb = await getRepoById(id); + // Same behavior as Mongo and NeDB + expect(fromDb?.users.canPush).toEqual([]); + expect(fromDb?.users.canPush).not.toBeNull(); + }); + + it('applies the same invariant to canAuthorise', async () => { + const created = await createRepo( + createTestRepo({ name: 'auth-remove', url: 'https://example.com/ar.git' }), + ); + const id = created._id as string; + + await addUserCanAuthorise(id, 'reviewer'); + await removeUserCanAuthorise(id, 'reviewer'); + + const fromDb = await getRepoById(id); + expect(fromDb?.users.canAuthorise).toEqual([]); + expect(fromDb?.users.canAuthorise).not.toBeNull(); + }); + + it('keeps other users intact when removing one', async () => { + const created = await createRepo( + createTestRepo({ name: 'multi-perm', url: 'https://example.com/mp.git' }), + ); + const id = created._id as string; + + await addUserCanPush(id, 'alice'); + await addUserCanPush(id, 'bob'); + await removeUserCanPush(id, 'alice'); + + const fromDb = await getRepoById(id); + expect(fromDb?.users.canPush).toEqual(['bob']); + }); + }); + + describe('deleteRepo', () => { + it('deletes by _id', async () => { + const created = await createRepo( + createTestRepo({ name: 'del', url: 'https://example.com/del.git' }), + ); + await deleteRepo(created._id as string); + expect(await getRepoById(created._id as string)).toBeNull(); + }); + }); +}); diff --git a/test/db/postgres/repo.test.ts b/test/db/postgres/repo.test.ts new file mode 100644 index 000000000..29c89adb3 --- /dev/null +++ b/test/db/postgres/repo.test.ts @@ -0,0 +1,335 @@ +/** + * Copyright 2026 GitProxy Contributors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { describe, it, expect, beforeEach, vi } from 'vitest'; + +const mockQuery = vi.fn(); + +vi.mock('../../../src/db/postgres/helper', () => ({ + query: mockQuery, +})); + +describe('PostgreSQL - Repo', async () => { + const { + getRepos, + getRepo, + getRepoById, + getRepoByUrl, + updateRepo, + createRepo, + addUserCanPush, + addUserCanAuthorise, + removeUserCanPush, + removeUserCanAuthorise, + deleteRepo, + } = await import('../../../src/db/postgres/repo'); + + beforeEach(() => { + vi.clearAllMocks(); + }); + + describe('getRepos', () => { + it('builds WHERE clauses for name, project and url and maps rows', async () => { + mockQuery.mockResolvedValue({ + rowCount: 1, + rows: [ + { + _id: 'r1', + project: 'finos', + name: 'git-proxy', + url: 'https://example.com/finos/git-proxy', + users: { canPush: ['bob'], canAuthorise: [] }, + }, + ], + }); + + const repos = await getRepos({ + name: 'Git-Proxy', + project: 'finos', + url: 'https://example.com/finos/git-proxy', + }); + + const [sql, params] = mockQuery.mock.calls[0]; + expect(sql).toContain('WHERE'); + expect(sql).toContain('name = $1'); + expect(sql).toContain('project = $2'); + expect(sql).toContain('url = $3'); + expect(params).toEqual(['git-proxy', 'finos', 'https://example.com/finos/git-proxy']); + expect(repos[0].users.canPush).toEqual(['bob']); + }); + + it('omits the WHERE clause when no query is supplied', async () => { + mockQuery.mockResolvedValue({ rowCount: 0, rows: [] }); + await getRepos(); + const [sql] = mockQuery.mock.calls[0]; + expect(sql).not.toContain('WHERE'); + }); + }); + + describe('getRepoByUrl', () => { + it('returns null when no row matches', async () => { + mockQuery.mockResolvedValue({ rowCount: 0, rows: [] }); + expect(await getRepoByUrl('https://missing')).toBeNull(); + }); + + it('maps the row when found', async () => { + mockQuery.mockResolvedValue({ + rowCount: 1, + rows: [ + { + _id: 'r1', + project: 'p', + name: 'n', + url: 'https://example.com/p/n', + users: { canPush: [], canAuthorise: ['amy'] }, + }, + ], + }); + const repo = await getRepoByUrl('https://example.com/p/n'); + expect(repo?.users.canAuthorise).toEqual(['amy']); + }); + }); + + describe('deleteRepo', () => { + it('issues a DELETE by _id', async () => { + mockQuery.mockResolvedValue({ rowCount: 1, rows: [] }); + await deleteRepo('r1'); + const [sql, params] = mockQuery.mock.calls[0]; + expect(sql).toContain('DELETE FROM repos WHERE _id = $1'); + expect(params).toEqual(['r1']); + }); + }); + + describe('read normalization', () => { + it('returns empty arrays when stored users is null', async () => { + mockQuery.mockResolvedValue({ + rowCount: 1, + rows: [ + { + _id: 'r-1', + project: 'p', + name: 'n', + url: 'https://example.com/p/n', + users: null, + }, + ], + }); + + const repo = await getRepoById('r-1'); + expect(repo?.users.canPush).toEqual([]); + expect(repo?.users.canAuthorise).toEqual([]); + }); + + it('lower-cases the name on getRepo', async () => { + mockQuery.mockResolvedValue({ rowCount: 0, rows: [] }); + await getRepo('MixedCase'); + expect(mockQuery.mock.calls[0][1]).toEqual(['mixedcase']); + }); + }); + + describe('createRepo', () => { + it('serialises default users JSONB and stamps _id from RETURNING', async () => { + mockQuery.mockResolvedValue({ rowCount: 1, rows: [{ _id: 'generated-uuid' }] }); + + const created = await createRepo({ + project: 'finos', + name: 'git-proxy', + url: 'https://github.com/finos/git-proxy.git', + users: { canPush: [], canAuthorise: [] }, + } as never); + + expect(created._id).toBe('generated-uuid'); + const params = mockQuery.mock.calls[0][1] as unknown[]; + // Last param is the JSONB string for users. + expect(JSON.parse(params[3] as string)).toEqual({ canPush: [], canAuthorise: [] }); + }); + }); + + describe('add/remove user — empty array invariant', () => { + it('lower-cases user on add', async () => { + mockQuery.mockResolvedValue({ rowCount: 1, rows: [] }); + await addUserCanPush('r-1', 'Bob'); + const params = mockQuery.mock.calls[0][1] as unknown[]; + expect(params).toContain('bob'); + }); + + it('addUserCanAuthorise lower-cases user and targets the canAuthorise role', async () => { + mockQuery.mockResolvedValue({ rowCount: 1, rows: [] }); + await addUserCanAuthorise('r-1', 'Amy'); + const params = mockQuery.mock.calls[0][1] as unknown[]; + expect(params).toContain('amy'); + expect(params).toContain('{canAuthorise}'); + }); + + it('removeUserCanPush coalesces filtered array to [] when last user leaves', () => { + // The whole point of the issue: the SQL fragment must coalesce a NULL + // aggregate result back to '[]'::jsonb so the array does not collapse + // to null when the last user is removed. + mockQuery.mockResolvedValue({ rowCount: 1, rows: [] }); + return removeUserCanPush('r-1', 'bob').then(() => { + const [sql] = mockQuery.mock.calls[0]; + expect(sql).toContain('coalesce('); + expect(sql).toContain("'[]'::jsonb"); + }); + }); + + it('removeUserCanAuthorise applies the same empty-array coalesce', async () => { + mockQuery.mockResolvedValue({ rowCount: 1, rows: [] }); + await removeUserCanAuthorise('r-1', 'bob'); + const [sql] = mockQuery.mock.calls[0]; + expect(sql).toContain('coalesce('); + expect(sql).toContain("'[]'::jsonb"); + }); + }); + + describe('updateRepo', () => { + it('writes only the supplied fields', async () => { + mockQuery.mockResolvedValue({ rowCount: 1, rows: [] }); + + await updateRepo({ _id: 'r1', name: 'renamed' }); + + const [sql, params] = mockQuery.mock.calls[0]; + expect(sql).toContain('UPDATE repos SET name = $1'); + expect(sql).toContain('WHERE _id = $2'); + expect(params).toEqual(['renamed', 'r1']); + }); + + it('serialises the users permission object as jsonb', async () => { + mockQuery.mockResolvedValue({ rowCount: 1, rows: [] }); + + const users = { canPush: ['alice'], canAuthorise: [] }; + await updateRepo({ _id: 'r1', users }); + + const [sql, params] = mockQuery.mock.calls[0]; + expect(sql).toContain('users = $1::jsonb'); + expect(params).toEqual([JSON.stringify(users), 'r1']); + }); + + it('resets a field back to its column default when set to undefined', async () => { + mockQuery.mockResolvedValue({ rowCount: 1, rows: [] }); + + await updateRepo({ _id: 'r1', project: undefined, name: 'keep' }); + + const [sql] = mockQuery.mock.calls[0]; + expect(sql).toContain('project = DEFAULT'); + expect(sql).toContain('name = $1'); + }); + + it('ignores unknown fields', async () => { + mockQuery.mockResolvedValue({ rowCount: 1, rows: [] }); + + await updateRepo({ _id: 'r1', name: 'x', bogus: 'y' } as never); + + const [sql] = mockQuery.mock.calls[0]; + expect(sql).not.toContain('bogus'); + }); + + it('requires an _id', async () => { + await expect(updateRepo({ name: 'x' })).rejects.toThrow('updateRepo requires a repo _id'); + expect(mockQuery).not.toHaveBeenCalled(); + }); + + it('rejects an update with nothing to change', async () => { + await expect(updateRepo({ _id: 'r1' })).rejects.toThrow( + 'updateRepo requires at least one field to update', + ); + expect(mockQuery).not.toHaveBeenCalled(); + }); + }); + + describe('repo date fields', () => { + it('defaults dateCreated and lastModified on create', async () => { + mockQuery.mockResolvedValue({ rowCount: 1, rows: [{ _id: 'r1' }] }); + + const repo = await createRepo({ + project: 'p', + name: 'n', + url: 'https://github.com/p/n.git', + } as never); + + const [sql, params] = mockQuery.mock.calls[0]; + expect(sql).toContain('date_created'); + expect(sql).toContain('last_modified'); + expect(repo.dateCreated).toBeTruthy(); + expect(repo.lastModified).toBe(repo.dateCreated); + expect(params[4]).toBe(repo.dateCreated); + }); + + it('keeps caller-supplied dates on create', async () => { + mockQuery.mockResolvedValue({ rowCount: 1, rows: [{ _id: 'r1' }] }); + + const repo = await createRepo({ + project: 'p', + name: 'n', + url: 'https://github.com/p/n.git', + dateCreated: '2026-01-01T00:00:00.000Z', + lastModified: '2026-01-02T00:00:00.000Z', + } as never); + + expect(repo.dateCreated).toBe('2026-01-01T00:00:00.000Z'); + expect(repo.lastModified).toBe('2026-01-02T00:00:00.000Z'); + }); + + it('updateRepo writes dateCreated and lastModified columns', async () => { + mockQuery.mockResolvedValue({ rowCount: 1, rows: [] }); + + await updateRepo({ + _id: 'r1', + dateCreated: '2026-01-01T00:00:00.000Z', + lastModified: '2026-01-01T00:00:00.000Z', + }); + + const [sql, params] = mockQuery.mock.calls[0]; + expect(sql).toContain('date_created = $1'); + expect(sql).toContain('last_modified = $2'); + expect(params).toEqual(['2026-01-01T00:00:00.000Z', '2026-01-01T00:00:00.000Z', 'r1']); + }); + + it('bumps last_modified when permissions change', async () => { + mockQuery.mockResolvedValue({ rowCount: 1, rows: [] }); + + await addUserCanPush('r1', 'Alice'); + await removeUserCanPush('r1', 'Alice'); + + for (const [sql, params] of mockQuery.mock.calls) { + expect(sql).toContain('last_modified = $5'); + expect(typeof params[4]).toBe('string'); + } + }); + + it('returns the date fields from reads', async () => { + mockQuery.mockResolvedValue({ + rowCount: 1, + rows: [ + { + _id: 'r1', + project: 'p', + name: 'n', + url: 'u', + users: null, + date_created: '2026-01-01T00:00:00.000Z', + last_modified: '2026-01-02T00:00:00.000Z', + }, + ], + }); + + const repos = await getRepos(); + + expect(repos[0].dateCreated).toBe('2026-01-01T00:00:00.000Z'); + expect(repos[0].lastModified).toBe('2026-01-02T00:00:00.000Z'); + }); + }); +}); diff --git a/test/db/postgres/schemaMigrations.integration.test.ts b/test/db/postgres/schemaMigrations.integration.test.ts new file mode 100644 index 000000000..a68b1b109 --- /dev/null +++ b/test/db/postgres/schemaMigrations.integration.test.ts @@ -0,0 +1,76 @@ +/** + * Copyright 2026 GitProxy Contributors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { describe, it, expect } from 'vitest'; + +import { connect, query, resetConnection } from '../../../src/db/postgres/helper'; + +const shouldRunPostgresTests = process.env.RUN_POSTGRES_TESTS === 'true'; + +// Drop everything so the next `connect()` exercises the migration runner from a +// genuinely empty database. The initial `query` self-bootstraps the schema; the +// DROP then clears it, and `resetConnection` releases the once-per-process latch +// so the following `connect()` re-runs migrations. +const resetToEmptyDatabase = async () => { + await query('DROP TABLE IF EXISTS schema_migrations, pushes, repos, users CASCADE'); + await resetConnection(); +}; + +describe.runIf(shouldRunPostgresTests)('PostgreSQL Schema Migration Integration Tests', () => { + it('creates schema_migrations and the app tables and records version 1', async () => { + await resetToEmptyDatabase(); + + // First pool acquisition triggers the migration runner. + await connect(); + + const versions = await query<{ version: number }>( + 'SELECT version FROM schema_migrations ORDER BY version', + ); + expect(versions.rows.map((row) => row.version)).toEqual([1, 2]); + + const tables = await query<{ tablename: string }>( + `SELECT tablename FROM pg_tables + WHERE schemaname = 'public' AND tablename IN ('users', 'repos', 'pushes')`, + ); + expect(tables.rows.map((row) => row.tablename).sort()).toEqual(['pushes', 'repos', 'users']); + }); + + it('upgrades the users table to the version 2 shape on a fresh database', async () => { + await resetToEmptyDatabase(); + await connect(); + + const emailColumn = await query<{ is_nullable: string }>( + `SELECT is_nullable FROM information_schema.columns + WHERE table_name = 'users' AND column_name = 'email'`, + ); + expect(emailColumn.rows[0].is_nullable).toBe('YES'); + + const publicKeysColumn = await query<{ data_type: string }>( + `SELECT data_type FROM information_schema.columns + WHERE table_name = 'users' AND column_name = 'public_keys'`, + ); + expect(publicKeysColumn.rows[0].data_type).toBe('jsonb'); + }); + + it('is idempotent — re-running migrations does not duplicate the version row', async () => { + await connect(); + await resetConnection(); + await connect(); + + const versions = await query<{ version: number }>('SELECT version FROM schema_migrations'); + expect(versions.rows.map((row) => row.version)).toEqual([1, 2]); + }); +}); diff --git a/test/db/postgres/schemaMigrations.test.ts b/test/db/postgres/schemaMigrations.test.ts new file mode 100644 index 000000000..b1f933675 --- /dev/null +++ b/test/db/postgres/schemaMigrations.test.ts @@ -0,0 +1,120 @@ +/** + * Copyright 2026 GitProxy Contributors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { describe, it, expect, vi } from 'vitest'; + +import { runMigrations, MIGRATIONS } from '../../../src/db/postgres/schemaMigrations'; + +const SELECT_VERSIONS = /SELECT version FROM schema_migrations/; + +// Build a fake pg Pool whose single client records every query. `appliedRows` +// is what the `SELECT version FROM schema_migrations` lookup returns. +const makePool = (appliedRows: { version: number }[] = []) => { + const query = vi + .fn() + .mockImplementation((sql: string) => + SELECT_VERSIONS.test(sql) + ? Promise.resolve({ rows: appliedRows, rowCount: appliedRows.length }) + : Promise.resolve({ rows: [], rowCount: 0 }), + ); + const release = vi.fn(); + const pool = { connect: vi.fn().mockResolvedValue({ query, release }) }; + return { pool, query, release }; +}; + +const sqlsOf = (query: ReturnType) => query.mock.calls.map((call) => String(call[0])); + +describe('PostgreSQL - migrations', () => { + it('exposes an ordered, append-only migration list starting at version 1', () => { + expect(MIGRATIONS[0].version).toBe(1); + + const versions = MIGRATIONS.map((m) => m.version); + expect(versions).toEqual([...versions].sort((a, b) => a - b)); + expect(new Set(versions).size).toBe(versions.length); + }); + + it('locks, then creates schema_migrations, then commits — in that order', async () => { + const { pool, query, release } = makePool([]); + + await runMigrations(pool as never); + + const sqls = sqlsOf(query); + expect(sqls[0]).toBe('BEGIN'); + expect(sqls[1]).toMatch(/pg_advisory_xact_lock/); + expect(sqls[2]).toMatch(/CREATE TABLE IF NOT EXISTS schema_migrations/); + expect(sqls[sqls.length - 1]).toBe('COMMIT'); + expect(release).toHaveBeenCalledTimes(1); + }); + + it('applies every pending migration and records its version', async () => { + const { pool, query } = makePool([]); + + await runMigrations(pool as never); + + const inserts = query.mock.calls.filter((call) => + /INSERT INTO schema_migrations/.test(String(call[0])), + ); + expect(inserts).toHaveLength(MIGRATIONS.length); + expect(inserts[0][1]).toEqual([MIGRATIONS[0].version, MIGRATIONS[0].name]); + + // The migration body runs before its bookkeeping insert. + const sqls = sqlsOf(query); + expect(sqls).toContain(MIGRATIONS[0].sql); + }); + + it('skips migrations already recorded as applied', async () => { + const allApplied = MIGRATIONS.map((m) => ({ version: m.version })); + const { pool, query } = makePool(allApplied); + + await runMigrations(pool as never); + + const inserts = query.mock.calls.filter((call) => + /INSERT INTO schema_migrations/.test(String(call[0])), + ); + expect(inserts).toHaveLength(0); + expect(sqlsOf(query)).toContain('COMMIT'); + }); + + it('rolls back and releases the client when a migration fails', async () => { + const query = vi.fn().mockImplementation((sql: string) => { + if (SELECT_VERSIONS.test(sql)) return Promise.resolve({ rows: [], rowCount: 0 }); + if (sql === MIGRATIONS[0].sql) return Promise.reject(new Error('migration boom')); + return Promise.resolve({ rows: [], rowCount: 0 }); + }); + const release = vi.fn(); + const pool = { connect: vi.fn().mockResolvedValue({ query, release }) }; + + await expect(runMigrations(pool as never)).rejects.toThrow('migration boom'); + + expect(sqlsOf(query)).toContain('ROLLBACK'); + expect(release).toHaveBeenCalledTimes(1); + }); + + it('rethrows the original error even when ROLLBACK also fails', async () => { + const query = vi.fn().mockImplementation((sql: string) => { + if (SELECT_VERSIONS.test(sql)) return Promise.resolve({ rows: [], rowCount: 0 }); + if (sql === MIGRATIONS[0].sql) return Promise.reject(new Error('migration boom')); + if (sql === 'ROLLBACK') return Promise.reject(new Error('rollback boom')); + return Promise.resolve({ rows: [], rowCount: 0 }); + }); + const release = vi.fn(); + const pool = { connect: vi.fn().mockResolvedValue({ query, release }) }; + + // The migration failure must surface, not the secondary rollback failure. + await expect(runMigrations(pool as never)).rejects.toThrow('migration boom'); + expect(release).toHaveBeenCalledTimes(1); + }); +}); diff --git a/test/db/postgres/users.integration.test.ts b/test/db/postgres/users.integration.test.ts new file mode 100644 index 000000000..9aad5bf50 --- /dev/null +++ b/test/db/postgres/users.integration.test.ts @@ -0,0 +1,294 @@ +/** + * Copyright 2026 GitProxy Contributors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { describe, it, expect } from 'vitest'; +import { + createUser, + findUser, + findUserByEmail, + findUserByGitAccount, + findUserByOIDC, + findUserBySSHKey, + getUsers, + updateUser, + deleteUser, + addPublicKey, + removePublicKey, + getPublicKeys, +} from '../../../src/db/postgres/users'; +import { DuplicateSSHKeyError } from '../../../src/errors/DatabaseErrors'; +import { PublicKeyRecord, User } from '../../../src/db/types'; + +const shouldRunPostgresTests = process.env.RUN_POSTGRES_TESTS === 'true'; + +describe.runIf(shouldRunPostgresTests)('PostgreSQL Users Integration Tests', () => { + const createTestUser = (overrides: Partial = {}): User => { + const timestamp = Date.now(); + return new User( + overrides.username || `testuser-${timestamp}`, + overrides.password || 'hashedpassword123', + overrides.gitAccount || `git-${timestamp}`, + overrides.email || `test-${timestamp}@example.com`, + overrides.admin ?? false, + overrides.oidcId || null, + ); + }; + + describe('createUser', () => { + it('lowercases username and email on insert', async () => { + const user = createTestUser({ username: 'CreateUser', email: 'Create@Example.COM' }); + await createUser(user); + + const found = await findUser('createuser'); + expect(found?.username).toBe('createuser'); + expect(found?.email).toBe('create@example.com'); + }); + }); + + describe('findUser', () => { + it('finds a user by username (case-insensitive)', async () => { + await createUser(createTestUser({ username: 'findme' })); + const result = await findUser('FINDME'); + expect(result?.username).toBe('findme'); + }); + + it('returns null for a non-existent user', async () => { + expect(await findUser('non-existent-user')).toBeNull(); + }); + }); + + describe('findUserByEmail', () => { + it('finds a user by email (case-insensitive)', async () => { + await createUser(createTestUser({ email: 'findbyemail@test.com' })); + const result = await findUserByEmail('FindByEmail@TEST.com'); + expect(result?.email).toBe('findbyemail@test.com'); + }); + + it('returns null for a non-existent email', async () => { + expect(await findUserByEmail('nonexistent@test.com')).toBeNull(); + }); + }); + + describe('findUserByGitAccount', () => { + it('finds a user by git account (case-insensitive), mirroring mongo', async () => { + await createUser(createTestUser({ username: 'gitacctuser', gitAccount: 'findbygit-acct' })); + const result = await findUserByGitAccount('FindByGit-Acct'); + expect(result?.gitAccount).toBe('findbygit-acct'); + }); + + it('returns null for a non-existent git account', async () => { + expect(await findUserByGitAccount('non-existent-git-account')).toBeNull(); + }); + }); + + describe('findUserByOIDC', () => { + it('finds a user by OIDC ID', async () => { + const oidcId = `oidc-${Date.now()}`; + await createUser(createTestUser({ oidcId })); + const result = await findUserByOIDC(oidcId); + expect(result?.oidcId).toBe(oidcId); + }); + + it('returns null for a non-existent OIDC ID', async () => { + expect(await findUserByOIDC('non-existent-oidc')).toBeNull(); + }); + }); + + describe('getUsers', () => { + it('retrieves users without their password', async () => { + await createUser(createTestUser({ username: 'getusers1' })); + await createUser(createTestUser({ username: 'getusers2' })); + + const result = await getUsers(); + + expect(result.length).toBeGreaterThanOrEqual(2); + result.forEach((user) => { + // Mirrors mongo's projection — passwords are null in list responses. + expect(user.password).toBeNull(); + }); + }); + + it('filters by username (lowercased)', async () => { + await createUser(createTestUser({ username: 'filteruser', email: 'filter@test.com' })); + await createUser(createTestUser({ username: 'otheruser', email: 'other@test.com' })); + + const result = await getUsers({ username: 'FilterUser' }); + + expect(result.length).toBe(1); + expect(result[0].username).toBe('filteruser'); + }); + + it('filters by email (lowercased)', async () => { + await createUser(createTestUser({ username: 'emailfilter', email: 'unique-email@test.com' })); + + const result = await getUsers({ email: 'Unique-Email@TEST.com' }); + + expect(result.length).toBe(1); + expect(result[0].email).toBe('unique-email@test.com'); + }); + }); + + describe('updateUser', () => { + it('updates by username and lowercases new fields', async () => { + await createUser(createTestUser({ username: 'updateme', admin: false })); + + await updateUser({ username: 'UpdateMe', admin: true }); + + const updated = await findUser('updateme'); + expect(updated?.admin).toBe(true); + }); + + it('updates by _id when provided', async () => { + await createUser(createTestUser({ username: 'updatebyid' })); + const created = await findUser('updatebyid'); + await updateUser({ _id: created?._id as string, gitAccount: 'new-git-account' }); + + const updated = await findUser('updatebyid'); + expect(updated?.gitAccount).toBe('new-git-account'); + }); + + it('lowercases email during update', async () => { + await createUser(createTestUser({ username: 'lowercaseupdate' })); + await updateUser({ username: 'LowerCaseUpdate', email: 'NEW@EMAIL.COM' }); + + const updated = await findUser('lowercaseupdate'); + expect(updated?.email).toBe('new@email.com'); + }); + + it('inserts when no row matches and only username is provided', async () => { + await updateUser({ + username: 'brand-new-user', + email: 'brand-new@example.com', + gitAccount: 'brand-new-git', + }); + + const inserted = await findUser('brand-new-user'); + expect(inserted?.email).toBe('brand-new@example.com'); + expect(inserted?.gitAccount).toBe('brand-new-git'); + }); + + it('allows multiple users without an email', async () => { + // e.g. users synced from AD, where the mail attribute is optional + await updateUser({ username: 'no-email-1', gitAccount: 'git-1' }); + await updateUser({ username: 'no-email-2', gitAccount: 'git-2' }); + + expect((await findUser('no-email-1'))?.username).toBe('no-email-1'); + expect((await findUser('no-email-2'))?.username).toBe('no-email-2'); + }); + + it('still rejects a duplicate non-empty email', async () => { + await createUser(createTestUser({ username: 'emailowner', email: 'taken@example.com' })); + + await expect( + createUser(createTestUser({ username: 'emailthief', email: 'taken@example.com' })), + ).rejects.toThrow(/duplicate key/); + }); + }); + + describe('deleteUser', () => { + it('deletes a user by username (case-insensitive)', async () => { + await createUser(createTestUser({ username: 'deleteme' })); + await deleteUser('DeleteMe'); + expect(await findUser('deleteme')).toBeNull(); + }); + }); + + describe('SSH public keys', () => { + const makeKey = (suffix: string): PublicKeyRecord => ({ + key: `ssh-ed25519 AAAAC3NzaC1lZDI1NTE5-${suffix}`, + name: `key-${suffix}`, + addedAt: new Date().toISOString(), + fingerprint: `SHA256:${suffix}`, + }); + + it('starts with an empty publicKeys array', async () => { + await createUser(createTestUser({ username: 'sshempty' })); + await expect(getPublicKeys('sshempty')).resolves.toEqual([]); + }); + + it('adds a key and finds the user by it', async () => { + const key = makeKey('add-and-find'); + await createUser(createTestUser({ username: 'sshadd' })); + await addPublicKey('sshadd', key); + + await expect(getPublicKeys('sshadd')).resolves.toEqual([key]); + const found = await findUserBySSHKey(key.key); + expect(found?.username).toBe('sshadd'); + }); + + it('rejects a key already registered to another user', async () => { + const key = makeKey('cross-user'); + await createUser(createTestUser({ username: 'sshowner' })); + await createUser(createTestUser({ username: 'sshthief' })); + await addPublicKey('sshowner', key); + + await expect(addPublicKey('sshthief', key)).rejects.toThrow(DuplicateSSHKeyError); + }); + + it('rejects a duplicate key for the same user', async () => { + const key = makeKey('same-user-dup'); + await createUser(createTestUser({ username: 'sshdup' })); + await addPublicKey('sshdup', key); + + await expect(addPublicKey('sshdup', key)).rejects.toThrow('SSH key already exists'); + }); + + it('rejects adding a key for a missing user', async () => { + await expect(addPublicKey('ssh-ghost', makeKey('ghost'))).rejects.toThrow('User not found'); + }); + + it('removes a key by fingerprint and leaves the rest', async () => { + const keep = makeKey('keep'); + const drop = makeKey('drop'); + await createUser(createTestUser({ username: 'sshremove' })); + await addPublicKey('sshremove', keep); + await addPublicKey('sshremove', drop); + + await removePublicKey('sshremove', drop.fingerprint); + + await expect(getPublicKeys('sshremove')).resolves.toEqual([keep]); + expect(await findUserBySSHKey(drop.key)).toBeNull(); + }); + + it('keeps an empty array (not null) after the last key is removed', async () => { + const key = makeKey('last-key'); + await createUser(createTestUser({ username: 'sshlast' })); + await addPublicKey('sshlast', key); + await removePublicKey('sshlast', key.fingerprint); + + await expect(getPublicKeys('sshlast')).resolves.toEqual([]); + }); + + it('is a no-op when removing an unknown fingerprint', async () => { + const key = makeKey('stable'); + await createUser(createTestUser({ username: 'sshnoop' })); + await addPublicKey('sshnoop', key); + + await removePublicKey('sshnoop', 'SHA256:does-not-exist'); + + await expect(getPublicKeys('sshnoop')).resolves.toEqual([key]); + }); + + it('round-trips publicKeys through createUser', async () => { + const key = makeKey('roundtrip'); + const user = createTestUser({ username: 'sshseeded' }); + user.publicKeys = [key]; + await createUser(user); + + await expect(getPublicKeys('sshseeded')).resolves.toEqual([key]); + }); + }); +}); diff --git a/test/db/postgres/users.test.ts b/test/db/postgres/users.test.ts new file mode 100644 index 000000000..59ccf087a --- /dev/null +++ b/test/db/postgres/users.test.ts @@ -0,0 +1,344 @@ +/** + * Copyright 2026 GitProxy Contributors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { describe, it, expect, beforeEach, vi } from 'vitest'; + +const mockQuery = vi.fn(); + +vi.mock('../../../src/db/postgres/helper', () => ({ + query: mockQuery, +})); + +describe('PostgreSQL - Users', async () => { + const { + findUser, + findUserByEmail, + findUserByGitAccount, + findUserByOIDC, + findUserBySSHKey, + createUser, + deleteUser, + getUsers, + updateUser, + addPublicKey, + removePublicKey, + getPublicKeys, + } = await import('../../../src/db/postgres/users'); + + beforeEach(() => { + vi.clearAllMocks(); + }); + + describe('case insensitivity', () => { + it('lower-cases username on findUser', async () => { + mockQuery.mockResolvedValue({ rowCount: 0, rows: [] }); + await findUser('Mixed-Case'); + expect(mockQuery.mock.calls[0][1]).toEqual(['mixed-case']); + }); + + it('lower-cases email on findUserByEmail', async () => { + mockQuery.mockResolvedValue({ rowCount: 0, rows: [] }); + await findUserByEmail('USER@Example.COM'); + expect(mockQuery.mock.calls[0][1]).toEqual(['user@example.com']); + }); + + it('lower-cases gitAccount on findUserByGitAccount', async () => { + mockQuery.mockResolvedValue({ rowCount: 0, rows: [] }); + await findUserByGitAccount('Alice-Git'); + expect(mockQuery.mock.calls[0][1]).toEqual(['alice-git']); + }); + + it('lower-cases username/email on createUser', async () => { + mockQuery.mockResolvedValue({ rowCount: 1, rows: [] }); + await createUser({ + username: 'Alice', + password: 'pw', + gitAccount: 'alice-git', + email: 'Alice@Example.com', + admin: false, + } as never); + + const params = mockQuery.mock.calls[0][1] as unknown[]; + expect(params[0]).toBe('alice'); + expect(params[1]).toBe('alice@example.com'); + }); + + it('lower-cases username on deleteUser', async () => { + mockQuery.mockResolvedValue({ rowCount: 1, rows: [] }); + await deleteUser('Alice'); + expect(mockQuery.mock.calls[0][1]).toEqual(['alice']); + }); + }); + + describe('row mapping', () => { + it('maps a DB row to a User on findUser', async () => { + mockQuery.mockResolvedValue({ + rowCount: 1, + rows: [ + { + _id: 'u1', + username: 'alice', + email: 'alice@example.com', + password: 'hash', + git_account: 'alice-git', + admin: true, + oidc_id: null, + display_name: 'Alice A.', + title: 'Dev', + }, + ], + }); + + const user = await findUser('alice'); + + expect(user).toMatchObject({ + _id: 'u1', + username: 'alice', + email: 'alice@example.com', + password: 'hash', + gitAccount: 'alice-git', + admin: true, + displayName: 'Alice A.', + title: 'Dev', + }); + }); + }); + + describe('findUserByGitAccount', () => { + it('queries by git_account and returns null when absent', async () => { + mockQuery.mockResolvedValue({ rowCount: 0, rows: [] }); + const user = await findUserByGitAccount('alice-git'); + const [sql, params] = mockQuery.mock.calls[0]; + expect(sql).toContain('WHERE git_account = $1'); + expect(params).toEqual(['alice-git']); + expect(user).toBeNull(); + }); + }); + + describe('findUserByOIDC', () => { + it('queries by oidc_id and returns null when absent', async () => { + mockQuery.mockResolvedValue({ rowCount: 0, rows: [] }); + const user = await findUserByOIDC('oidc-123'); + const [sql, params] = mockQuery.mock.calls[0]; + expect(sql).toContain('WHERE oidc_id = $1'); + expect(params).toEqual(['oidc-123']); + expect(user).toBeNull(); + }); + }); + + describe('getUsers', () => { + it('omits password from the SELECT projection', async () => { + mockQuery.mockResolvedValue({ rowCount: 0, rows: [] }); + await getUsers({}); + const [sql] = mockQuery.mock.calls[0]; + expect(sql).toContain('NULL::text AS password'); + }); + + it('builds lower-cased username and email filters', async () => { + mockQuery.mockResolvedValue({ rowCount: 0, rows: [] }); + await getUsers({ username: 'Alice', email: 'Alice@Example.com' }); + const [sql, params] = mockQuery.mock.calls[0]; + expect(sql).toContain('username = $1'); + expect(sql).toContain('email = $2'); + expect(params).toEqual(['alice', 'alice@example.com']); + }); + }); + + describe('updateUser', () => { + it('updates by _id when provided', async () => { + mockQuery.mockResolvedValue({ rowCount: 1, rows: [] }); + + await updateUser({ _id: 'abc-123', displayName: 'Alice A.' } as never); + + const [sql, params] = mockQuery.mock.calls[0]; + expect(sql).toContain('UPDATE users SET'); + expect(sql).toContain('WHERE _id = $'); + expect(params).toEqual(['Alice A.', 'abc-123']); + }); + + it('upserts by username in a single atomic statement', async () => { + mockQuery.mockResolvedValue({ rowCount: 1, rows: [] }); + + await updateUser({ username: 'new-user', email: 'new@example.com', admin: true } as never); + + expect(mockQuery).toHaveBeenCalledTimes(1); + const [sql, params] = mockQuery.mock.calls[0]; + expect(sql).toContain('INSERT INTO users'); + expect(sql).toContain('ON CONFLICT (username) DO UPDATE SET'); + // Only the supplied fields are merged onto an existing row. + expect(sql).toContain( + 'username = EXCLUDED.username, email = EXCLUDED.email, admin = EXCLUDED.admin', + ); + expect(sql).not.toContain('password = EXCLUDED.password'); + // username is the first INSERT param. + expect((params as unknown[])[0]).toBe('new-user'); + }); + + it('lower-cases username and email in the upsert', async () => { + mockQuery.mockResolvedValue({ rowCount: 1, rows: [] }); + + await updateUser({ username: 'ExistingUser', email: 'Updated@Example.com' } as never); + + const [, params] = mockQuery.mock.calls[0]; + expect((params as unknown[]).slice(0, 2)).toEqual(['existinguser', 'updated@example.com']); + }); + + it('throws if neither _id nor username is supplied', async () => { + await expect(updateUser({ admin: true } as never)).rejects.toThrow( + 'updateUser requires either _id or username', + ); + }); + + it('throws when no updatable field is supplied', async () => { + await expect(updateUser({ _id: 'abc-123' } as never)).rejects.toThrow( + 'updateUser requires at least one field to update', + ); + expect(mockQuery).not.toHaveBeenCalled(); + }); + }); + + describe('SSH public keys', () => { + const keyRecord = { + key: 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAA-test', + name: 'work laptop', + addedAt: '2026-01-01T00:00:00.000Z', + fingerprint: 'SHA256:abc123', + }; + + const userRow = (overrides: Record = {}) => ({ + _id: 'u1', + username: 'alice', + email: 'alice@example.com', + password: null, + git_account: 'alice-git', + admin: false, + oidc_id: null, + public_keys: [], + display_name: null, + title: null, + ...overrides, + }); + + describe('findUserBySSHKey', () => { + it('queries with JSONB containment on the key', async () => { + mockQuery.mockResolvedValue({ rowCount: 0, rows: [] }); + const user = await findUserBySSHKey(keyRecord.key); + const [sql, params] = mockQuery.mock.calls[0]; + expect(sql).toContain('public_keys @> $1::jsonb'); + expect(params).toEqual([JSON.stringify([{ key: keyRecord.key }])]); + expect(user).toBeNull(); + }); + + it('maps public_keys onto the returned User', async () => { + mockQuery.mockResolvedValue({ + rowCount: 1, + rows: [userRow({ public_keys: [keyRecord] })], + }); + const user = await findUserBySSHKey(keyRecord.key); + expect(user?.publicKeys).toEqual([keyRecord]); + }); + }); + + describe('addPublicKey', () => { + it('appends the key to the user public_keys array', async () => { + mockQuery + .mockResolvedValueOnce({ rowCount: 0, rows: [] }) // findUserBySSHKey + .mockResolvedValueOnce({ rowCount: 1, rows: [userRow()] }) // findUser + .mockResolvedValueOnce({ rowCount: 1, rows: [] }); // UPDATE + + await addPublicKey('Alice', keyRecord); + + const [sql, params] = mockQuery.mock.calls[2]; + expect(sql).toContain('public_keys = public_keys || $2::jsonb'); + expect(params).toEqual(['alice', JSON.stringify([keyRecord])]); + }); + + it('throws DuplicateSSHKeyError when the key belongs to another user', async () => { + mockQuery.mockResolvedValueOnce({ + rowCount: 1, + rows: [userRow({ username: 'bob', public_keys: [keyRecord] })], + }); + + await expect(addPublicKey('alice', keyRecord)).rejects.toThrow( + "SSH key already in use by user 'bob'", + ); + expect(mockQuery).toHaveBeenCalledTimes(1); + }); + + it('allows re-checking a key that already maps to the same user', async () => { + mockQuery.mockResolvedValueOnce({ + rowCount: 1, + rows: [userRow({ public_keys: [keyRecord] })], + }); + mockQuery.mockResolvedValueOnce({ + rowCount: 1, + rows: [userRow({ public_keys: [keyRecord] })], + }); + + await expect(addPublicKey('ALICE', keyRecord)).rejects.toThrow('SSH key already exists'); + }); + + it('throws when the user does not exist', async () => { + mockQuery + .mockResolvedValueOnce({ rowCount: 0, rows: [] }) // findUserBySSHKey + .mockResolvedValueOnce({ rowCount: 0, rows: [] }); // findUser + + await expect(addPublicKey('ghost', keyRecord)).rejects.toThrow('User not found'); + }); + + it('throws when the fingerprint already exists for the user', async () => { + const existing = { ...keyRecord, key: 'ssh-ed25519 DIFFERENT-KEY' }; + mockQuery + .mockResolvedValueOnce({ rowCount: 0, rows: [] }) + .mockResolvedValueOnce({ rowCount: 1, rows: [userRow({ public_keys: [existing] })] }); + + await expect(addPublicKey('alice', keyRecord)).rejects.toThrow('SSH key already exists'); + }); + }); + + describe('removePublicKey', () => { + it('filters the fingerprint out of public_keys and lower-cases username', async () => { + mockQuery.mockResolvedValue({ rowCount: 1, rows: [] }); + + await removePublicKey('Alice', keyRecord.fingerprint); + + const [sql, params] = mockQuery.mock.calls[0]; + expect(sql).toContain(`(k->>'fingerprint') IS DISTINCT FROM $2`); + expect(params).toEqual(['alice', keyRecord.fingerprint]); + }); + }); + + describe('getPublicKeys', () => { + it('returns the user public keys', async () => { + mockQuery.mockResolvedValue({ + rowCount: 1, + rows: [userRow({ public_keys: [keyRecord] })], + }); + await expect(getPublicKeys('alice')).resolves.toEqual([keyRecord]); + }); + + it('returns [] when the column is null', async () => { + mockQuery.mockResolvedValue({ rowCount: 1, rows: [userRow({ public_keys: null })] }); + await expect(getPublicKeys('alice')).resolves.toEqual([]); + }); + + it('throws when the user does not exist', async () => { + mockQuery.mockResolvedValue({ rowCount: 0, rows: [] }); + await expect(getPublicKeys('ghost')).rejects.toThrow('User not found'); + }); + }); + }); +}); diff --git a/test/setup-integration-postgres.ts b/test/setup-integration-postgres.ts new file mode 100644 index 000000000..b9c03b2e9 --- /dev/null +++ b/test/setup-integration-postgres.ts @@ -0,0 +1,100 @@ +/** + * Copyright 2026 GitProxy Contributors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import { beforeAll, afterAll, afterEach } from 'vitest'; +import { Client } from 'pg'; + +import { resetConnection } from '../src/db/postgres/helper'; +import { invalidateCache } from '../src/config'; + +const DEFAULT_CONNECTION_STRING = 'postgresql://postgres:postgres@localhost:5432/git_proxy_test'; +const APP_TABLES = ['pushes', 'repos', 'users']; +const SESSION_TABLE = 'session'; +// Tracks applied schema versions. Persisted across tests (so the migration +// runner correctly skips already-applied versions) but dropped in afterAll so +// a re-run against the same database starts from a clean slate. +const MIGRATIONS_TABLE = 'schema_migrations'; + +let client: Client | null = null; + +const getConnectionString = () => + process.env.GIT_PROXY_POSTGRES_CONNECTION_STRING || DEFAULT_CONNECTION_STRING; + +const shouldRun = () => process.env.RUN_POSTGRES_TESTS === 'true'; + +beforeAll(async () => { + if (!shouldRun()) return; + + try { + client = new Client({ connectionString: getConnectionString() }); + await client.connect(); + console.log(`PostgreSQL connection established for integration tests`); + } catch (error) { + console.error('Failed to connect to PostgreSQL:', error); + throw error; + } +}); + +afterEach(async () => { + if (client) { + // Truncate app tables so each test starts from a known clean state. + // RESTART IDENTITY isn't needed (UUID PKs), but CASCADE keeps us future- + // proof in case a follow-up commit adds FK relationships. + try { + await client.query(`TRUNCATE TABLE ${APP_TABLES.join(', ')} CASCADE`); + } catch (error) { + console.warn('Failed to truncate app tables during integration test cleanup', error); + } + try { + // The session table is created lazily by connect-pg-simple; ignore the + // error if it does not yet exist. + await client.query(`TRUNCATE TABLE "${SESSION_TABLE}"`); + } catch { + // intentionally swallowed — table may not exist yet + } + } + + try { + await resetConnection(); + } catch (error) { + console.warn('Failed to reset Postgres pool during integration test cleanup', error); + } + invalidateCache(); +}); + +afterAll(async () => { + try { + await resetConnection(); + } catch (error) { + console.warn('Failed to reset Postgres pool during integration test cleanup', error); + } + + if (client) { + try { + for (const table of APP_TABLES) { + await client.query(`DROP TABLE IF EXISTS ${table} CASCADE`); + } + await client.query(`DROP TABLE IF EXISTS ${MIGRATIONS_TABLE} CASCADE`); + await client.query(`DROP TABLE IF EXISTS "${SESSION_TABLE}"`); + } catch (error) { + console.warn('Failed to drop Postgres test tables during cleanup', error); + } + await client.end(); + client = null; + } + + console.log('PostgreSQL integration test cleanup complete'); +}); diff --git a/vitest.config.integration.postgres.ts b/vitest.config.integration.postgres.ts new file mode 100644 index 000000000..ec51a96a9 --- /dev/null +++ b/vitest.config.integration.postgres.ts @@ -0,0 +1,43 @@ +/** + * Copyright 2026 GitProxy Contributors + * + * Licensed under the Apache License, Version 2.0 (the "License"); + * you may not use this file except in compliance with the License. + * You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +import path from 'path'; +import { defineConfig } from 'vitest/config'; + +export default defineConfig({ + test: { + include: ['test/db/postgres/**/*.integration.test.ts'], + testTimeout: 30000, + hookTimeout: 10000, + setupFiles: ['test/setup-integration-postgres.ts'], + pool: 'forks', + poolOptions: { + forks: { + singleFork: true, + }, + }, + env: { + NODE_ENV: 'test', + RUN_POSTGRES_TESTS: 'true', + CONFIG_FILE: path.resolve(__dirname, 'test-integration.postgres.proxy.config.json'), + // Default for local runs; an exported GIT_PROXY_POSTGRES_CONNECTION_STRING + // (e.g. in CI or a non-default local setup) takes precedence. + GIT_PROXY_POSTGRES_CONNECTION_STRING: + process.env.GIT_PROXY_POSTGRES_CONNECTION_STRING || + 'postgresql://postgres:postgres@localhost:5432/git_proxy_test', + }, + }, +}); diff --git a/website/docs/architecture/architecture.md b/website/docs/architecture/architecture.md index 2efe2fb9c..9b3834dcc 100644 --- a/website/docs/architecture/architecture.md +++ b/website/docs/architecture/architecture.md @@ -484,10 +484,64 @@ Sample values: #### `sink` -List of database sources. The first source with `enabled` set to `true` will be used. Currently, MongoDB and filesystem databases ([NeDB](https://www.npmjs.com/package/@seald-io/nedb)) are supported. By default, the filesystem database is used. +List of database sources. The first source with `enabled` set to `true` will be used. GitProxy supports three sink backends: + +- **`fs`** — filesystem-backed [NeDB](https://www.npmjs.com/package/@seald-io/nedb). Default. Suitable for single-process deployments. +- **`mongo`** — MongoDB via `connect-mongo` for session storage. +- **`postgres`** — PostgreSQL via [`pg`](https://node-postgres.com/) + [`connect-pg-simple`](https://github.com/voxpelli/node-connect-pg-simple) for session storage. Each entry has its own unique configuration parameters. +##### PostgreSQL configuration + +The `postgres` backend stores `users`, `repos`, `pushes`, and the `connect-pg-simple` `session` table in a single PostgreSQL database. The required tables are created and kept up to date on startup by a built-in versioned migration runner (see [Schema migrations](#schema-migrations) below), so pointing the proxy at an empty database is enough to get running. + +```json +{ + "sink": [ + { + "type": "postgres", + "connectionString": "postgresql://user:pass@host:5432/gitproxy", + "enabled": true + } + ] +} +``` + +If `connectionString` is omitted on the config entry, GitProxy falls back to the `GIT_PROXY_POSTGRES_CONNECTION_STRING` environment variable. This mirrors the behaviour of the mongo backend's `GIT_PROXY_MONGO_CONNECTION_STRING`. + +##### Schema migrations + +Schema changes are applied by a small built-in migration runner (`src/db/postgres/migrations.ts`). On every startup it: + +- ensures a `schema_migrations` bookkeeping table exists, +- takes a transaction-scoped advisory lock so concurrently starting processes do not race, and +- applies any migrations whose version has not been recorded yet, in order, recording each as it goes. + +Migrations are an ordered, append-only list of SQL statements defined in code. Version 1 is the initial schema; because it uses `CREATE TABLE IF NOT EXISTS`, databases that were bootstrapped by earlier releases adopt the runner transparently (version 1 is simply recorded). To evolve the schema, append a new entry with the next version number; never edit or reorder migrations that have already shipped. + +Notes and current limitations: + +- All pending migrations run inside a single transaction, so a statement that cannot run transactionally (for example `CREATE INDEX CONCURRENTLY`) is not yet supported by the runner. +- Repo permissions (`canPush` / `canAuthorise`) are stored as a JSONB column on the `repos` table. A future PR may normalise these into a `repo_users` join table. +- No data migration utility from `fs` or `mongo` to `postgres` — copy data yourself if needed. +- No AWS RDS IAM authentication helper (the mongo backend has one via `AWS_CREDENTIAL_PROVIDER`); use a standard connection string for v1. +- Only the `connectionString` form is supported; split `PGHOST`/`PGPORT`/`PGUSER`/`PGPASSWORD`/`PGDATABASE` env vars are not consulted. +- If `postgres` is selected as the active sink and the connection string cannot be resolved, GitProxy refuses to start rather than silently falling back to an in-memory session store. + +##### PostgreSQL design decisions + +The adapter follows a few deliberate choices, made for parity with the existing backends rather than for idiomatic SQL: + +- **Pushes stay documents.** A push is an audit record: written once, updated through a handful of state flips, and read back whole. The `pushes` table therefore keeps the entire action as a JSONB `data` column, with typed columns (`timestamp`, the status booleans) only for the fields that queries filter and sort on. This mirrors how the mongo and NeDB backends treat pushes and keeps the row shape stable as the `Action` type evolves. +- **Users and repos are typed rows with JSONB edges.** Fields that queries touch get real columns; genuinely document-shaped parts (a user's `publicKeys`, the repo permission map) are JSONB. +- **Identifiers are server-generated UUIDs** (`gen_random_uuid()`), the SQL analogue of mongo's ObjectIds. No compatibility between the backends' id formats is assumed anywhere in the app. +- **Timestamps the app treats as strings stay strings.** `dateCreated` and `lastModified` are ISO-8601 `TEXT` columns so values round-trip byte-for-byte identically to the mongo and NeDB backends, with no timezone conversion on the way through. +- **Same case rules as mongo**: usernames are lowercased on permission changes, and repo name lookups are lowercase. +- **Email uniqueness is best-effort**, enforced by a partial unique index: any number of users may have no email (the ActiveDirectory `mail` attribute is optional), while a real address can only be claimed once. This matches the permissive behaviour of the other backends. +- **Sessions use `connect-pg-simple`**, the postgres counterpart of the mongo backend's `connect-mongo` session store. +- **Failures are loud.** If `postgres` is the active sink and no connection can be resolved, GitProxy refuses to start rather than silently degrading to an in-memory session store. + Extending GitProxy to support other databases requires adding the relevant handlers and setup to the [`/src/db`](https://github.com/finos/git-proxy/blob/main/src/db/) directory. Feel free to [open an issue](https://github.com/finos/git-proxy/issues) requesting support for any specific databases - or [open a PR](https://github.com/finos/git-proxy/pulls) with the desired changes! #### `authentication`