Skip to content

Support declaring columns in fromSql to get a typed query usable as a table (#902) - #912

Merged
IlyaSemenov merged 1 commit into
romeerez:mainfrom
IlyaSemenov:IlyaSemenov/typed-raw-sql-relation-usable-as-a-query-source
Nov 17, 2026
Merged

IlyaSemenov merged 1 commit into
romeerez:mainfrom
IlyaSemenov:IlyaSemenov/typed-raw-sql-relation-usable-as-a-query-source

Conversation

@IlyaSemenov

@IlyaSemenov IlyaSemenov commented Nov 17, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

Fixes #902.

fromSql accepts a column shape, like withSql, and returns a typed query that works as a table. The keys of the declared shape name the SQL result columns by position:

const score = db.$qb
  .fromSql(
    (t) => ({ authorId: t.integer(), points: t.integer() }),
    sql`SELECT author_id, sum(points)::int FROM vote GROUP BY author_id`,
  )
  .as('score');

// SELECT * FROM (SELECT ...) "score"("authorId", "points") WHERE "score"."points" > $1
score.where({ points: { gt: 5 } });

db.author.join(score, 'score.authorId', 'id').select('name', 'score.points');
db.author.whereExists(score, 'score.authorId', 'id'); // threw before
db.author.select('name', {
  score: (q) => score.where({ authorId: q.ref('id') }).take(),
});

Two fixes along the way:

// before: (SELECT row_to_json(t.*) FROM (...)), fails with missing FROM-clause entry for table "t"
// after:  (SELECT row_to_json(t.*) FROM (...) "t")
db.author.select({ vote: () => db.$qb.from(db.vote.select('points')).take() });

// before: WHERE "v"."a" = "author"."id", a column the table doesn't have
// after:  WHERE "v"."author_id" = "author"."id"
db.author.whereExists(db.vote.select({ a: 'authorId' }).as('v'), 'v.a', 'id');

Motivation

Another follow-up to #904, #905, and #909, for the same ACL optimization. One relation, a UNION ALL of several sources, backs many access filters across tables as EXISTS and IN conditions, some of them correlated with an outer row. Without a typed standalone query, all of it had to be written as raw SQL with hand-picked aliases.

Verification

Added database-backed tests for fromSql with a shape: filtering, ordering, and parsing, and use in join, whereExists (with arguments and with a callback), whereIn, and a select sub-query.

Called on a table, the tests cover a renamed column (group, a qualified get, the default alias) and a decimal column (JSON of a sub-query, also after a later from). Other edge cases are a column with name(), a quoted column name, and an empty shape.

@ts-expect-error covers an undeclared column and fromSql after select.

The shared whereExists tests for a sub-query with selected aliases expected "as"."one", a column the table doesn't have; they now expect the underlying column.

pnpm verify passes. Tests ran with postgres-js; other adapters and Bun were not checked.

TypeScript instantiations, tsc --noEmit --extendedDiagnostics in the package directory, the base commit vs. this PR without its test changes:

  • pqb: 2,333,680 → 2,335,888 (+2,208, +0.09%), check time 0.69 s → 0.69 s.
  • orm: 2,439,373 → 2,442,632 (+3,259, +0.13%), check time 0.73–0.80 s → 0.73–0.78 s, within run-to-run noise.

The new overload and its result type are evaluated only where fromSql is called with a shape.

Checklist

  • I have read the final diff and can explain every change.
  • I will address review feedback.
  • I added a changeset (pnpm changeset) if this PR includes a feature or bug fix in a published package.

@netlify

netlify Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

👷 Deploy request for orchid-orm pending review.

Visit the deploys page to approve it

Name Link
🔨 Latest commit a1fbb0c

@IlyaSemenov
IlyaSemenov force-pushed the IlyaSemenov/typed-raw-sql-relation-usable-as-a-query-source branch from 93a0960 to a1fbb0c Compare November 17, 2026 07:40
@IlyaSemenov

Copy link
Copy Markdown
Collaborator Author

I won't claim I fully understand the new code. I checked that it looks reasonable, it's isolated, it's covered by tests, and I had it reviewed multiple times by different AIs. I also measured instantiations, which surfaced a problem: originally I put the new code into fromAndAs.sql.ts instead of a separate module, and that alone added ~18k instantiations. It's not on a hot path, it's a side feature, so I suppose it's enough for now.

The surrounding code is buggy, see #914, #915, #916, and the PR deliberately leaves holes, e.g.:

  • from([...]) with several sources still builds the JSON of a nested selection from the columns of the table itself, as before; this PR only stops a preceding fromSql from leaking into it. Building it from the sources ran into duplicate column names, sources with unknown columns, and decimals turning into numbers, so I left it as is.
  • The untyped fromSql on a table still aliases its source as "t", while its types use the table name (fromSql on a table names its source t, while its types use the table name #915). Only the new typed form defaults to the table name, to avoid changing the SQL of existing queries.

@IlyaSemenov
IlyaSemenov merged commit 1e5494b into romeerez:main Nov 17, 2026
3 checks passed
@IlyaSemenov
IlyaSemenov deleted the IlyaSemenov/typed-raw-sql-relation-usable-as-a-query-source branch November 17, 2026 07:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Typed raw SQL relation usable as a query source, without a CTE

1 participant