Skip to content

ericprud/SWObjects

Repository files navigation

Hi, welcome to SWObjects and all that. You probably want to:

  cmake -B build
  cmake --build build -j8        # tools land in bin/, e.g. bin/sparql
  ctest --test-dir build         # run the test suites

Feature selection that used to live in the CONFIG file is now done with
cmake options (see the options block at the top of CMakeLists.txt):

  cmake -B build -DSWOBJ_XML_PARSER=LIBXML2 -DSWOBJ_SQL_MYSQL=ON

Some test suites (test_Turtle, test_Trig, test_RDFXML, test_SPARQL11)
expect the W3C test data unpacked under tests/ ; fetch it with:

  sh tests/fetch-test-suites.sh

CI (.github/workflows/ci.yml) runs the same steps on Linux and macOS.

# Debian instructions

you'll need these packages:

- cmake
- flex (>= 2.6) and libfl-dev (FlexLexer.h)
- bison (>= 3.2)
- uuid-dev
- libexpat1-dev (or libxml2-dev with -DSWOBJ_XML_PARSER=LIBXML2)
- libmysqlclient-dev (only with -DSWOBJ_SQL_MYSQL=ON)
- libboost-dev
- libboost-log-dev
- libboost-regex-dev
- libboost-test-dev
- libboost-program-options-dev
- libboost-iostreams-dev
- libboost-thread-dev
- libboost-filesystem-dev

On macOS: brew install cmake boost bison flex expat
(cmake prefers Homebrew's bison/flex over the older /usr/bin ones.)

# ShEx

lib/ShExCParser.ypp and lib/ShExSchema.{hpp,cpp} implement ShEx 2.1
(ShExC syntax including IMPORT, ABSTRACT and EXTENDS; validation by
interval-based matching over SORBE-normalized triple expressions, after
jena-shex). EXTERNAL shapes resolve through ShEx::ExternalResolver
(e.g. against a .shextern schema). lib/ShExShapeMap.{hpp,cpp} parses
fixed and query shape maps (node@shape, node@!shape, START,
{FOCUS <p> _} patterns) and the shexTest JSON map format.
tests/test_ShEx.cpp runs the shexSpec/shexTest suite (schemas,
negativeSyntax, negativeStructure and validation manifests);
tests/fetch-test-suites.sh clones the test data.

bin/sparql runs shex-webapp-style validation manifests (JSON or YAML;
see lib/ShExManifest.hpp and tests/ShExManifest/):

  bin/sparql --shex-manifest tests/ShExManifest/manifest.yaml
  bin/sparql --shex-manifest m.json --shex-run '2-5,0'

--shex-run selects 0-based entries: ranges like "1", "2-5", "5-2"
(descending) or "*" (everything), comma-separated.

lib/ShExMap.{hpp,cpp} implements the http://shex.io/extensions/Map/#
semantic-action language for mapping between two schemas: validating
against a source schema whose constraints carry %Map:{ var %} (or
regex(...)/hashmap(...) lift functions) collects variable bindings,
and materialization walks a target schema emitting the bound (or
lowered) values. Manifest entries drive it with outputSchema[URL],
outputShape, createRoot, staticVars and optional expectedBindingsURL /
outputDataURL comparisons (see tests/ShExMap/, ported from shex.js's
packages/extension-map examples):

  bin/sparql --shex-manifest tests/ShExMap/manifest.yaml

--shex-endpoint validates a shape map against a *live* SPARQL protocol
endpoint instead of an in-memory graph: node neighborhoods are fetched
and cached as validation visits them (lib/BNodeResolver.hpp's
RemoteGraphProvider wired in as Validator::NeighborhoodSource), so
response blank nodes crossing the round trips this needs are told-bnode-
safe (see the algae3 repo's doc/told-bnodes.md). Fixed associations and
ONE query pattern are both supported — the pattern's FOCUS candidates
are fetched with a single additional round trip
(lib/ShExShapeMap.hpp's PatternMatcher abstracts this from the existing
local-graph matching, now LocalPatternMatcher). Demo (from tests/, with
bin/sparql itself serving the evidence data):

  ../bin/sparql --serve http://127.0.0.1:8099/sparql \
      -d Algae3/data/evidence.ttl -d Algae3/data/evidence-groups.ttl
  ../bin/sparql --shex-endpoint http://127.0.0.1:8099/sparql \
      --shex-schema schema.shex \
      --shex-map 'evgrp:PeerReviewedCohorts@<GroupShape>'
  # or validate every node matching a pattern in one go:
  ../bin/sparql --shex-endpoint http://127.0.0.1:8099/sparql \
      --shex-schema schema.shex --shex-map '{FOCUS ex:member _}@<GroupShape>'

Node-side prefixes in --shex-map borrow the schema's own PREFIX
declarations (there's no separate "data document" to carry its own).
"@!shape" asserts the node should NOT conform (exit code counts
mismatches, matching --shex-manifest's PASS/FAIL convention).

# Algae 3

lib/Algae3Parser.ypp, lib/Algae3Scanner.lpp and lib/Algae3.{hpp,cpp}
implement Algae 3, a pipeline-of-actions query language (a modernized
Algae2); bin/algae3 is the CLI. The language itself — data model,
actions, pattern combinators, the two evaluation modes — is specified
in the proposal repo, https://github.com/ericprud/algae3 (doc/).

Run the examples from tests/ (load paths are relative):

  cd tests
  ../bin/algae3 Algae3/evidence-dnf.topdown.a3     # the flagship (below)
  ../bin/algae3 --mode bottomup Algae3/evidence-dnf.bottomup.a3
  ../bin/algae3 --proofs Algae3/rule.a3            # per-row provenance
  ../bin/algae3 --algebra Algae3/connectives.a3    # print the algebra tree
  ../bin/algae3 --debug Algae3/negation-scoped-topdown.a3   # REPL debugger
                # step/next/into/where/print/bindings/watch/try/mode/break/run/reset
  ../bin/algae3 --serve 8080 Algae3/evidence-dnf.topdown.a3
                # then open SPARQL-explorer.html (repo root): paste a raw
                # .rq and sparql-to-a3.js compiles it to either mode and
                # steps the evaluation visually

tests/test_Algae3.cpp is the suite (golden results plus differential
topdown-vs-bottomup runs); tests/Algae3/regen-compiled.mjs regenerates
the compiled .a3 forms from the .rq sources via sparql-to-a3.js.

The flagship query, tests/Algae3/evidence-dnf.rq, is a DNF rule engine
abstracted from a production query: a rule fires when ALL selectors
pass, a selector when ANY disjunct passes, a disjunct when ALL its
conjuncts pass — universal quantification by counting, at two levels,
with the proof log assembled by nested GROUP_CONCATs:

  SELECT (STRAFTER(STR(?rule), "#") AS ?ruleHash) ?message ?evidences ?log
  FROM <data/evidence-rules.ttl>
  FROM <data/evidence-groups.ttl>
  FROM <data/evidence.ttl>
  WHERE { {
    ?rule a ex:Rule ; ex:action [ ex:message ?message ] .
    { SELECT ?rule (COUNT(?selector) AS ?selectorsPassed) … (…AS ?log) …
      WHERE {
        ?rule ex:selector ?selector .
        { SELECT ?evidence ?selector … WHERE {
            ?selector ex:or ?disjunct
            { SELECT ?evidence ?disjunct (COUNT(?conjunct) AS ?conjunctsPassed) …
              WHERE {
                ?disjunct ex:and ?conjunct .
                {   # conjunct kinds: group membership (via bnode
                    # memberships!), attribute comparison, type test —
                    # each with positive and MINUS-based negative polarity
                    …
                } } GROUP BY ?evidence ?disjunct }
            { SELECT ?disjunct (COUNT(*) AS ?conjunctCount) WHERE {
                ?disjunct ex:and ?conjunct . } GROUP BY ?disjunct }
            FILTER (?conjunctsPassed = ?conjunctCount)   # ALL conjuncts
        } } GROUP BY ?rule } }
    { SELECT ?rule (COUNT(*) AS ?selectorCount) WHERE {
        ?rule ex:selector ?selector . } GROUP BY ?rule }
    FILTER (?selectorCount = ?selectorsPassed)           # ALL selectors
  } }
  ORDER BY ?message ?evidences ?ruleHash

(abridged — see the file for the runnable whole). Expected: two rows,
"confound-sensitive" firing on OBS-2214 and "underpowered" on
OBS-2214 + RNC-0859.1, each with a log like
confound(confound_selector0(OBS-2214, …or0(…and0∧…)∨…or1(…))).
Both compiled forms agree across both evaluation modes.

# Told bnodes (BNodeResolver)

lib/BNodeResolver.{hpp,cpp} emulates "told bnodes" (rejected in SPARQL
1.0): executing over a SPARQL endpoint, each response blank node is
identified by a minimal uniqueness-verified fragment of in/out arcs
(found by instrumenting the query with interrogation round trips,
recursing through neighbor bnodes) and interned to a stable proxy that
later queries can mention. RemoteGraphProvider caches fetched triple
patterns with subsumption and feeds both ShEx validation over an
endpoint (Validator::NeighborhoodSource) and remote Algae 3 execution
(Engine::remote). The algorithm is written up in the algae3 repo,
doc/told-bnodes.md.

Run it:

  ctest --test-dir build -R BNodeResolver     # or:
  cd tests && ../build/test_BNodeResolver

The suite runs everything against a client that freshly relabels every
response bnode, so nothing can pass by label luck. Its flagship case
executes evidence-dnf.topdown.a3 (the query above, whose group
memberships are anonymous bnodes) purely through such a client — no
local load — and requires the exact local golden results, proof logs
included; a second copy of that case drives an in-process HTTP
endpoint through the real protocol client.

`attach` in an .a3 script runs against a live endpoint (bin/algae3
installs an HTTPSPARQLClient factory; plain http — the ASIO agent
doesn't speak TLS). Demo with bin/sparql serving the evidence data
(from tests/):

  ../bin/sparql --serve http://127.0.0.1:8098/sparql \
      -d Algae3/data/evidence.ttl \
      -d Algae3/data/evidence-groups.ttl \
      -d Algae3/data/evidence-rules.ttl &
  sed 's|^load .*|attach <http://127.0.0.1:8098/sparql> ep|' \
      Algae3/evidence-dnf.topdown.a3 | awk '!/^attach/ || !seen++' > /tmp/attach.a3
  ../bin/algae3 /tmp/attach.a3        # == the local results, proof logs included

Endpoint data should use absolute IRIs: relative ones resolve
differently between data load and query parse, so they cannot survive
the protocol round trip (the evidence fixtures carry explicit absolute
bases for exactly this reason).

# More info

To learn more about SWObjects, checkout some tutorials given at SWAT4LS in 2010 and 2011:
2010: http://www.w3.org/2010/Talks/1208-egp-swobjects/
2011: https://sites.google.com/site/swobjectstutorial/documentation

About

Semantic Web swiss army knife C++ libraries

Resources

License

MIT, Unknown licenses found

Licenses found

MIT
LICENSE
Unknown
COPYING

Stars

15 stars

Watchers

8 watching

Forks

Releases

No releases published

Packages

 
 
 

Contributors