Skip to content

Missing sha1 digests from nexus-maven-repository-index.gz #692

Closed
Closed
Missing sha1 digests from nexus-maven-repository-index.gz#692
Labels
bugSomething isn't working

Description

@ariel-miculas
ariel-miculas
opened on Jul 18, 2025

Affected version

latest

Bug description

There are some sha1 digests missing from nexus-maven-repository-index.gz
One such example is: 92b6631659ba35ca09e44874d3eb936edfeee532 related to jakarta.validation-api

$ curl https://search.maven.org/solrsearch/select\?q\="1:92b6631659ba35ca09e44874d3eb936edfeee532"

{"responseHeader":{"status":0,"QTime":111,"params":{"q":"1:92b6631659ba35ca09e44874d3eb936edfeee532","core":"","indent":"off","fl":"id,g,a,v,p,ec,timestamp,tags","start":"","sort":"score desc,timestamp desc,g asc,a asc,v desc","rows":"20","wt":"json","version":"2.2"}},"response":{"numFound":1,"start":0,"docs":[{"id":"jakarta.validation:jakarta.validation-api:3.0.2","g":"jakarta.validation","a":"jakarta.validation-api","v":"3.0.2","p":"jar","timestamp":1653588099000,"ec":["-sources.jar",".pom","-javadoc.jar",".jar"],"tags":["jakarta","bean","validation"]}]}}%

Looking at the index with Luke I get the following results when searching for g:jakarta.validation v:3.0.2:

17136615	8.715791	u=jakarta.validation|jakarta.validation-api|3.0.2|sources|jar.asc.sha512;
17136616	8.715791	1=b1581d1ef516be94c473c0f4d97e59e394c70150; u=jakarta.validation|jakarta.validation-api|3.0.2|sources|jar;
17136617	8.715791	1=eb8bb58bbce6761e2e8095f48f383da117fb109c; u=jakarta.validation|jakarta.validation-api|3.0.2|javadoc|jar;
62019267	8.715791	u=jakarta.validation|jakarta.validation-api|3.0.2|NA|pom.asc.sha512;
62019268	8.715791	u=jakarta.validation|jakarta.validation-api|3.0.2|sources|jar.asc.sha256;
62019269	8.715791	u=jakarta.validation|jakarta.validation-api|3.0.2|javadoc|jar.asc.sha512;
62019270	8.715791	u=jakarta.validation|jakarta.validation-api|3.0.2|javadoc|jar.asc.sha256;

Here, I can only see the sha1sums for sources.jar and javadoc.jar, but not for the main jar with digest 92b6631659ba35ca09e44874d3eb936edfeee532
Image

It may be related to this other IndexSearcher issue.

What I also tried was running the latest version of the indexer on the following directory structure:

$ tree jakarta
jakarta
└── validation
    └── jakarta.validation-api
        └── 3.0.2
            ├── jakarta.validation-api-3.0.2-javadoc.jar
            ├── jakarta.validation-api-3.0.2-javadoc.jar.asc
            ├── jakarta.validation-api-3.0.2-javadoc.jar.asc.md5
            ├── jakarta.validation-api-3.0.2-javadoc.jar.asc.sha1
            ├── jakarta.validation-api-3.0.2-javadoc.jar.asc.sha256
            ├── jakarta.validation-api-3.0.2-javadoc.jar.asc.sha512
            ├── jakarta.validation-api-3.0.2-javadoc.jar.md5
            ├── jakarta.validation-api-3.0.2-javadoc.jar.sha1
            ├── jakarta.validation-api-3.0.2-sources.jar
            ├── jakarta.validation-api-3.0.2-sources.jar.asc
            ├── jakarta.validation-api-3.0.2-sources.jar.asc.md5
            ├── jakarta.validation-api-3.0.2-sources.jar.asc.sha1
            ├── jakarta.validation-api-3.0.2-sources.jar.asc.sha256
            ├── jakarta.validation-api-3.0.2-sources.jar.asc.sha512
            ├── jakarta.validation-api-3.0.2-sources.jar.md5
            ├── jakarta.validation-api-3.0.2-sources.jar.sha1
            ├── jakarta.validation-api-3.0.2.jar
            ├── jakarta.validation-api-3.0.2.jar.asc
            ├── jakarta.validation-api-3.0.2.jar.asc.md5
            ├── jakarta.validation-api-3.0.2.jar.asc.sha1
            ├── jakarta.validation-api-3.0.2.jar.asc.sha256
            ├── jakarta.validation-api-3.0.2.jar.asc.sha512
            ├── jakarta.validation-api-3.0.2.jar.md5
            ├── jakarta.validation-api-3.0.2.jar.sha1
            ├── jakarta.validation-api-3.0.2.pom
            ├── jakarta.validation-api-3.0.2.pom.asc
            ├── jakarta.validation-api-3.0.2.pom.asc.md5
            ├── jakarta.validation-api-3.0.2.pom.asc.sha1
            ├── jakarta.validation-api-3.0.2.pom.asc.sha256
            ├── jakarta.validation-api-3.0.2.pom.asc.sha512
            ├── jakarta.validation-api-3.0.2.pom.md5
            └── jakarta.validation-api-3.0.2.pom.sha1

4 directories, 32 files

$ java -jar indexer-cli/target/indexer-cli-7.1.7-SNAPSHOT-cli.jar -X -i /tmp/jakarta-validation-download/.index -d /tmp/jakarta-validation-download/.index -r /tmp/jakarta-validation-download
Version: 7.1.7-SNAPSHOT
Repository Folder: /tmp/jakarta-validation-download
Index Folder:      /tmp/jakarta-validation-download/.index
Output Folder:     /tmp/jakarta-validation-download/.index
Repository name:   .index
Indexers: [min, jarContent]
Will not create checksum files.
Will create baseline file.
Scanning started
       1 jakarta/validation/jakarta.validation-api/3.0.2/jakarta.validation-api-3.0.2.pom
       2 jakarta/validation/jakarta.validation-api/3.0.2/jakarta.validation-api-3.0.2.pom
       3 jakarta/validation/jakarta.validation-api/3.0.2/jakarta.validation-api-3.0.2.pom
       4 jakarta/validation/jakarta.validation-api/3.0.2/jakarta.validation-api-3.0.2.pom
Artifacts added:   4
Artifacts deleted: 0
[main] INFO org.apache.maven.index.packer.DefaultIndexPacker - Unable to read properties file, will force index regeneration
Total time:   0 sec
Final memory: 29M/260M

And I got the following error when unpacking the index:

$ java -jar ~/work/maven-indexer/indexer-cli/target/indexer-cli-7.1.7-SNAPSHOT-cli.jar -u
Index Folder:      /private/tmp/jakarta-validation-download/.index
Output Folder:     /private/tmp/jakarta-validation-download/.index
java.lang.IllegalArgumentException: cannot change field "m" from index options=NONE to inconsistent index options=DOCS_AND_FREQS_AND_POSITIONS
        at org.apache.lucene.index.FieldInfo.verifySameIndexOptions(FieldInfo.java:273)
        at org.apache.lucene.index.FieldInfos$FieldNumbers.verifySameSchema(FieldInfos.java:567)
        at org.apache.lucene.index.FieldInfos$FieldNumbers.addOrGet(FieldInfos.java:476)
        at org.apache.lucene.index.IndexWriter.addIndexes(IndexWriter.java:3074)
        at org.apache.maven.index.updater.IndexDataReader.readIndexMT(IndexDataReader.java:268)
        at org.apache.maven.index.updater.IndexDataReader.readIndex(IndexDataReader.java:132)
        at org.apache.maven.index.updater.DefaultIndexUpdater.unpackIndexData(DefaultIndexUpdater.java:344)
        at org.apache.maven.index.updater.DefaultIndexUpdater.unpackIndexData(DefaultIndexUpdater.java:324)
        at org.apache.maven.index.cli.NexusIndexerCli.unpack(NexusIndexerCli.java:500)
        at org.apache.maven.index.cli.NexusIndexerCli.execute(NexusIndexerCli.java:157)
        at org.apache.maven.index.cli.NexusIndexerCli.main(NexusIndexerCli.java:116)

I'm curious to know what the search API does in order to find this digest that isn't present in the central maven index and what I can do in order to accurately get all the sha1sums identifying the java packages.

Activity

  1. ariel-miculas
    added
    bugSomething isn't working
    on Jul 18, 2025
  2. ariel-miculas
    added a commit that references this issue on Jul 23, 2025

    Include missing sha1 digests in the index

    514b4ee

  3. cstamas commented on Jul 30, 2025

    @cstamas
    cstamas
    on Jul 30, 2025
    Member

    Yes, and sadly index publishing is totally out of our control. It is done by Sonatype with unknown version (to us. probably ancient) of Maven Indexer. @mbien @brianf

  4. ariel-miculas commented on Aug 7, 2025

    @ariel-miculas
    ariel-miculas
    on Aug 7, 2025
    ContributorAuthor

    So what would be the way forward? I guess the first step is to get my PR merged, then wait for a new release of the maven indexer.
    Do you have any contacts at Sonatype that could clarify what version of the indexer they're using? And then convince them to use the latest one?

  5. cstamas commented on Aug 7, 2025

    @cstamas
    cstamas
    on Aug 7, 2025
    Member

    Ideally, to have @brianf chime in and see what options we have here

  6. ariel-miculas commented on Aug 18, 2025

    @ariel-miculas
    ariel-miculas
    on Aug 18, 2025
    ContributorAuthor

    @brianf Any chance you could clarify the index publishing process and how we could update the maven-indexer that's doing the publishing?

  7. slachiewicz
    added a commit that references this issue on Sep 25, 2026

    Fix missing sha1 digests from the maven index (#693)

    e7c5245

  8. slachiewicz commented on Sep 25, 2026

    @slachiewicz
    slachiewicz
    on Sep 25, 2026
    Member

    Fixed in #693 for the indexer itself: .sha256/.sha512 sidecar files were being indexed as the main artifact and overwrote its record, dropping the SHA-1. The index published on Central is produced by Sonatype, so it will only pick this up once they move to a release that includes the fix.

  9. slachiewicz
    closed this as completedon Sep 25, 2026
  10. slachiewicz
    added a commit that references this issue on Sep 26, 2026

    Never index checksum or signature files as artifacts

    ae2b36c
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions