RANGER-5680: Make Solr optional in docker so Ranger can run without it by paras200 · Pull Request #1058 · apache/ranger

paras200 · 2026-07-09T15:35:43Z

What changes were proposed in this pull request?

RANGER-5680 introduces a RANGER_AUDIT_STORE environment variable that selects the audit backend without requiring manual edits to install.properties, mirroring the existing RANGER_DB_TYPE pattern for database selection.

Audit store selection

export RANGER_AUDIT_STORE=opensearch # default — no change needed
export RANGER_AUDIT_STORE=solr # opt-in to Solr

ranger.sh rewrites the active audit block in install.properties at container startup based on RANGER_AUDIT_STORE, so setup.sh picks up the correct configuration. OpenSearch is now the default; Solr remains fully supported via the env-var.

Changes

.env: add RANGER_AUDIT_STORE=opensearch as the default
docker-compose.ranger.yml: pass RANGER_AUDIT_STORE into the ranger container environment
scripts/admin/ranger.sh: rewrite the active audit block in install.properties before setup.sh runs when RANGER_AUDIT_STORE=solr is set (OpenSearch is the no-op default, matching the properties file default)
ranger-admin-install-*.properties (postgres/mysql/oracle/sqlserver): flip default active audit block from Solr to OpenSearch; fix inconsistent dangling active elasticsearch param lines in mysql/oracle/sqlserver; add missing OpenSearch block to sqlserver
docker-compose.ranger-ozone.yml: add keytab provisioning wait to datanode/scm/om startup commands so they block on /etc/keytabs/.provisioned before attempting Kerberos login — fixes a race where Ozone services started before the KDC had finished generating keytabs
CI: switch services-docker-build and plugins-docker-build from docker-compose.ranger-solr.yml to docker-compose.ranger-opensearch.yml, consistent with OpenSearch being the default; update container checklists accordingly

How was this patch tested?

docker compose config validated across all documented stack combinations (core, pdp, ozone, all-containers, audit stacks, solr opt-in) — all resolve cleanly
Full end-to-end OpenSearch audit pipeline tested locally: OpenSearch index bootstrapped by Ranger Admin on startup, audit posted via ingestor → Kafka → dispatcher → indexed in OpenSearch, visible in Ranger Admin UI (audit_store=opensearch active, no Solr container running)
RANGER_AUDIT_STORE=solr path verified: ranger.sh rewrites install.properties correctly and ranger.audit.source.type=solr is written by setup.sh

ramackri · 2026-07-27T05:29:33Z

@paras200
Check build is getting build it seems docker container start up issues

paras200 · 2026-07-27T11:24:58Z

Thanks @ramackri this is a draft PR, will review once I am marking it ready.
you are right, with removal of solr, this is the error - Error response from daemon: No such container: ranger-solr

ramackri · 2026-08-06T13:34:13Z

LGTM

kumaab · 2026-08-06T16:27:52Z

Thanks @paras200 for the patch. Few questions:

Can we see if the official opensearch image can be used ? https://hub.docker.com/r/opensearchproject/opensearch, instead of building our own image of opensearch in Dockerfile.ranger-opensearch, similar to how it is done for Solr currently.
OPENSEARCH_VERSION=1.3.19 version in .env is EOL, why not use OpenSearch (3.x) ?

Thanks

CC: @mneethiraj

paras200 · 2026-08-07T10:04:16Z

Thanks @kumaab for the review!
Done — switched to opensearchproject/opensearch:2.19.6 directly; custom Dockerfile and startup scripts removed.

kumaab · 2026-08-17T21:33:34Z


 # 1. Start OpenSearch first (Ranger Admin's bootstrapper needs it on startup)
 docker compose -f docker-compose.ranger.yml -f docker-compose.ranger-opensearch.yml \
  -f docker-compose.ranger-kafka.yml -f docker-compose.ranger-hadoop.yml \


I'm trying to understand why do we need hadoop container here? Opensearch requires kafka and audit server only right ?

kumaab · 2026-08-17T21:35:09Z

+# For Solr: export RANGER_AUDIT_STORE=solr and replace -f docker-compose.ranger-opensearch.yml with -f docker-compose.ranger-solr.yml below.

-docker compose -f docker-compose.ranger.yml -f docker-compose.ranger-solr.yml -f docker-compose.ranger-usersync.yml -f docker-compose.ranger-tagsync.yml -f docker-compose.ranger-pdp.yml -f docker-compose.ranger-kms.yml up -d
+docker compose -f docker-compose.ranger.yml -f docker-compose.ranger-opensearch.yml -f docker-compose.ranger-usersync.yml -f docker-compose.ranger-tagsync.yml -f docker-compose.ranger-pdp.yml -f docker-compose.ranger-kms.yml up -d


ranger-opensearch ==> ranger-${RANGER_AUDIT_STORE}

Please update other references as well.

kumaab · 2026-08-17T21:38:35Z


-For **existing Solr-based installs**, switch stores by setting `audit_store=opensearch` (and the
-`audit_opensearch_*` properties) in install.properties and restarting Ranger Admin.
+For **existing Solr-based installs**, set `RANGER_AUDIT_STORE=opensearch`, recreate the Ranger


typo: RANGER_AUDIT_STORE=opensearch ==> RANGER_AUDIT_STORE=solr

this paragraph is to describe migration to OpenSearch for existing Solr-based installs (set RANGER_AUDIT_STORE=opensearch). Re wrote this sentence to make it clearer.

kumaab · 2026-08-17T21:39:59Z

 docker compose -f docker-compose.ranger.yml -f docker-compose.ranger-opensearch.yml up -d
 ~~~

 #### OpenSearch audit flow (replace Solr for access audits)


Suggestion: (replace Solr for access audits) may be removed.

kumaab · 2026-08-17T22:13:23Z

      - ./scripts/ozone/docker-config
    # Do not pass KERBEROS_ENABLED: Ozone libexec/entrypoint.sh waits for krb5:8081.
-    command: bash -c "unset KERBEROS_ENABLED; source /opt/hadoop/ranger-ozone-plugin/ranger-ozone-setup.sh && (test -f /data/metadata/om/current/VERSION || /opt/hadoop/bin/ozone om --init) && exec /opt/hadoop/bin/ozone om"
+    command: bash -c "until [ -f /etc/keytabs/.provisioned ]; do echo 'waiting for keytabs...'; sleep 3; done && unset KERBEROS_ENABLED; source /opt/hadoop/ranger-ozone-plugin/ranger-ozone-setup.sh && (test -f /data/metadata/om/current/VERSION || /opt/hadoop/bin/ozone om --init) && exec /opt/hadoop/bin/ozone om"


Tried bringing this container in local setup. This change is causing the om container to print waiting for keytabs..., reverting this change causes the container to proceed further. please check.

…e in docker - Introduce RANGER_AUDIT_STORE env-var (opensearch | solr; default: opensearch) so the docker stack can switch audit backends without editing install.properties - Switch docker-compose.ranger-opensearch.yml to use the official opensearchproject/opensearch image directly (version 3.7.0); remove the custom Dockerfile.ranger-opensearch and its associated setup scripts - Update all four ranger-admin-install-*.properties files to activate the opensearch audit block by default (audit_store=opensearch) while keeping the solr block available via RANGER_AUDIT_STORE=solr - Add AuditOpenSearchDispatcher and dispatcher-common unit tests; replace sun.misc.Unsafe constructor bypass with package-private test constructors - Fix docker-compose.ranger-ozone.yml: replace broken .provisioned sentinel wait loop with ozone-service-start.sh for all three Ozone services (datanode, scm, om), add om volume mount and restart: on-failure:3 - Update CI workflow to smoke-test the OpenSearch audit stack instead of Solr - Update README with OpenSearch audit flow documentation

kumaab · 2026-08-18T20:58:28Z

@@ -163,7 +162,7 @@ else
      extractOzoneIfNeeded
    elif [[ $arg == 'opensearch' ]]


Let's remove this if condition, for opensearch - download-archives.sh is not applicable.

kumaab · 2026-08-18T20:59:30Z

   ~~~
   chmod +x download-archives.sh
   # use a subset of the below to download specific services
   ./download-archives.sh hadoop hive hbase kafka knox ozone opensearch


opensearch should be removed here.

kumaab · 2026-08-18T22:27:12Z

 ~~~
 #### Bring up ozone containers

 ##### Ozone action-matcher feature flag


This section is not really relevant here, could you please help this move out to a different readme in the same directory, may be something like OZONE-ACTION-MATCHER.md, ref: 37b6e02#diff-da6584afb9b693c71ad5427c1a5e1e0720e306e0fbfb664f9fb79c7ce44c29fa

kumaab · 2026-08-18T22:28:38Z

+Set `RANGER_AUDIT_STORE=opensearch` — no manual `install.properties` edits needed.

 ~~~
 # Prerequisites: build Ranger artifacts (admin, audit ingestor/dispatcher, ...) and download archives


L178-182 can be skipped, they are already covered earlier in the setup process.

paras200 force-pushed the RANGER-5680 branch 4 times, most recently from 1229d50 to 3393f80 Compare August 5, 2026 13:06

paras200 marked this pull request as ready for review August 5, 2026 13:11

paras200 force-pushed the RANGER-5680 branch 2 times, most recently from 987a176 to ccc8ffb Compare August 6, 2026 11:32

paras200 force-pushed the RANGER-5680 branch 3 times, most recently from c7e3364 to 0be020b Compare August 7, 2026 10:01

paras200 force-pushed the RANGER-5680 branch from 0be020b to 5437319 Compare August 17, 2026 09:35

kumaab reviewed Aug 17, 2026

View reviewed changes

paras200 force-pushed the RANGER-5680 branch 2 times, most recently from 0b15436 to f8271fd Compare August 18, 2026 08:49

paras200 force-pushed the RANGER-5680 branch from 127a16a to 9288d95 Compare August 18, 2026 15:28

kumaab reviewed Aug 18, 2026

View reviewed changes

		@@ -163,7 +162,7 @@ else
		extractOzoneIfNeeded
		elif [[ $arg == 'opensearch' ]]

Uh oh!

Conversation

paras200 commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ramackri commented Jul 27, 2026

Uh oh!

paras200 commented Jul 27, 2026

Uh oh!

ramackri commented Aug 6, 2026

Uh oh!

kumaab commented Aug 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

paras200 commented Aug 7, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

paras200 commented Jul 9, 2026 •

edited

Loading

kumaab commented Aug 6, 2026 •

edited

Loading