Stop variables export/import from silently corrupting values - #70944
Merged
potiuk merged 2 commits intoAug 18, 2026
Conversation
`airflow variables export` writes a bare value for any variable without a
description, while `airflow variables import` treats every dict carrying a
"value" key as a {"value": ..., "description": ...} envelope. A variable whose
own value happens to have that shape is therefore unwrapped on the way back in:
its value is replaced by the inner "value" and a description is invented from
the inner "description". Nothing warns the operator, so a round-trip through a
file - the documented way to migrate variables between environments - quietly
rewrites their data.
Export also decodes the stored JSON before writing it out, which erases the
difference between a variable holding raw text and one holding a JSON-encoded
string. Re-importing flattens the latter, and any Dag reading it with
deserialize_json=True starts failing on a value that is no longer valid JSON.
Import's reconstruction is deterministic - strings are stored verbatim,
everything else is JSON-encoded, and one envelope layer is unwrapped - so export
alone can be made lossless. Fixing it there leaves import untouched and keeps
hand-written import files working exactly as before.
Eason09053360
marked this pull request as ready for review
August 2, 2026 16:59
Eason09053360
requested review from
bugraoz93,
dheerajturaga,
henry3260 and
potiuk
as code owners
August 2, 2026 16:59
henry3260
reviewed
Aug 2, 2026
Eason09053360
force-pushed
the
fix-variable-export-import-roundtrip
branch
from
August 2, 2026 17:20
92d0c1b to
0aa93cb
Compare
henry3260
reviewed
Aug 4, 2026
henry3260
left a comment
henry3260
left a comment
Contributor
There was a problem hiding this comment.
The export change looks like it regresses the UI import path for the same two cases this PR fixes for the CLI, since the CLI and UI use different unwrap rules — as far as I can tell there's no export-only change that's correct for both. The PR notes the UI limitation as follow-up, but given export now depends on the unwrap rules, I'd lean towards fixing the UI side in this PR rather than deferring it.
I'm not familiar with the UI part though, so I'd like to hear your thoughts. cc @bbovenzi
henry3260
reviewed
Aug 4, 2026
henry3260
reviewed
Aug 4, 2026
potiuk
approved these changes
Aug 18, 2026
Contributor
Backport successfully created: v3-3-testNote: As of Merging PRs targeted for Airflow 3.X In matter of doubt please ask in #release-management Slack channel.
|
github-actions Bot
pushed a commit
to aws-mwaa/upstream-to-airflow
that referenced
this pull request
Aug 18, 2026
…ues (apache#70944) * Stop variables export/import from silently corrupting values `airflow variables export` writes a bare value for any variable without a description, while `airflow variables import` treats every dict carrying a "value" key as a {"value": ..., "description": ...} envelope. A variable whose own value happens to have that shape is therefore unwrapped on the way back in: its value is replaced by the inner "value" and a description is invented from the inner "description". Nothing warns the operator, so a round-trip through a file - the documented way to migrate variables between environments - quietly rewrites their data. Export also decodes the stored JSON before writing it out, which erases the difference between a variable holding raw text and one holding a JSON-encoded string. Re-importing flattens the latter, and any Dag reading it with deserialize_json=True starts failing on a value that is no longer valid JSON. Import's reconstruction is deterministic - strings are stored verbatim, everything else is JSON-encoded, and one envelope layer is unwrapped - so export alone can be made lossless. Fixing it there leaves import untouched and keeps hand-written import files working exactly as before. * apply suggestion --------- (cherry picked from commit f549b34) Co-authored-by: Y-C <easoneason0905@gmail.com> Co-authored-by: Eason09053360 <185830721+Eason09053360@users.noreply.github.com>
aws-airflow-bot
pushed a commit
to aws-mwaa/upstream-to-airflow
that referenced
this pull request
Aug 18, 2026
…ues (apache#70944) * Stop variables export/import from silently corrupting values `airflow variables export` writes a bare value for any variable without a description, while `airflow variables import` treats every dict carrying a "value" key as a {"value": ..., "description": ...} envelope. A variable whose own value happens to have that shape is therefore unwrapped on the way back in: its value is replaced by the inner "value" and a description is invented from the inner "description". Nothing warns the operator, so a round-trip through a file - the documented way to migrate variables between environments - quietly rewrites their data. Export also decodes the stored JSON before writing it out, which erases the difference between a variable holding raw text and one holding a JSON-encoded string. Re-importing flattens the latter, and any Dag reading it with deserialize_json=True starts failing on a value that is no longer valid JSON. Import's reconstruction is deterministic - strings are stored verbatim, everything else is JSON-encoded, and one envelope layer is unwrapped - so export alone can be made lossless. Fixing it there leaves import untouched and keeps hand-written import files working exactly as before. * apply suggestion --------- (cherry picked from commit f549b34) Co-authored-by: Y-C <easoneason0905@gmail.com> Co-authored-by: Eason09053360 <185830721+Eason09053360@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
airflow variables exportfollowed byairflow variables importcan silently rewrite a variable's value and invent a description for it.Export writes a bare value when a variable has no description, while import treats any dict carrying a
"value"key as a{"value": ..., "description": ...}envelope — nothing distinguishes the envelope from a value that happens to look like one. Export also decodes the stored JSON, losing the difference between raw text and a JSON-encoded string.Round-tripping through a file on
main:{"value": "a", "description": "b"}{"value": "a", "description": "b"}a, description →b{"value": 1, "other": 2}{"value": 1, "other": 2}1,otherdropped"hello"hellohello— no longer valid JSON, sodeserialize_json=Truestarts raisingImport's reconstruction is deterministic — strings are stored verbatim, everything else is JSON-encoded, and one envelope layer is unwrapped — so export alone can be made lossless, leaving hand-written import files behaving exactly as before. Export now emits the stored form when the decoded value is a string, and wraps values that are themselves envelope-shaped so import's unwrap consumes our envelope rather than the user's data.
Exported files are unchanged for values that were never ambiguous. One visible change worth flagging: a variable stored as a JSON-encoded string (
Variable.set(k, "text", serialize_json=True)) now exports as"\"text\""rather than"text". The old output could not be re-imported without corruption, so there is no lossless way to keep it.Left for follow-up PRs: import still truncates a hand-written
{"value": 1, "other": 2}to1, and the UI import path does not unwrap the envelope at all.Was generative AI tooling used to co-author this PR?
Generated-by: Claude Code (Opus 5) following the guidelines