PIANO performance data parser

[lrobion]

This PR is part of #158 and implements a converter of PIANO performance outputs to AEIC .toml files that follow existing conventions + carry additional information not contained in .PTF files (most notably cruise performance is a function of (FL, mass, mach).

This PR implements a stub of the PianoPerformanceModel in performance/piano.py based on data stored in the .toml but does not implement the actual performance evaluation method evaluate_impl(), this is for a future PR.

PIANO output description

PIANO outputs with dummy data have been added to tests/data/piano/ and are representative of real outputs. Data is stored in 3 files, one for cruise, one for climb and one for descend. I recommend taking a look at these files first to make sense of the rest of the description here.

Climb

PIANO climb outputs assume a fixed speed schedule with PIANO-default values: CAS=250 kts when altitude < FL100, aircraft specific constant CAS for FL100 < altitude < crossover_altitude, and an aircraft specific constant Mach for altitude > crossover_altitude.

The table contains the following columns: Altitude, time, distance, cumulative fuel burn, net thrust per engine, rate of climb, drag.

This can be computed for a range of starting masses, such that there are multiple climb tables in a single climb file. However climb table headers which contain (starting mass, speed schedule...) only appear above a table when the cruise has reached its target altitude (which it does not always reach):

• The speed schedule is independent of starting mass so if there is at least one climb table header in the file, we can infer the schedule from the file directly.
• We cannot do this for starting mass as it varies from climb to clim, such that the users need to provide this data in the PianoOverrides dataclass

Cruise

PIANO computes a performance sweep at cruise as a function of (FL, mass, mach) and provides: TAS, CAS, Drag, Max. Cruise Rating, L/D, FuelFlow, SFC, Specific Air Range, Max Climb Rating / engine, Rate of climb at Max climb rating for a fixed Mach, Rate of Climb at max climb rating for a fixed CAS (+ some others, I am only storing these for now).

On top of these "parameter sweep" points, it also provides performance at notable operating points: max specific air range, 99% specific air range, maxLim. These points are appended at the end of a mach sweep: their (FL, mass, mach) coordinate may be repeated from the sweep but there is no guarantee that the values associated are the exact same as the ones from the sweep (probably some rounding in the PIANO output). Currently I overwrite this with data from the sweep but we could do the opposite.

Descent

Very similar to climb, except that they in my experience always succeed such that the descent table header is always present and can be parsed to get the speed schedule and mass, simplifying the processing (no need for PianoOverrides. PIANO also provides the altitude after which thrust is set to idle, this is also recorded in the .toml file if we ever have use for it.

PIANO.toml schema

A PIANO based .toml file looks like this (dummy numbers again):

# Performance model type (one of: legacy, bada, tasopt, piano).
model_type = "piano"

# ==============================================================================
#
#  COMMON FIELDS
#
# Fields common to all performance model types.

aircraft_name = "piano_aircraft, piano_engine"
aircraft_class = "narrow" # wide, narrow, small, freight
ISA_offset = 0
maximum_altitude_ft = 41000
maximum_payload_kg = 200
number_of_engines = 2 # Number of engines

# ------------------------------------------------------------------------------
#
# Speed data
#

[speeds.climb]
cas_low = 120
cas_high = 125
mach = 0.75
crossover_altitude_m = 10000.0

[speeds.descent]
cas_low = 120
cas_high = 125
mach = 0.75
crossover_altitude_m = 10000.0

# ------------------------------------------------------------------------------
#
# LTO data
#

[LTO_performance]
source = "EDB"
ICAO_UID = "01P11CM121" # Add UID for EDB data
rated_thrust = 11000.0

[LTO_performance.mode_data.idle]
thrust_frac = 0.07
fuel_kgs =0.01
EI_NOx = 0.01
EI_HC = 0.01
EI_CO = 0.01

[LTO_performance.mode_data.approach]
thrust_frac = 0.3
fuel_kgs = 0.01
EI_NOx = 0.01
EI_HC = 0.01
EI_CO = 0.01

[LTO_performance.mode_data.climb]
thrust_frac = 0.85
fuel_kgs = 0.01
EI_NOx = 0.01
EI_HC = 0.01
EI_CO =0.01

[LTO_performance.mode_data.takeoff]
thrust_frac = 1.0
fuel_kgs = 0.01
EI_NOx = 0.01
EI_HC = 0.01
EI_CO = 0.01


# ==============================================================================
#
#  MODEL-TYPE SPECIFIC FIELDS
#

# ------------------------------------------------------------------------------
#
# Performance table data.
#

[climb_flight_performance]
cols = [
  "fl",  # Flight levels
  "mass",  # kg
  "tas",  # m/s
  "rocd",  # m/s
  "fuel_flow",  # kg/s - REQUIRED; OUTPUT COLUMN
  "time",  # s - cumulative from start of phase
  "distance",  # m - cumulative from start of phase
  "burn",  # kg - cumulative from start of phase
  "fn_per_engine",  # N - net thrust per engine
  "drag"  # N
]

data = [
  [0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01],
...
]

[cruise_flight_performance]
cols = [
  "fl",  # Flight levels
  "mass",  # kg
  "mach",  # Mach number
  "tas",  # m/s
  "cas",  # m/s
  "rocd",  # m/s
  "fuel_flow",  # kg/s - REQUIRED; OUTPUT COLUMN
  "drag",  # N
  "mcr_pct",  # percent of maximum cruise thrust
  "lift_to_drag",  # Lift-to-drag ratio
  "sfc",  # kg/(N.s) - specific fuel consumption
  "sar",  # m/kg - specific air range
  "mcl_avail_per_engine",  # N - maximum climb thrust available per engine
  "rocd_mcl_fix_mach",  # m/s - at maximum climb thrust, fixed Mach
  "rocd_mcl_fix_cas"  # m/s - at maximum climb thrust, fixed CAS
]

data = [
  [ 0.01,  0.01, 0.01, 0.01,0.01,  0.01, 0.01,  0.01, 0.01, 0.01, 0.01, 0.01, 0.01, 0.01],
# Repeated (N FL, M cruise masses, K Machs)
]

[descent_flight_performance]
cols = [
  "fl",  # Flight levels
  "mass",  # kg
  "tas",  # m/s
  "rocd",  # m/s
  "fuel_flow",  # kg/s - REQUIRED; OUTPUT COLUMN
  "time",  # s - cumulative from start of phase
  "distance",  # m - cumulative from start of phase
  "burn",  # kg - cumulative from start of phase
  "fn_per_engine"  # N - net thrust per engine
]

data = [
  [ 0.01, 0.01,  0.01, 0.01, 0.01, 0.01, 0.01, 0.01],
...
]

[cruise_reference_mach]
cols = [
  "fl",  # Flight levels
  "mass",  # kg
  "max_sar",  # Mach at maximum specific air range
  "sar_99",  # Mach at 99% of maximum specific air range
  "max_lim"  # Maximum limiting Mach
]

data = [
  [ 0.01,0.01, 0.01, 0.01, 0.01],
# Repeated (N FL, M cruise masses)
]

[descent_idle_thrust]
cols = [
  "mass",  # kg
  "idle_thrust_altitude"  # m - altitude below which idle thrust is used
]

data = [
  [ 0.01, 0.01],
# Repeated N descent masses
]

Code refactors

• Moved shared .toml writer functions from the legacy write code in commands/make_performance_models.py to commands/_performance_models_toml.py and only kept legacy specific and PIANO specific CLI entry points in commands/make_performance_models.py
• Split the existing PerformanceTableInput data structure into a table specific class, and a performance table class. The new TableInput dataclass (in performance/types.py) accepts empty tables and is responsible for normalizing columns, checking shape consistency and provides a column accessor. Now PerformanceTableInput inherits from it and only performs validation on the columns making sure they contain the minimum performance data required.

These refactors do not change the behavior of the existing CLI command for legacy models as tested with the reference tests/data/performance/legacy_golden.toml generated with the old code, and tests/test_make_performance_model.py which verifies that the new code leads to a byte identical file.

New code to support PIANO

Mainly in parsers/piano_reader.py and tests/.

More details to come

In progress

Code:

• Update doc
• Cleanup Claude comments...
• Evaluate test coverage

Modeling decisions:

• Get feedback on if this is the data we need / format etc...
• Simplify climb speed schedule parsing logic: assume that a PIANO climb output file will contain at least 1 valid header block describing the speed schedule (leaving PianoOverrides to only include climb masses) Deferred to later when the database is updated (current code would still work then anyway)
• Discuss cruise mach Speed value for a PIANO which does not have a set cruise mach
• Operating empty weight handling: this field is not written to toml file because BADA assumes oew = min_mass / 1.2, whereas PIANO can provide the exact number (though it is not in the outputs we parse)
• Decide what to do with MCR and MCL fields that may be unset by PIANO ... but the rest of the row is valid

[ian-ross]

Just took a quick look at this. I know it's only a draft and you're likely to make changes, but I wanted to head off some stuff that you (or Claude) are doing that's not good.

[ian-ross]

Oh yeah, to add to my review comments: in principle, this looks great, and it's a good first step to adding more and better performance models. We just need to wrangle the Claude.

[lrobion]

I've refactored the code such that:

• commands/make_performance_models.py only handles CLI and basic argument parsing before calling a performance model builder, and a performance model writer
• performance/model_builder.py now contains the code to create a performance model object build_*_performance_model(...) -> PerformanceModel, and a writer write_performance_model(output_path, model) -> None
• Added a schema description for each PIANO file type in performance/parsers/piano_reader.py, the placement feels a bit awkward but I was not sure where else to put it. This file is largest because it has to deal with all the PIANO specific parsing... I am not sure if I should split it up in 3 (climb, cruise, descent) but then I also need to share helper functions so it gets a little messy.
• conftest.py now has fixtures for PIANO data and LTO data, I had to use a functools.cache to get the caching working as session- or module-level scoping of the fixtures did not work with the default_config setup
• Rewrote all tests to test the builder outputs to avoid any invocation of click which is fine now that the CLI does very light work before passing it to the functions we do test. I think this setup is definitely an improvement over what I had before

On top of updating the doc, there are couple of decisions left which are more science related:

1. AEIC .toml files do not store the operating empty weight of the aircraft even though this is required for simulations. We get away with this because the LegacyPerformanceModel does the BADA assumption of take the min_tabuled_mass / 1.2 to estimated OEW. This does not work for PIANO because we don't know how BADA chooses that mass and we can do better anyway. PIANO can provide OEW directly, thought it is not in the output files we parse. Currently it can be passed as a CLI argument to aeic make-performance-model ... piano but did not make it a required argument because I was confused by this. Moving forwards I think we need to store OEW in the .toml, next to maximum_payload_kg and have the reader load it from there instead of the BADA factor.
2. The Speeds class requires a SpeedData value for its .cruise member. For tabulated data with Mach sweeps, I think this does not make sense anymore as there is no set cruise speed (unlike BADA). I went around this by allowing .cruise to be None but this feels like a weird fix to a deeper conceptual issue that the Speeds class models. I've added the option to set a cruise speed manually via a CLI argument but I do not see why we would want to use it. I'd like other people's opinions on how to work around / redesign this in the future @WyattGiroux @aditeyashukla @ShreyaSharma2023
3. Speed schedule headers for climbs are not always present in PIANO which is why there are 4 optional CLI flags to pass them manually. This is a bit annoying and if we are willing to bake in the assumption that these headers always exist, this could simplify some code. On the PIANO data end, @ShreyaSharma2023 and I could fix this by regenerating some outputs to ensure the schedules are present.

I think point 1. is the only functionality I would want decided/finalized before considering this to be done (barring code improvements). Point 2. and 3. could wait.

[ian-ross]

This is looking good. There are only a few things remaining:

1. As you kind of suggested, I think it would be good to split the AEIC.parsers.piano_reader module. You can just make a src/AEIC/parsers/piano_reader directory and treat it as a normal module, with whatever files you want in there and a __init__.py file. I would also move the schema description out of the code into documentation and refer to it from the code. (You could put the documentation either under "Available parsers" or, probably better, in the performance modeling section of the docs.)
2. I don't like the functools.cache in conftest.py. Can you show me what was going wrong with the default_config fixture when you used scope="module" in a normal fixture for the Piano data? If that was breaking, it's really a bug in the configuration setup, which is a fragile kind of hack, but needs to be made to work with normal Pytest approaches to things. If you can show me what was happening, I'll see if I can figure it out.
3. Do add operating empty weight to the TOML files. Make it an optional field. Then the LegacyPerformanceModel can use its current rule to replace the value, and other performance models can raise an error if it's missing. That has always been a wart that I had in mind to handle better.
4. The speed stuff is a little weird and is very BADA-focused, so that side of things should be redesigned to cover the newer performance model types. I think it's OK to have a kind of "union schema" for the performance model TOML files, and for individual model classes to check that the fields they need are there (or not there) when they process the TOML inputs. As long as most performance models are effectively table-based, I think it makes sense to share as much code between them as possible, but we shouldn't be rigid about it.

[lrobion]

Thanks for the feedback, I'll make the changes.

For the failing test, all that's needed is to add the scoping to the pytest fixtures. Now the cached function is called only when the fixture is instantiated at the module or session scope (removing the cache decorator / moving the cached function body into the fixture does not change anything).

# conftest.py
@pytest.fixture(scope="module")
def lto() -> LTOPerformanceInput: ...

@pytest.fixture(scope="session")
def piano_data() -> PianoData: ...

Setting either of these fixtures to scope="module" or scope="session" leads the tests that call them to fail with:

# Example with lto() set with scope="module"
ERROR tests/test_legacy_performance_model.py::test_built_model_round_trips_through_the_loader - ValueError: AEIC configuration is not set
ERROR tests/test_make_performance_model.py::test_legacy_output_byte_identical - ValueError: AEIC configuration is not set
ERROR tests/test_model_writer.py::test_table_sections_are_written_in_spec_order - ValueError: AEIC configuration is not set
ERROR tests/test_model_writer.py::test_empty_table_writes_its_columns_and_no_rows - ValueError: AEIC configuration is not set
ERROR tests/test_model_writer.py::test_none_table_omits_its_whole_section - ValueError: AEIC configuration is not set
ERROR tests/test_model_writer.py::test_apu_name_omitted_when_unset - ValueError: AEIC configuration is not set
ERROR tests/test_model_writer.py::test_unset_speed_phase_is_omitted - ValueError: AEIC configuration is not set
ERROR tests/test_model_writer.py::test_isa_offset_round_trips - ValueError: AEIC configuration is not set
ERROR tests/test_model_writer.py::test_model_type_without_a_write_spec_raises - ValueError: AEIC configuration is not set
ERROR tests/test_piano_performance_model.py::test_written_file_loads_as_a_piano_model - ValueError: AEIC configuration is not set
ERROR tests/test_piano_performance_model.py::test_every_emitted_column_survives_the_round_trip - ValueError: AEIC configuration is not set
ERROR tests/test_piano_performance_model.py::test_empty_mass_requires_an_operating_empty_mass - ValueError: AEIC configuration is not set
ERROR tests/test_piano_performance_model.py::test_cruise_speeds_written_only_with_cruise_mach - ValueError: AEIC configuration is not set
ERROR tests/test_piano_performance_model.py::test_name_and_altitude_overrides - ValueError: AEIC configuration is not set
ERROR tests/test_piano_performance_model.py::test_isa_offset_comes_from_the_exports - ValueError: AEIC configuration is not set

I think it's because fixture are created from largest scope to smallest scope such that the session or module level fixtures get created before the default_config (test-level). Then there is no AEIC config available when they are called.

Without caching, the tests (with --ignore="tests/test_storage.py" because that one varies more) take ~9.3s, with caching there are down to 8.7s. This is not very significant but it'd be nice to reuse the fixtures as you said.

[lrobion]

I think the PR is ready for review. We also still need to setup the develop branch.

I have:

• Split the PIANO reader module into 5 files
• Created / updated documentation on performance models, the PIANO parser and its expected schema (I split parser.md into a BADA, an LTO and a PIANO page)
• Always set the cruise altitude SpeedData to None for PIANO
• Added the operating empty weight to TOML files (PIANO requires it, Legacy warn that the value is ignored)

I have marked two things as questions in the doc (docs/parsers/piano_reader.md):

1. PIANO does not provide TAS at every point for climb/descent but it usually gives you the speed schedule it flies. We recover TAS by converting CAS to TAS assuming an ISA. However when PIANO does not provide the speed schedule, the user has to provide it in the CLI. In that case I think it is useful to sanity check the consistency of the user input with an estimated TAS from ground speed and ROC. I've put a tolerance on the median deviations to be less than 5% but I am not sure how useful that is, it only catches gross inconsistencies (bad unit, wrong column...) or speed schedule parsing errors.
2. PIANO does not provide climb/descent fuel flow directly, and instead provides the cumulative fuel burn at different timesteps. I estimate fuel burn as the backward difference with the previous timestep / timestep size. This does not work for the first time step, so I use a forward difference instead but I am not 100% if that makes sense either here.